Last week some of The Instillery team were living it up in Vegas while attending the world’s largest public cloud conference. Referred to by some as “Cloud Heaven” the AWS re:Invent conference attracts over 24,000 people from around the world and has over 400 breakout sessions, bootcamps, certification opportunities and training events. And what a week it was. The team have come back fizzing about the next tranche of exciting new technologies, managed services and features that were announced at the re:Invent conference by Amazon CTO Werner Vogels. Here’s the skinny on those key AWS re:Invent announcements you absolutely need to know about. Go grab yourself a coffee and buckle in.
1. AWS SNOWMOBILE
AWS Snowmobile is an Exabyte-scale data transfer service used to move massive amounts of data to AWS without the concern of the toll over your direct connect, or internet service provider. You can transfer up to 100PB per Snowmobile. When this literal monster truck made predominantly of a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck hit the stage, the audience stood in applause and amazement.
The AWS Snowmobile makes it easy to move massive volumes of data to the cloud, including video libraries, image repositories, or even a complete data centre migration. Transferring data with Snowmobile is secure, fast and cost-effective. After an initial assessment and confirmation of data quality, a Snowmobile will be transported to your data centre and AWS personnel will configure it for you so it can be accessed as a network storage target. When the Snowmobile is on site, AWS personnel will work with your team to connect a removable, high-speed network switch from Snowmobile to your local network and you can begin your high-speed data transfer from any number of sources within your data centre to the Snowmobile. After data is loaded, the Snowmobile is driven back to AWS where data is imported into Amazon S3 or Amazon Glacier. Snowmobile uses multiple layers of security designed to protect your data including dedicated security personnel, GPS tracking, alarm monitoring, 24/7 video surveillance, and an optional escort security vehicle while in transit. Watch the Snowmobile in action here.
2. AWS GLUE
AWS Glue is a fully managed data catalogue and Extract, Transform and Load service. Glue and several other new AWS offerings are intended to round out the AWS data-architecture offering. When used in conjunction with existing products for data analytics and warehousing, which Vogels spent considerable time summarising as a “comprehensive data architecture on AWS”.
Glue allows “building a data catalogue, so you can point to various data sources — any JDBC (Java Database Connectivity API) database, even if it’s on-premises,” says Vogels. “It reads the metadata. Glue then allows transforming data and prepare it into a format that your analytics engine needs. And it allows scheduling and running jobs. If data changes, it will make adjustments.”
3. AWS BATCH
AWS introduced "Batch", which Vogels described as a fully managed batch-processing service that dynamically handles batch processing at any scale. It takes advantage automatically of spot pricing, which can save considerably on AWS spending. “You longer have to worry about this pain point in very large data processing,” he said.
4. AMAZON PINPOINT
Amazon Pinpoint is a new mobile marketing analytics feature within Amazon Web Services that helps app makers / SaaS providers run targeted push notification campaigns. Pinpoint enables real-time analytics with dashboards for analysing user engagement, monetisation, user demographics, custom events, and funnels so you can understand how users engage with your application. You can analyse and understand your user data by drilling down based on the segments you’ve defined, segmentation attributes, or time.
With Pinpoint, you can define target segments from a variety of different data sources. You can identify target segments from app user data collected in Pinpoint. You can build custom target segments from user data collected in other AWS services such as Amazon S3 and Amazon Redshift, and import target user segments from third party sources such as Salesforce via S3. You also only pay for the number of users you target or collect events from, the messages you send, and events you collect, so you can start small and scale as your application grows.
5. AWS SHIELD (next level Web Application Firewall)
AWS ‘Shield’ was also and is a a managed Distributed Denial of Service (DDoS) protection service that safeguards web apps running on AWS – protecting websites from crippling online attacks such as those experienced in recent months by Netflix, Twitter and Krebsonsecurity. AWS Shield provides always-on detection and automatic inline mitigations that minimise application downtime and latency, so there is no need to engage AWS Support to benefit from DDoS protection. There are two tiers of AWS Shield - Standard and Advanced.
All AWS customers benefit from the automatic protections of AWS Shield Standard, at no additional charge. AWS Shield Standard defends against most common, frequently occurring network and transport layer DDoS attacks that target your website or SaaS.For higher levels of protection against attacks targeting your web applications running on Elastic Load Balancing (ELB), Amazon CloudFront, and Amazon Route 53 resources, you can subscribe to AWS Shield Advanced. In addition to the common network and transport layer protections that come with Standard, AWS Shield Advanced provides additional detection and mitigation against large and sophisticated DDoS attacks, near real-time visibility into attacks, and integration with AWS WAF, a web application firewall. AWS Shield Advanced also gives you access to the AWS DDoS Response Team (DRT) and protection against DDoS related spikes in your ELB, CloudFront or Route 53 charges.
6. AWS BLOX
AWS Blox are a collection of open-source projects for container management and orchestration on the Amazon ECS container service. It lets developers build schedulers and integrate third-party schedulers on top of AWS ECS, while using ECS to manage and scale clusters. It was announced during the keynote that Netflix (one of AWS largest clients) is already this service across their business.
7. LAMBDA@EDGE
With Lambda@Edge, developers can process edge locations without having to go back to the original source. These functions can inspect HTTP requests and take actions on them. By literally cutting the physical distance that information must flow through in order to execute parts of an application, developers will be able to improve the performance of their applications running with AWS Lambda.
Part and parcel of this announcement was Amazon stating it would bring its Lambda compute tools that allow it to work on IoT devices. Traditionally, developers would write it in Python, but C# is also now available. This moves the computation to the edge of the network to avoid the latency that a round-trip between the device and the cloud would incur. All this is important because, as users start to demand more performance and efficiency, every millisecond between operations is going to count. By skipping the round-trip, the services being tapped into on devices are being delivered in a more real-time fashion. It could potentially also remove the cost of bandwidth that companies (or their users) might incur to complete those round-trips.
8. AWS X-RAY
Developers who deploy applications on AWS have an awesome new set of tools from Amazon to help them monitor, track and respond to application health and issues in the form of AWS X-Ray and the AWS Personal Health Dashboard. AWS X-Ray is a fully-managed service that allows developers to debug their distributed applications. Using trace data from applications, X-Ray creates a visual console to easily visualise key metrics and health issues in code.
9. AWS PERSONAL HEALTHBOARD
This new dashboard allows app developers and tech teams to monitor all their infrastructure in one place.
The dashboard can provide real-time alerts and notifications and can also perform immediate responses using Lambda functions.
The Personal Health Dashboard will be a welcome change for AWS developers who have had to rely on third-party services to augment the very basic dashboard previously available from Amazon. Amazon’s moves come shortly after Microsoft’s release of Azure Application Insights, a similar application analytics solution released at the Microsoft Summit earlier this year.
10. AWS STEP FUNCTIONS
AWS Step Functions enables users to coordinate the components of distributed apps using visual workflows. It includes an editor that can map out the desired relationships among Lambda functions, whether they are in parallel, branches or sequences. “This is really going to change the way you build distributed applications,” Vogels said.
AWS Step Functions manages the operations and underlying infrastructure for you to help ensure your application is available at any scale. With AWS Step Functions, you pay only for the transition from one step of your application workflow to the next, referred to as a state transition. Billing is metered by state transition, regardless of how long each state persists (up to one year). The AWS Step Functions free tier includes 4,000 state transitions each month.
Like what you’ve read? Stay tuned for a couple of re:Invent deep dives from The Instillery team over the next few days. Or if you’d like to get the detail on these announcements firsthand then get in touch and we’ll take you through it.
