Federated Query Engine or Data Warehouse?
With a growing demand to scale data and visualize analytics in real-time, no company is leading the big data resource market like AWS. Two of their most popular cloud-based data services, Amazon Athena and Amazon Redshift, are being sought out by organizations of all sizes to optimize processes, enable smarter decisions, and better serve customers.
While both services are excellent for those looking for a scalable big data analytics solution, there are some features that make the two similar AWS tools unique from one another. In the following article, we will give a quick overview of what Amazon Athena and Amazon Redshift are; compare their pricing, performance, and user experience; and why you should choose one service over the other.
What is Amazon Athena?
Amazon Athena is a serverless interactive query tool for running ad-hoc or pre-created ANSI SQL queries on data stored within Amazon S3. It allows users to perform complex analyses on massive datasets without having to worry about the underlying infrastructure, cost, or maintenance associated with traditional database management systems (DMBS). With Athena, AWS automatically handles all the infrastructure, so users only pay for the data scanned during the queries.
What is Amazon Redshift?
Amazon Redshift is a fully managed, column-based data warehouse designed for online analytical processing (OLAP). Users who have Amazon Redshift as their data warehouse and Amazon Simple Storage Service (Amazon S3) as their data lake can integrate the two seamlessly for a lake house approach.
As with Athena, Redshift allows users to combine multiple complex queries to provide insights on massive data sets. However, Redshift can handle queries on an even larger scale, with better performance and only slightly more cost.
Performance
Athena works with several data formats, including JSON, ORC, CSV, Avro, Parquet, and uses Presto with ANSI SQL support. Athena is ideal for non-technical users to handle quick, ad-hoc querying. But the platform can also perform complex analysis, including large joins, window functions, and arrays.
Compared to other enterprise cloud data warehouses, Amazon Redshift has up to three times better price performance, and the price-performance advantage improves as the data warehouse grows from gigabytes to exabytes. Amazon Redshift’s architecture has been configured to capitalize on AWS-designed hardware and machine learning (ML) to deliver the most cost-effective data solution at any scale. This includes using the AWS Nitro System to speed up data compression and encryption, ML techniques to analyze queries, and graph optimization algorithms to automatically organize and store data for accelerated query results.
Pricing
As mentioned before, Athena users only pay for the queries that they run and are charged based on the volume of data scanned in each query. However, users can get significant cost savings and performance gains by compressing, partitioning, or converting data to a columnar format.
With Amazon Redshift, users choose from several options to decide what is right for their business needs. This gives them the ability to scale storage without over-provisioning compute costs, and the flexibility to grow compute capacity without increasing storage costs. Redshift users can choose from:
Pay-as-you-go pricing
On-demand pricing
Managed storage pricing
Reserved instance pricing
User Experience
Amazon Athena is a serverless interactive query service, meaning there is zero infrastructure to manage. Users do not need to worry about configuration, software updates, failures, or scaling the infrastructure as their datasets and user base grows. Since Athena automatically takes care of all the infrastructure, users can focus on the data instead of scaling when demand is high.
With Redshift’s cloud-based data warehouse, users get a cost-efficient, fast-performing, reliable, and scalable solution that acts as a data-warehouse-as-a-service (DWaaS). In addition, as AWS fully manages the clusters, there are no database admin tasks to routinely perform, and the server performs continuous backups to make sure users do not lose their data in the event of a breach.
Should You Use Amazon Athena or Redshift?
Amazon Athena is the easiest option when looking to provide multiple users with the ability to run ad-hoc queries on data in Amazon S3. Being that there is no infrastructure to set up or manage, users simply create a database, choose a table name, specify where the data is on Amazon S3, and start analyzing data immediately.
Redshift’s data warehouse is best for users who have frequently accessed data that needs to be stored in a consistent, highly structured format. This gives them the flexibility to store their structured data in the Redshift data warehouse and use Amazon Redshift Spectrum to extend their queries out to data in the Amazon S3 data lake.
Amazon Athena vs. Redshift Summary
Redshift is best for large enterprises since it gives users with vast amounts of data the freedom to store their data where they want, in the format they want, and have it available for processing at the click of a button.
Since Athena users can query data without having to setup and manage servers or data warehouses, it is the better choice for non-technical users who can use Athena Federated Query (AFQ) Connectors to deliver powerful analytics instantly — no matter where their data resides.
While the better choice is dependent on the type of data being queried and analyzed, users will get the most data performance by combining the two services. Trianz AFQ Extensions allow users to scan data from S3 and execute the Lambda-based connectors to read data from on-premises Teradata, Amazon Redshift, Google BigQuery, and SAP HANA to simplify BI and facilitate cross data-source analytics.
