With organizations making rapid strides in their digital transformation journey, they see an evolving application and database system portfolio. Sequential query language (SQL) databases have revolutionized the way organizations manage the huge volume of data. SQL databases are intelligent, scalable, and relational databases built for cloud to optimize the durability and performance of analytics capabilities.
However, the surge in data volume and velocity also brings numerous challenges with analyzing and managing the massive data, and more so if the databases are of different types and dissimilar. An unorganized approach to data warehousing can hinder business users’ ability to query data reliably and at scale, and of course compromising the IT resources and costs efficiencies when the time is of the essence.
Unlike traditional MySQL databases, services like Amazon Athena are reinventing the way we approach data querying in the cloud. Amazon Athena is an interactive querying service that enables users to analyze Amazon Simple Storage Service (Amazon S3) objects and buckets. The service is serverless, meaning users simply execute queries without needing to worry about hosting or infrastructure maintenance. Since users choose how many queries to run, the service is provided on a pay-as-you-go (PAYG) model meant for significant cost savings and performance gains by compressing, partitioning, or converting data to a columnar format. By doing so, the size of the data surface goes down significantly and consequently speeding the querying execution.
While Amazon Athena is a powerful service, multi-cloud deployments and hybrid-data sources make it difficult to move data around. Trianz Athena Federated Query (AFQ) Connectors break down the data barriers by providing the ability to connect and query databases across on-prem and other public cloud environments. Custom Athena query federation connectors are integrated with Athena SDK for seamless query authoring experience and implement the security best practices defined by AWS. Connectors use existing AWS identity and access management policies, data protection using cryptographic services for credential management, and are integrated with cloud watch for audit management.
The Trianz Athena Federated Query Connectors supports SQL, Java Database Connectivity (JDBC), and Open Database Connectivity (ODBC) across public/private cloud, hybrid-cloud, and on-premises IT infrastructure types.
Current limitations with Amazon Athena
Currently, enterprises are experiencing some issues with data management when using Amazon Athena:
Data federation is unavailable outside of the AWS ecosystem
Third-party data federation solutions are too expensive
Data sources on AWS cannot connect to external non-AWS data sources
Business reporting across multiple data sources is not possible
Third-party analytics solutions lack scalability
High costs during data migration and ongoing governance for new analytics capabilities
The main problem is data federation. This is a process whereby a virtual database is created, with multiple data sources being ingested simultaneously. These multiple data sources are automatically cleansed and converted on the fly to establish a common data model, creating a single-source-of-the-truth (SSOT) in which front-end applications can execute queries.
Extending Capabilities with the Trianz Athena Federated Query Accelerator
The Trianz AFQ Connectors are purpose-built to overcome these challenges. Instead of moving data between databases or infrastructure platforms, federated databases allow data to be aggregated in real-time.
The federated database runs as a virtualization layer over the top of existing database sources. This greatly reduces bandwidth and storage requirements by minimizing the quantity of data-in-transit. For users, this means MySQL, NoSQL, and Oracle DB sources can be accessed and queried using a standardized access method across multi-cloud, hybrid-cloud, and on-premise infrastructures.
The AFQ Accelerator drives enhanced analytics and business intelligence capabilities in the enterprise:
Enable Your Data Workforce
Using the AFQ Accelerator, data scientists and analysts can execute queries across relational database management systems (RDBMS), non-RDBMS, data lakes or warehouses, and custom data sources from a single access point.
Increase Data Agility
With the AFQ Accelerator, enterprise data will become more agile, enabling faster query responses and a broader number of uses for analytics.
Business Intelligence
Data scientists and analysts can start analyzing data in place, enabling higher query throughput with minimal cost overhead. Furthermore, holistic access to data sources provides a fuller picture when generating insights and data visualizations. Users can work with any reporting platform to generate reports across hybrid environments.
Save Time
Time is money, and this is especially true when it comes to analytics. The AFQ Accelerator expedites time-to-market for analytics architecture using pre-defined components and design patterns. It empowers data scientists and data analysts with the ability to easily run queries across data stored in RDBMS, data lakes, and custom data sources without requiring IT interventions for most reports.
Broad Data Source Support
On top of existing AWS-provided connectors, Trianz extends native capabilities with the AFQ Accelerator, including:
Teradata Connector
Support for Teradata enables data analytics and predictive intelligence capabilities across hybrid- and multi-cloud environments.
Snowflake Connector
Support for the Snowflake data warehousing platform enables big data storage and analytics with no data siloes.
Google BigQuery Connector
Support for Google BigQuery serverless multi-cloud data warehousing with high querying throughput thanks to ANSI-SQL.
Cloudera Connector
Support for Cloudera enables cloud-agnostic multi-function data warehousing and analytics capabilities from the edge through to AI, with Cloudera Impala being at least 6x faster than Apache Hive.
Additionally, Hortonworks from Cloudera provides an open-source framework for distributed data storage and processing.
SAP HANA Connector
Support for SAP HANA brings a high-performance in-memory database that expedites data-reliant activities and enables real-time business decision-making.
OracleDB Connector
Support for OracleDB enables data storage and processing such as with online transaction processing (OLTP), and a lower storage footprint thanks to JavaScript Object Notation (JSON) structures.