Today’s DBA typically manages ten, hundreds or even thousands of databases -- RDBMS, NoSQL DBMS, and/or Hadoop clusters -- often from multiple vendors, both on-premises and in the cloud.
While management automation has made substantial strides enabling DBAs to handle larger workloads, the care and feeding of these databases is still all too often a burdensome task.
Data warehouses, data marts, and data lakes usually require the most attention. Let’s discuss how using Snowflake RDBMS can dramatically reduce the DBA’s workload!
It consists of three distinct layers.
A VDW can operate against any database; it is not allocated to a specific database.
Let’s look at the common tasks and requirements for creation and management.
The initial creation of a database typically requires the following steps. This is neither an exhaustive list nor applicable to every DBMS:
Snowflake’s separation of compute and storage means that ETL/ELT does not run on the same compute as operational DML queries. ETL/ELT processing is completely isolated and self-contained, leaving time for the DBA to work on legacy problems!
Tuning a legacy database platform varies tremendously depending on the platform. Common tuning work includes:
None of the performance tuning discussed previously is required for Snowflake.
Snowflake’s approach to performance tuning is based on the elasticity of its VDW.
For large, complex queries, the solution is to scale up the VDW to a larger size, i.e. more servers in a cluster. Again, this is a simple SQL or web UI action. A VDW can be scaled up (or down) dynamically; this won’t affect running queries, only queries submitted after the size change.
A multi-cluster VDW is used to support high concurrency workload which can be created with a simple SQL or web UI action. This type of VDW may also be “auto-scaled”, allowing dynamic activation and suspension of the number of clusters.
Another key feature for Snowflake performance is the ability to create multiple VDWs, each supporting a different type of workload or business area. The VDWs are completely independent of each other.
Finally, Snowflake’s “micro-partition” architecture eliminates the need for traditional performance tuning. This approach takes partition pruning to a new level, using an incredibly rich meta-data store, enabling both vertical elimination of partitions and horizontal examination only of relevant columns. Once again, leaving the DBA time to address legacy problems!
Director of Analytics
Jeff Jacobs is a senior data technology professional in the Data and Analytics Practice at Trianz. The Data and Analytics Practice works with enterprises to achieve significant competitive advantage via modern cloud technologies, with a particular focus on Snowflake Computing ecosystem.
Contact Us Today
Connecting more people to data has become imperative for organizations worldwide. In Top Trends in Data & Analytics for 2022, Gartner stated, “Connections between diverse and distributed data and people create truly impactful insight and innovation. These connections are critical to assisting humans and machines in making quicker, more accurate, trustworthy, and contextualized decisions while considering an increasing number of factors, stakeholders, and data sources.”Explore
Since the dawn of business, users have looked for three main components when it comes to data: Search | Secure| Share. Now let's talk about the evolution of data over the years. It's a story in itself if one pays attention. Back then, applications were created to handle a set of processes/tasks. These processes/tasks, when grouped logically, became a sub-function, a set of sub-functions constituted a function, and a set of functions made up an enterprise. Phase 1 – Data-AwareExplore
Practitioners in the data realm have gone through various acronyms over the years. It all started with "Decision Support Systems" followed by "Data Warehouse", "Data Marts", "Data Lakes", "Data Fabric", and "Data Mesh", amongst storage formats of RDBMS, MPP, Big Data, Blob, Parquet, Iceberg, etc., and data collection, consolidation, and consumption patterns that have evolved with technology.Explore
Enterprises have, over time, invested in a variety of tools, technologies, and methodologies to solve the critical problem of managing enterprise data assets, be it data catalogs, security policies associated with data access, or encryption/decryption of data (in motion and at rest) or identification of PII, PHI, PCI data. As technology has evolved, so have the tools and methodologies to implement the same. However, the issue continues to persist. There are a variety of reasons for the same:Explore
Finding Hidden Patterns and Correlations Innovative technologies such as artificial intelligence (AI), machine learning (ML) and natural language processing (NLP) are transforming the way we approach data analytics. AI, ML and NLP are categorized under the umbrella term of “cognitive analytics,” which is an approach that leverages human-like computer intelligence to identify hidden patterns and correlations in data.Explore
The Rise in Big Data Analytics According to Internet World Stats, global internet usage increased by 1,339.6% between 2000-2021. With nearly thirteen times as many people using the internet, this has resulted in a massive increase in the amount of data being processed daily. Our increased sharing and consumption of digital media also compounds this increased usage to create an enormous pool of data for big data analytics firms to process.Explore