Data and databases have evolved tremendously in recent years, in many cases moving from on-premises solutions to the cloud, or to a combination of both. This is especially important when it comes to cloud-based solutions such as ServiceNow. Because the concepts and terminology have changed so much, we at SnowMirror thought it would be good to spend some time reviewing the concepts and terminology behind modern data storage and retrieval.
One of the most important concepts in data is that of ETL (extract, transform and load), where data handling is divided into individual steps. ETL has been a key concept in databases since the 1970s, but what is relatively recent is how the cloud has been brought into play. In ETL, data is copied from one or more sources into a destination system which represents the data differently from the source, or in a different context than the source.
ETL is often used in data warehousing, which is where SnowMirror comes in. SnowMirror is a tool for replicating data from cloud services like ServiceNow or Salesforce, so you can consider it more on the extract part of ETL. SnowMirror enables you to extract your data, and then you can do various things with it.
One of the most common use cases for SnowMirror is to create better reports based on the data from ServiceNow or Salesforce, using business intelligence (BI) tools such as PowerBI, Tableau, Qlik or SAP Business Objects. In this ETL scenario, the data is extracted from ServiceNow, and placed in a “staging area”, an intermediate step where data is stored in a data warehouse before being transformed to prepare it for further analysis..
The transformation steps in ETL involve applying rules or functions to data to prepare it for loading into the end target. This might include things like cleaning the data, selecting only certain columns to load, joining data or de-duplicating data.
Finally, we have the load step, in which in our use case the transformed data is prepared for final presentation. This might include creating tables, charts and graphs based on the transformed data.
Combined with advances in cloud storage and other technologies, ETL has also evolved over time. Services like Snowflake – which is also supported by SnowMirror – enable users to replicate raw data into their data warehouses, where it can be transformed as needed using SQL. This lets them skip preload transformations.
We’ll look more at modern methods of data warehousing, staging and ELT in the second part of this post.
Wherever your business has its replication needs – whether it’s for advanced reporting, disaster recovery or integration with other applications – SnowMirror is a vital tool in your toolchain.
If you’d like to talk with us about how SnowMirror can help your business get the most out of ServiceNow or Salesforce, get in touch here.
