News

Data unlocking via the ANVA Data Platform (ANVA Data Warehouse)

8
March
2023

The growing demand for data disclosure is an important issue. For you as a customer, but certainly also for us as a software supplier. In the last magazine we wrote about our renewed API strategy and zoomed in on our new way of looking at data. In it, we also briefly mentioned that ANVA has developed a new data warehouse solution for customers, the ANVA Data Platform. In this blog, I'd like to tell you more about the deployment of the datastore and elaborate on the various options for unlocking data for use in connected solutions.

For real-time data linking, ANVA uses several applications within its software solutions. This is something I am happy to tell you about later in another article. For non real-time data, ANVA has recently spent a lot of time developing a data warehouse solution (DWH). A DWH centralizes and consolidates large amounts of data from the entire ANVA landscape, allowing you to gain valuable insights and access your data in a simplified way, without putting pressure on the performance of your ANVA system. Not to confuse your own DWH solution with the DWH technology ANVA offers to customers, we call ANVA's data warehouse solution: ANVA Dataplatform.

Engineering Data platform (ANVA Data warehouse).

ANVA's Data Platform, which we provide at no cost to every customer, is built using two tools: Snowflake and Matillion. Snowflake is a data warehouse solution consisting of storage and compute. Storage is the storage of the data available within the data warehouse. Compute is the computing power available to perform information from or operations on the data in the data warehouse. Matillion is an ETL (extract, transform, and load) tool used to transform the raw data from ANVA systems into usable data within the Data Platform. The advantage of Matillion is that data operations are performed on the DWH and therefore use the compute of the data warehouse as opposed to when the operations are done in memory.

If we look at the process of getting ANVA 4/5 data available in the Data Platform we distinguish the following 5 steps:

  • Data is extracted from the database into CSV files (ANVA Extract). These are only the files needed for the Data Platform;
  • The CSV files are uploaded to ANVA's Cloud environment (AWS S3);
  • Matillion starts the process to load (load) the raw data from the CSV files into the Snowflake environment;
  • Matillion starts a process to convert the raw data to data in the correct data format;
  • The data is theoretically usable by parties to use. To increase the value of the Data Platform, another transformation is underway;
  • We are currently working on a faster way to get the data into the DataWareHouse without the intervention of Matillion using Snowpipe. The data then goes directly from ANVA 4/5 via ANVA Extract to Snowflake. This allows us to refresh data at very short intervals.

Advantage Data Platform

The advantage of the Data Platform is that we can load data from different sources and then merge them to get combined insights. The first example is that we combine both ANVA 4/5 and ANVA Hub and partner data to get a complete view of the data in our systems. We are exploring what other external sources we want to add to the data warehouse to get even more combined insights into the business process.

Future Vision

Using our platform offers ample opportunities to innovate, for you as a customer, but also for us as ANVA now and in the future. In addition to BI solutions, we see other potential applications. For example, portals can use the Data Platform to easily access combined data to then display to users. Machine Learning applications such as automated policy acceptance or risk profiling across the portfolio offer us and our clients the opportunity to develop far-reaching automation applications in the future.

Read more

Delve into the possibilities on our ANVA Data Platform page.

Want to know more?
Contact us
Contact