Data Governance using Azure Purview
Growth of data creation is unprecedented in current times and is expected to grow explode in near future. Ever since storage and data processing/computation capabilities have enhanced, it led to capturing every bit of data. It is available for analysis and deliver insight within.
https://www.statista.com/chart/17727/global-data-creation-forecasts/
As per “statista”, data creation in year 2035 will be around 2142 zettabytes as compared to 2 zettabytes in year 2010. This growth is simply phenomenal.
Data being captured will have many characteristics like
1. Structured Data
2. Semi — Structured Data
3. Unstructured data
In the Big Data domain, there are Vs to describe features of data
1. Volume
2. Variety
3. Veracity
4. Velocity
5. Value
Each V provides a unique challenge about the data management and throws challenges to build applications to store and compute. Deriving value out of the underline data is crucial. Transforming data into meaningful information unlocks value of underling raw data.
There could be many problems with huge ocean of data in the data lake e.g. Duplicate data. Many a times, a question is asked “What is the source of Truth ?”, “Who is the Data Owner” and many more.
Hence, with the explosion of data, Governance around this data must be applied. There are many tools available to simply the governance process. One such tool is Microsoft Azure Purview Introduction to Microsoft Purview — Microsoft Purview | Microsoft Docs
https://docs.microsoft.com/en-us/azure/purview/media/overview/high-level-overview-large.png#lightbox
Source: Microsoft Azure
Microsoft Purview is a unified data governance service that helps you manage and govern your on-premises, multi-cloud, and software-as-a-service (SaaS) data. Create a holistic, up-to-date map of your data landscape with automated data discovery, sensitive data classification, and end-to-end data lineage. Enable data curators to manage and secure your data estate. Empower data consumers to find valuable, trustworthy data.
Microsoft Purview automates data discovery by providing data scanning and classification as a service for assets across your data estate. Metadata and descriptions of discovered data assets are integrated into a holistic map of your data estate. Atop this map, there are purpose-built apps that create environments for data discovery, access management, and insights about your data landscape.
