The reality we face today is an exponential growth of data with the potential to drive operational efficiencies and innovation. But this deluge of data needs to be managed, in a structured way, through processes, frameworks, practices and ideally driven by technology with a minimum of human manual intervention.
Michael Covarrubias giving the Xtensible Lightning Talk around Maturing Your Data Management Reality. UAI Summit, Nashville, 2024
It is possible to mature your data management reality by focusing on the 3 key steps:
- Knowing and understanding your data
- Being aware of how your data flows
- Bridging designs to your implementations
The rewards are limited only by your imagination.
1. Knowing and understanding your data
Meet the “Data Catalog” and the “Data Dictionary”, both are required when you want to know and understand your data.
Your reality can improve if you use the Data Catalog capability to include all your data objects. This is accomplished by importing schemas and interfaces related to your operational systems and other technologies. This is only a partial reality improvement when it comes to knowing and understanding your data as you would still have multiple and potentially even conflicting definitions for your data objects.
The other part is accomplished via the Data Dictionary, populated utilizing selected information from the Data Catalog. This is where the IT reality meets the business reality, thus harmonizing understandings to that the same language can be used. Now, you have an accepted definition of your data object, which could be “Customer = Fist Name, Last Name” and where this “Customer” lives, in an officially accepted address format.
“The business user can understand what data is available with a clear lineage to the source system that provides it. The catalog and the dictionary is connected through technology. Think about what that means to the organization!”
Labels are configurable and applied to the data according to your business needs.
But wait, there is more. Consider the classification of the data, or where the system of record is from. This is where data labeling becomes more than handy, in fact it’s a necessity.
Data labels can tell you if the data is classified, if it needs to be protected, and what purposes the data can be used for. You can designate “System of Record”, Data Owners and Data Stewards and create labels that reflect your organizational needs.
CAPABILITIES
 |
Data Catalog is comprised of data models, data schemas, integration schemas, and other data definitions. |
 |
Data Dictionary provides names, definitions, and attributes for the data being used for both IT and OT. |
2. Being aware of how your data flows
Meet “Data Mappings and Transformation” needed to understand how your data flows and how it transforms.
Data always flows from one system to another. When building a Dataware house, even for an enterprise analytics solution, the reality is that this is accomplished in word documents, excel spreadsheets, and sometimes it is done through a technology provider for that purpose.
Depicting "Data Lineage" from source to sink, including transformations.
Disconnected technologies are creating a maintenance headache. Imagine instead the new reality where, source system and target system(s) are connected with the actual mapping and transformation logic.
CAPABILITIES
 |
Data Mappings and Transformation include both object and attribute mapping from source to target, as well as any transformation the data content needs to undergo in the process. |
3. Bridging designs to your implementations
Meet “Build Automation” and “Data Lineage” to physicalize your data and integration designs and to manage change.
This is where the new reality really happens. Imagine if the mapping and transformations from source to target system could be physicalized right to your enterprise analytics platform for instance. In other words, take the logical model and apply it to a physical model.
Generating your analytics platform instance connected to and based on your design.
You may also want to add elements such as attributes for look-up tables, or complex calculations for analytics. In the example above we are actually generating the Snowflake instance of the model.
“We have connectivity from our logical model, our data sources, governance all the way to what is actually being generated and ultimately deployed.”
The connected nature of our design gives us the lineage we need to not just deploy once, but to maintain and update our models and deployments when technologies change. Your enterprise analytics platform can be created and updated using the same connected technology, Affirma.
CAPABILITIES
 |
Build Automation design artifacts are connected to the generated code, ready for deployment. |
 |
Data Lineage traces an attribute’s journey all the way upstream to the system of origin, and all the way downstream to all the systems consuming or passing the attribute along. |
The Data Journey
By knowing and understanding your data, by being aware of how the data flows, by bridging your design to implementation you can really help your organization mature its data management capabilities and practices. Watch the Maturing your Data Management Reality Video for more details as given during the event.
About Affirma
Affirma is a seamless and single point of reference for data modeling, mapping, analytics, and integration for those who are seeking to act and digitally transform through their data in support of operations and changing business needs while embracing new technologies and innovation, developed by Xtensible. Learn more about Affirma and how to navigate change! Speak to a member of the Xtensible team.
About Xtensible
Connected data and technologies provide insights adding unimaginable value for our common future. At Xtensible, we ensure that you have frictionless access to your data across your entire organization all the time. Our strategic framework defines our engagement approach, helping you do more with less. Our approach increases the trust in data used for decision making, reduces risk, and enables you to make financially sound decisions. Our services are based on a partnership mindset and deep information and data, and systems architecture knowledge and experience. We leverage industry standards to support you through your entire journey towards strategic business objectives.
ABOUT THE AUTHOR
Michael Covarrubias
VP, Solutions and Strategy at Xtensible