Data Management: THE Cookbook
Data Management: THE Cookbook
DATA MANAGEMENT
COOKBOOK
A POCKET GUIDE FOR IMPLEMENTATION
OF DATA MANAGEMENT
Irina Steenbeek
“All roads lead to data.”
Copyright © 2018 Irina Steenbeek. All right reserved.
All rights reserved. This book or any portion thereof may not be reproduced or used in any
manner whatsoever without the express written permission of the publisher except for the
use of brief quotations in a book review.
ISBN: 1984149938
ISBN-13: 978-1984149930
Table of contents
Introduction 4
EXTRA MATERIALS:
Case study: Data Quality 26
Step-by-step checklist 28
3
Introduction
The amount of data available in your company and in the business environment grows
exponentially. Thus the task of keeping the data in control becomes more and more
difficult with time.
A lot of companies realize that data is an important asset and has to be managed ac-
cordingly. They would also like to get value from data. Everyone wants to be ‘data-driv-
en’ these days. What lies beneath this idea, is the wish to make the decision-making
process easier and more effective. It means delivering the required data of acceptable
quality to the decision makers when and where they need it. In short: a lot of companies
understand the vital necessity of proper management of their data. The main question
now is: how put this into practice?
Knowing the potential of your data, and managing it correctly is the key to a suc-
cessful business. As a result of well-implemented data management, you will be able
to reduce risks and costs, increase efficiency, ensure business continuity and successful
growth.
4
In this book, we invite you for a five-course dinner. During each course we will explain
the steps of our 5-step system one by one.
The content and the order of the steps are a result of practical experience, but as any
recipe, they are more of a guideline than a list of strict instructions. Tastes (and busi-
nesses) differ, so feel free to choose and adjust, make iterations, or even skip steps, if
that fits your purpose.
5
Setting up the table
FIND YOUR DRIVER
Before diving into the development of a data management function, let us align our
understanding of the basics.
TOGAF 9.1 stipulates that ‘business function delivers business capabilities closely
aligned to the organization’4. It also defines ‘capability’ as ‘an ability that organization,
person, or system possesses. Capabilities…typically…require a combination of organi-
zation, people, processes and technology to achieve’5.
So the following conclusion derives from these statements: data management is a
business function, and in order for it to operate properly, you need to combine organi-
zation, processes, technology, and data all together.
6
just ‘come up’, it comes from a long process of thinking, analyzing and evaluating the
needs of the company. This all leads to a certain point in time when necessity of setting
up or extending a formal implementation of this function cannot be overlooked any
longer.
Very often, this realization comes along with some urgent challenges. These chal-
lenges are to some extent the business drivers we are talking about.
After finishing this very first step, you should have a pretty clear idea of the main drivers
and goals for the data management initiative. And hopefully you have already gotten
acquainted with a few main stakeholders, which will come in very handy during the fol-
lowing steps.
7
First course:
apéritif
DEFINE DATA NEEDS
Ingredients Preparation
data stakeholders BEFORE YOU START
Every company deals with some issues around data. It
data challenges is often difficult to find volunteers who would want to
take on the responsibility for managing this. Uncertainty
data requirements in such responsibilities will result in a waste of time and
resources.
data quality
INSTRUCTIONS
1.Identify all stakeholders within your company. They
are your main partners in making your data managea-
ble. Usually the main stakeholders are top management,
IT, finance, sales, production, and other business units.
2. Collect and align the requirements of different stake-
holders. Do not be surprised if all of you have different
needs and requirements for the data, its quality, the fre-
quency of its delivery and the tools and devices involved
in data delivery.
8
RESULTS
As a result of your efforts you will:
• know who the most influential stakeholders are and how to approach them;
• have improved efficiency of your business partnering communication;
• be able to protect your future business and information needs and requirements.
DELIVERABLES
1. Stakeholders’ map and assessment.
2. Communication approach.
3. Business and data requirements.
It is advisable to revise data requirements at least once a year. This has several reasons,
but mainly because almost every year you might receive new requirements from regu-
lators, thus your management reports will require regular updates.
You should organize revision before the the start of the budgeting cycle. By this time,
you will have better insight in additional investments regarding new technologies, ap-
plications, projects, developments of reporting, etc.
9
Second course:
appetizer
DIVIDE TASKS AND
RESPONSIBILITIES
Data management is a shared responsibility
between data professionals and others stake-
holders6. You need to establish data govern-
ance rules, starting with defining the main
data management principles.
Ingredients Preparation
data management BEFORE YOU START
principles The finance department often sees data as their respon-
sibility, although you know now that every other depart-
data governance ment also has their own data needs. Assigning a data
owner and defining a clear set of accountabilities, will
data owner result in more efficient data processing.
shared responsibility
INSTRUCTIONS
rules, roles, and tasks
1. Set up data management principles. There are a lot
of principles which your company can adapt and imple-
ment. For data management, you need to choose those
which meet your company goals and culture.
2. Agree on governance rules and procedures. This ac-
tion will allow you to define accountable functions per
specific data management task. Data governance con-
sists of:
• roles and responsibilities;
10
• data management tasks assigned to specific roles;
• processes and procedures;
• governance bodies.
The step is iterative. During the first stage, when the data management function is not
fully designed, you will comprise a preliminary list of tasks and allocate responsibilities.
Later on you might need to revise this list.
3. Identify the function responsible for data management. Although data management
is a shared responsibility, you still need to decide who will have the ultimate account-
ability for the coordination of the tasks. There is no agreed vision on the place of the
data management function in the company. Some companies consider it an IT function,
some assign it directly to the CIO (if present), others give the responsibility to the finan-
cial business unit. It is your company’s choice.
The most significant challenge for the person in this role, is balancing the (often
contradicting) interests of different stakeholders and guiding all of them to common
success.
4. Identify data-related roles, including the owners’ accountabilities. Aside from ‘data
owners’, you must have heard of ‘system owners’, ‘process owners’, ‘product owners’,
etc. The distinction between such functions is not always clear. It is crucial to align re-
sponsibilities of various roles and to assign them to business functions.
RESULTS
As soon as the data management principles, rules and roles are set up and agreed
upon, you will:
• know who you can approach to discuss your data-related (including data quality)
issues and solutions;
• be aware of all the concerns the main data stakeholders have;
• be sure that all the issues with data will be resolved according to an agreed proce-
dure.
DELIVERABLES
1. Data management principles.
2. Data policy, governance bodies, procedures, roles, responsibilities.
3. Matrix: data management tasks vs roles.
4. Matrix: roles vs functions.
11
Third course:
entree
ORGANIZE YOUR
DATA HOUSE
Clear communication between your colleagues
Part 1: and business partners is crucial in any business.
UNDERSTAND YOUR DATA You want to spend less time explaining what
you mean and avoid any unnecessary duplica-
tions in your reports. How do you achieve this?
Ingredients Preparation
professional language BEFORE YOU START
Speaking the same language with your co-workers and
internal and external external parties is not as easily achieved as it seems.
communication Usually there are two issues which you can encounter
while communicating on professional level:
business glossary
• You use the same term as your colleague does, but
it means something entirely different.
report flow
• You use different terms which all mean the same
thing.
This is often the case with internal, as well as external
communication, and is also reflected on the quality
of data and reports that are being exchanged. The re-
ports received by the stakeholders might not always be
as comprehensible for them as you intend. And this, in
its turn, can have an undesirable effect on their deci-
sion-making process.
This issue creates problems with reconciliation of re-
ports and figures from different departments and build-
ing enterprise DWH and BI solution.
12
INSTRUCTIONS
1. Create a business glossary by analysing:
• company policies;
• regulations relevant for your business;
• main reports.
2. Agree with stakeholders on the definitions of terms.
3. Create a catalogue of main management reports circulating in your company.
4. Create a report flow.
RESULTS
By now, you should be:
• aware which and how many reports contain duplicate information;
• able to optimize the number of your reports and remove the duplicate informa-
tion;
• able to reduce the efforts on reconciliation reports from different departments;
• able to communicate more productively and efficiently with your co-workers and
business partners.
DELIVERABLES
1. Company’s business glossary.
2. Report catalogue.
3. Report flow.
13
Third course:
entree
ORGANIZE YOUR
DATA HOUSE
Your dream is that pressing one button will
Part 2: make all required information magically ap-
LOCATE YOUR DATA pear on your screen. Unfortunately, such a tool
does not exist yet. It is not about the tooling
though: the key to easy access to data is know-
ing its exact location.
Ingredients Preparation
critical data elements BEFORE YOU START
You have already identified how many reports are circu-
applications lating in your company. Not all the data in these reports
is equally critical to your business. Now it is time to min-
a ‘golden’ source imize your efforts to a feasible minimum by identifying
critical data elements that constitute the reports. It is
also time to establish how many systems and applica-
tions are in use in your company.
INSTRUCTIONS
The main task is to identify the location of the most im-
portant elements within the applications involved in data
processing, and pinpoint the ‘golden’ sources where the
data elements were initially put into processing.
1. Identify critical data elements that have the biggest
influence on your work results.
14
2. Catalogue all of the main applications, services, and interfaces.
3. Map applications and critical data elements.
4. Identify the ‘golden’ sources for critical data elements.
5. Extend the list of data elements and repeat actions 1 to 4.
RESULTS
By now, you should know how to:
• search for the sources of information faster and more efficiently;
• save time on reconciliation of reports as you can acquire data from the ‘golden’
sources;
• decrease the number of processes, reduce duplicate applications, and save mon-
ey in the process.
DELIVERABLES
1. Catalogues of:
• (critical) data elements;
• applications;
• ‘golden’ sources.
2. Matrices:
• application vs (critical) data elements;
• ‘golden’ sources vs (critical) data elements.
15
Third course:
entree
ORGANIZE YOUR
DATA HOUSE
Once in a while you internal audit or external
Part 3: regulators request you to explain how you de-
MANAGE THE DATA FLOW rive to certain data in your reports. The same
data can lead to different results, depending
on transformation it undergoes along the way.
The key to knowing your data is understanding
these transformations and the way it travels
starting from the source to the end user.
Ingredients Preparation
data flow BEFORE YOU START
You and your colleagues from other departments use
data transformation the same data, but sometimes it leads to different out-
comes. Who is right? It costs a lot of time and effort to
data life cycle investigate what causes these issues, but such investiga-
tions still happen almost daily. Due to new regulations
data lineage (i.e., GDPR), the amount of such tasks will only grow.
Supporting the data flow information is one of the
most complicated data-related tasks that many compa-
nies deal with. The bigger the company the more com-
plex and costly the solution will be. Every company has
to find a feasible solution depending on its size and re-
sources.
The success lies not in the right software solution. The
key is to get all the staff involved in sharing their infor-
mation and be willing to make it maintainable.
16
INSTRUCTIONS
1. Document or find a technical solution for the presentation of the data flow. Data lin-
eage describes the changes that data undergoes from source to its end user. The main
components of data lineage are business processes, application landscape, business
roles, and technical metadata.
2. There are three solutions possible for documentation of data lineage:
Solution 1: Data lineage by design.
This requires highly automated software solutions that exist on the market. There are
several providers that offer such solutions.
Solution 2: Descriptive data lineage.
You analyze and describe your business processes, applications, aggregated data, and
controls. The most important challenges are:
• centralization of the information to make it available to all stakeholders;
• involvement of all stakeholders in the process;
• making this process maintainable.
Solution 3: A combination of solutions 1 and 2.
RESULTS
Making data flow information available will result in:
• saving a lot of time not having to investigate data quality (and other)issues;
• decreased operational risk due to the decreased amount of issues with data qual-
ity;
• increased work efficiency of your staff.
DELIVERABLES
The main deliverables will depend on the type of the solution you have chosen. As a
minimum you have to ensure centrally located and well-described documentation that
links the following components of data lineage:
• business processes;
• business roles;
• applications;
• (critical) data elements;
• business rules and controls.
17
Third course:
entree
ORGANIZE YOUR
DATA HOUSE
The saying ‘garbage in, garbage out’ is a pre-
Part 4: cise description of the relationship between
IMRPOVE DATA QUALITY the quality of your data and the information
on which decisions are based. You need to en-
sure the quality of your data in order to make
your business grow and prosper.
Ingredients Preparation
data quality BEFORE YOU START
Correcting errors made during data input or processing
manual corrections costs a lot of time and energy, it means extra hours for
the staff doing manual adjustments, repeated correc-
data quality dimensions tions at least on a monthly basis, etc. You want your staff
to focus on the analysis of the data, instead of investing
measurement criteria their time in endless corrections.
INSTRUCTIONS
1. Define who will be responsible for data quality within
business units. Unless, of course, you have already done
it while setting up data governance (Step 2). If you have
- well done, now you can focus on the next points!
2. Define critical data elements for which you will exam-
ine and improve data quality.
3. Define the measurement criteria for data quality.
18
There are several data quality dimensions such as: accuracy, completeness, consisten-
cy, timeliness, etc.7 For the maximum benefit of your company, you need to prioritize
the criteria which reflect the needs of your business the most.
4. Define techniques you wish to use for the ‘root – cause’ analysis. There can be a lot of
different reasons why your data is of insufficient quality. Some of the common sources
of such issues are:
• data entry process;
• data processing, including manual operations;
• system (mal)functions.8
5. Set up a data quality issue log.
6. Start executing internal data quality initiatives.
RESULTS
Working with data of better quality, you will:
• reduce operational risk;
• improve the decision making process;
• save resources on reconciliations and fixing reporting issues.
DELIVERABLES
1. A list of data quality responsibilities assigned to business functions.
2. A catalogue of data issues.
3. Data quality issues resolution plan.
4. Data quality issues resolution business process embedded in daily activities.
5. Fine-tuned analysis techniques for data quality issues.
19
Fourth course:
main course
ASSESS THE GAPS
Ingredients Preparation
business function BEFORE YOU START
It is important to be on the same page with your busi-
data management ness partners regarding feasibility and necessity of the
capability changes. Also, you need to know how to achieve re-
quired results using minimal resources and effort.
gap analysis
roadmap INSTRUCTIONS
You have to make a gap analysis between the current sit-
uation and the situation ‘to be’. The gap analysis consists
of the following steps:
1. Define which current business functions in your com-
pany are related to data-management.
2. Finalize the list of the data management functions
you wish to develop (the revision we were talking about
on p. 13).
20
3. Find the gaps between ‘now’ and ‘to be’. You can use the check-list in the end of this
book (Appendix 2) to assess which data management capabilities are in need of further
development.
4. Outline a roadmap to close these gaps.
RESULTS
As a result, you and your company’s management will have a clear strategic vision on:
• which data management capabilities your company needs;
• how long it will take to reach the desired situation and what it will cost;
• what your company will gain as the result of this effort.
DELIVERABLES
1. An overview of the required data management functions within your company.
2. A gap analysis between the current and desired future situation.
3. A roadmap.
4. A filled-in check list for data capabilities (see Appendix 2).
21
Fifth course:
dessert
KEEP GOING
Ingredients Preparation
maintenance and BEFORE YOU START
development
All the preliminary preparation is done, stop dreaming
and start doing.
INSTRUCTIONS
There are various approaches you can choose from. You
can work on project base, or you can go Agile-style. It
does not matter which approach you will choose, as
long as you keep the following key success factors in
mind:
• Data management needs to be set up as a busi-
ness function.
• Data management is a shared responsibility: the
staff from different departments need to be in-
volved on a daily basis.
• Top management has to be the main sponsor and
supporter of the implementation.
• Once set up, it requires permanent maintenance
and development.
22
• Some tasks of data management (i.e. data quality, data flow) are ongoing pro-
cesses.
• There are not too many widely experienced data management specialists on the
market. It would be wise to train and develop your own staff.
• You need to concentrate on small deliverables that can immediately improve your
current processes and deliver results.
RESULTS
As a result of this last step, you will:
• get a clear vision on the tasks to be done in a different time perspective;
• know which deliverables and results you have to request from your staff;
• know when you can expect improvements in your daily work, which investments
and which reduction of costs you need to plan.
DELIVERABLES
The main deliverable is a clear operational model for data management in your compa-
ny.
23
Notes
1. DAMA International. DAMA-DMBOK: Data Management Body of Knowledge, 2nd
edition. Technics Publications, 2017.
2. EDM Council. “Data Management Capability Assessment Model, DCAM 1.2.2. (As-
sessor’s Guide)” EDM Council, 23 Jan. 2018, www.edmcouncil.org/dcam.
3. The Open Group. “TOGAF Version 9.1”, The Open Group Standard no. G116, 2011.
4. TOGAF 9.1, 23.
5. TOGAF 9.1, 23.
6. DAMA-DMBOK 1, 5.
7. DAMA-DMBOK2, 465.
8. DAMA-DMBOK2, 467-469.
Works cited
DAMA International. DAMA Guide to the Data Managemen Body of Knowledge, 1st
edition. Technics Publications, 2010.
DAMA International. DAMA-DMBOK: Data Management Body of Knowledge, 2nd
edition. Technics Publications, 2017.
EDM Council. “Data Management Capability Assessment Model, DCAM 1.2.2. (Asses-
sor’s Guide)” EDM Council, 23 Jan. 2018, www.edmcouncil.org/dcam.
The Open Group. “TOGAF Version 9.1”, The Open Group Standard no. G116, 2011.
24
Extra
materials
CASE STUDY: DATA QUALITY
STEP-BY-STEP CHECKLIST
Appendix 1
Case study
DRIVER: DATA QUALITY
26
the business glossary and or by creating conceptual or logical data models.
14. Prepare an action plan for DQ issues resolution according to the DQ governance
procedures.
15. In order to execute root-cause analysis, you need to document data lineage for the
critical data elements. This process can be broken down into the following steps:
a. identifying related business processes;
b. creating a catalogue of applications;
c. identifying location of data elements in applications by documenting database
metadata;
d. documenting business rules;
e. analyzing existing business processes and data quality controls.
16. Identify data quality requirements of different data users.
17. Align your activities with main stakeholders, especially IT.
18. Clean historical data if needed.
19. Develop data quality checks and controls, based on DQ requirements.
20. Adjust existing data processing flows.
Take action
23. Realize your strategy for fit-for-purpose data delivery with the required level of
quality.
27
Appendix 2
Check list
DRIVER
DATA NEEDS
Stakeholder map
Stakeholder assessment
Stakeholder communication approach
Business needs and requirement
Data needs and requirements
28
DATA HOUSE
THE GAPS
FINAL PRODUCT
Implemented operational data management function
29
A lot of companies realize that data is an invaluable asset and has to be managed ac-
cordingly. They would also like to get value from data. Everyone wants to be ‘data-driv-
en’ these days. What lies beneath this idea, is the wish to make the decision-making
process easier and more effective. It means delivering the required data of accept-
able quality to the relevant decision makers when and where they need it. In short: a
lot of companies have the necessity to manage their data properly. The main question
is: how do you put this in practice?
Knowing the potential of your data, and managing it correctly is the key to an effective
and successful business. As a result of well-implemented data management, you will
be able to reduce risks and costs, increase efficiency, ensure business continuity and
successful growth.
In this book, we invite you for a five-course dinner. During each course we will explain
the steps of our 5-step programme which guarantees successful implementation of
data management.
ISBN: 1984149938
ISBN-13: 978-1984149930