Issue #2 - The Data Ecosystem: Where do you even start?
How to take that next step in your data knowledge
Read time: 5 minutes
Data isn't easy. It's incredibly complex. What is worse, however, is nobody appreciates how complex it is. Why?
People just see 𝐭𝐡𝐞𝐢𝐫 𝐨𝐰𝐧 fragment of the data ecosystem - be it engineering, machine learning, or analytics - and stop there. Their expertise ends and considerations for other related data domains (e.g., Governance, Platform, Security, etc.) are limited.
It’s like appreciating a single piece of a puzzle without acknowledging the entire picture.
But it isn’t their fault. It is how we are taught to learn about and succeed in the data industry.
You are expected to be an expert in one thing and own that, then have others take responsibility for the other bits. The problem is that this model relies on strong communication lines with a seamless understanding of how data domains are interconnected, which just isn’t a reality.
And this is why data is so hard. And why data is so siloed.
Can’t the CDO solve this?
When you think of the entire data picture, you may think, “Isn’t this the responsibility of the CDO or Head of Data.”
Yes, it is, but (having seen this in action across dozens of companies) they can only do so much. Here are the common limitations that prevent a holistic approach from the top:
CDO/ Head Of in name only – While this person has the title, they have few direct reports. Or the expectations of their role are guided by their boss (e.g., CTO, CIO, CMO, etc.), who has inherently different goals than them.
Lack of technical or business experience – Having both technical and business experience is rare. So while the CDO is supposed to understand the whole data ecosystem, they often lack experience in key data domains (e.g., platform, governance, engineering, etc.) or lack the business context in their career (e.g., managing c-suite stakeholders, building strategic plans, etc.), limiting their ability to speak across everything.
Inefficient org structure or operating model – This comes back to reporting lines not matching up and domains not speaking to each other. If, for example, the technology teams don’t work closely with any of the data teams, or adhere to the CDO’s requests, then you get hurdles trying to match up data engineering or governance with some of the main providers of data (e.g., CRM systems, social media data, etc.). Plus, the CDO often has no power to connect these teams.
Corporate politics – CDOs still don’t have enough power at the c-suite table to really drive their agenda. Hence, they often get hindered by other agendas, buzzy trends (like AI) or lack of budget. What you get is requests to deliver quick value at the expense of long-term foundational investments.
Too much responsibility – Ever see a CDO job description that highlights being technical and business oriented? A lot of companies have extremely high expectations of CDOs to code, do machine learning, clean the data, while growing the data team. This doesn’t work and takes these senior leaders out of the planning and strategy that helps data & analytics succeed.
Summing up all these points, the CDO, while important, is not going to solve the Data Ecosystem issues we highlighted last week. There is just a lack of experience, time, and capability to do it themselves!
So what do we do then?
Starting to Navigate the Data Ecosystem
Spoiler alert: I don’t have a perfect answer to the above question.
But it does start with three things:
Recognising the need
Prioritising the focus
Educating ourselves
The Need
As data professionals, we can’t afford to be myopic. We need to broaden our horizons, to understand how our work in one area impacts, and is impacted by, others.
Think about it: what good is an advanced analytical model if it’s built on poor quality data? What’s the use of cutting-edge data security if it’s so restrictive that it stifles analytical creativity? How does building this engineering pipeline impact governance rules?
This isn’t just about being versatile or a jack-of-all-trades. This is about actually succeeding in an increasingly data-driven world. We need to challenge ourselves, challenge our peers, to think beyond our self-proclaimed expertise.
And most individuals can still only speak to a sliver of the total picture. So the need is about understanding that gap in our knowledge. What knowledge gaps hinder our output and ability to deliver value?
The Focus
Once we recognise that need, we can begin to solve for it. Note that this isn’t about solving for everything, as your role won’t encompass all things data.
So prioritisation needs to come first. With finite resources, data teams and professionals need to take stock of their current situation, locate the key issues/ barriers, and understand what the root causes of those issues are.
Doing this at a holistic level is critical: for the whole data department, this might mean looking at all the domains at once. For a team, it might mean understanding issues internally and with other adjacent teams. For an individual, it might be determining what is primary, secondary, and tertiary to what you do on a daily basis.
This gives you a laundry list that you can look at and think about: “If I could solve 2-3 issues, where would I start? What would make the most impact?”
Education
Data professionals love learning new skills, especially when they are technical!
But education is more than learning new things. It is about teaching yourself how to think and absorb new ideas that will make you better.
So what are some learnings that may enhance your current domain?
As a Data Engineer, you could learn… Governance to ensure data quality is maintained across pipelines; security to address privacy concerns; analytics/ science to know why you are building these data products/ pipelines
As a Data Scientist, you could learn… Engineering to access clean data; business strategy/ acumen to build better models; data communication to translate outputs to stakeholders
As a Data Analyst, you could learn… Business acumen & needs gathering to work with business stakeholders; solution architecture to design analytical products; DevOps or software development principles to build more efficiently and effectively
Everybody’s learning path may differ and knowing why you have mapped out that journey is half the battle. The next step is finding the materials to navigate your own data ecosystem, which this newsletter will continue to provide.
Next week, we examine the Data Ecosystem in its entirety: What does it include? How does it map out? How does it link together?
It will definitely be a fun issue, so you won’t want to miss it!
Thanks for the read! Comment below and share the newsletter/ issue if you think it is relevant! Feel free to also follow me on LinkedIn (very active) or Medium (not so active). See you amazing folks next week!