Analysts have been a steady presence on our data teams from the very beginning, but sadly their role and reputation have been in decline in recent years. If we are to succeed in the new analytical era in which we are expected to turn our data into action, we must reverse course and rediscover the role of the humble analyst.
The current incarnation of the analyst was born out of the “insight era.” The main way of delivering data to the business was carefully packaging data into the likes of dashboards, charts, or reports. This was the territory of analysts, and it all went well…for a while.
First, our industry went through a hyper-specialization phase. The data pipeline got chopped up and doled out to new positions like data engineers, or more recently, analytics engineers. Analysts who had once enjoyed large swaths of the pipeline found themselves crowded out on all sides.
With specialist data roles becoming the standard, the status of the pure analyst deteriorated in the eyes of the data community, and those of our business partners, effectively turning the analyst into a junior role. This turned a once stable business-analyst partnership into one precariously built on tasks and hierarchy.
When I was a data analyst at a large company, the sense that our titles were holding us back was so acute that one day we were all promoted to data scientists. This was done without any training, without any change to our expectations or ways of working. It was purely an attempt to garner more respect and credibility with the wider business we were trying to influence.
This stigma extended beyond team dynamics. In 2020, data analysts earned less per year of experience than their data science or data engineering counterparts [1], silencing any analyst’s doubts about what we value in this industry.
Today’s analysts are left with one choice: specialize or die.
It’s worth pausing to remind ourselves why this is such a dangerous ultimatum. The answer to that is two-fold:
We may have a best-in-class data model, a hyper-performant database, or even state-of-the-art data apps, but if there isn’t an analyst in a position to integrate that information into the business, it means nothing.
To diminish the role of the analyst only serves to silence the role of data in any organization.
We need a bigger version of the analyst’s role.
Previously, we considered the endpoint of the data stack to be an output (e.g. a chart, a dashboard, or a report). Increasingly we are acknowledging the limitations of those mediums in more complex decision making and problem-solving. We must go past the “insights”, and into “action”.
We are dubbing the final step of the analytical workflow, from insight to decision, “The Last Mile”. If analysts are to continue to be the way the data stack delivers value to the business, then this is the space they must call their own.
Furthermore, if they are to own this space, they are going to have to develop a new set of skills and habits tailored to the last mile’s challenges.
Analysts already possess many of the traits required for the last mile of analytics including a strong mathematical foundation, and good data communication skills. However, there will be a few key skills required to adapt to this new land:
In my view, this is the most overlooked skill of analysts today. The most successful analysts are able to seamlessly bridge the gap between data and the business. They bring data into the context of the business problem, ensuring the team instantly understands not just what the chart says, but what it means to them. They can take ambiguous business requirements and translate them into upstream data requirements with ease, saving the entire team hours of rework.
In the future, this will not just be a talented analyst’s “secret sauce” but a skill identified and cultivated by all analysts.
In 2018, McKinsey emphasized the need for this translation skill by suggesting it should even become it's own [specialized] role:
“In their role, [analytics] translators help ensure that the deep insights generated through sophisticated analytics translate into impact at scale in an organization. By 2026, the McKinsey Global Institute estimates that demand for translators in the United States alone may reach two to four million.” -McKinsey (2018)
While I agree with the sentiment, I believe it will be future analysts that will be fulfilling the industry’s appetite for effective translation.
As decisions get more complex, we ask more and more of our data. Looking at data to help decide what should I do requires a more nuanced understanding of our methods.
To look at a dashboard of A/B test results and say with confidence that there is no bias in your interpretation, that you aren’t making decisions on a poorly designed experiment, or small sample sizes is nearly impossible for anyone, let alone a time-crunched executive. (This Netflix series of articles is an excellent example of how this might work.)
The analyst is uniquely positioned to help expertly guide the use of data in these complex decisions. This means they will be accountable for responsible and ethical use of data within organizations — something that lives in a convenient no-mans-land at the moment.
These discussions promote not just an improved use of data, but a more trusted use of data — something indispensable if data is to play a larger role in our business decisions.
Redefining the role of the analyst won’t happen overnight. This is a gradual change that I believe is already in motion. With metric layers, analysts and their business partners will have more autonomy than ever to use data to drive decisions, but they will need help doing that effectively.
Crossing the last mile may require analysts to both learn new skills and adopt new tools. We’ll also need to consider how our organizations and teams adapt to support new ways of working. In short, there’s no shortage of challenges, and thus, opportunities waiting for us in the last mile.
At Count, we’re building for the era of data-driven decision-making. Subscribe to hear more about the challenges and opportunities that lie ahead in More than Numbers.
[1] 2020 Developer Survey. StackOverflow. https://insights.stackoverflow.com/survey/2020#overview
