Count AI Beta is here!
Ad-hoc requests have a bad reputation.
At their worst, they are a data team’s nemesis - distracting the team from ‘valuable’ and ‘interesting’ projects, causing loads of rework, without having any idea how important any of these requests really are.
Most often though they are a reason for data teams not to make big changes: “I can’t build a metric tree or think about structure changes, our team is drowning in ad-hoc requests. We don’t have time for anything else.”
However, there is another world in which ad-hoc requests are not the monster we imagine. Where ad-hoc simply demonstrate a curiosity and a desire to use data to understand their own world. Where a ‘quick question’ could be an olive branch, inviting data teams into the business’s biggest challenges.
“Unpopular opinion... I actually quite like ad hoc requests. Of course they can be a pain if the process is badly managed or expectations aren't clear, but they can be great ways for data team members to understand the business better, and for the business to understand their world through data.” - Anonymous
In this article, we’ll explore the differences between healthy and unhealthy ad-hoc request cycles, and discuss the top pieces of advice we’ve heard on moving from unhealthy to healthy.
Big thanks to the following for their input and advice!
We've all experienced unhealthy ad-hoc cycles. This is when we're asked for 'a quick question' in Slack that turns into a 3-day scavenger hunt for an insight no one can describe. It's when you finally get to an endpoint, think you've helped the business find something valuable, an insight that might change the course of how that team works, or maybe just improve their metrics that quarter, only to find out they decided to do what they were going to do anyways. It's exasperating, draining, and fruitless.
In contrast, healthy ad-hoc can feel oddly...good. When you get a request from the business, you can usually anticipate why they're asking - what hypothesis they have that can crack the big problems you helped them identify. You are brought into their problem-solving, helping each other come up with new theories to test and lines of thinking to explore. Your time spent doing this is often more valuable than the time spent building another report or answering a vague data request. These are the days we love most in data.
But how do we get there?
The first step in getting a handle on your ad-hoc request problem is to track and measure it. Amanda Sjöstrand Henriksson, Analytics Manager at Insurello suggests creating a simple spreadsheet to track:
We, of all teams, know the importance of measuring what matters, so this is a fundamental step to getting a handle on your ad-hoc request problem.
You can see a template audit sheet in the link below:
With your auditable list from the step above, you can now begin to use it to make some tangible changes. Anzisca Van Rooyen, Senior Data Analyst at Teya advises:
“Most of the time ad-hoc requests can be grouped. For example, once a week someone asks for a list of customer details - build a self service product. A dashboard that always needs debugging- find the route cause or save the code/response needed to debug.”
Jerrie Kumalah, Analytics Engineer at Seat Geek echos this:
Figuring out the themes that are emerging from these ongoing ad-hoc requests. They often follow a pattern which will help the team build better processes.
If you’re looking to make major changes to your ad-hoc request process, this is where you can make the biggest difference by actioning the common sources of requests.
If you have been drowning in ad-hoc requests for long enough you are bound to be cynical about them and are likely looking for this article to teach you how to eliminate them. That may not be the best approach according to Amanda Sjöstrand Henriksson, Analytics Manager at Insurello:
"[Ad-hoc requests] can give you the chance to work closely and in-the-moment with your stakeholders, to be there when things happen and work as a team rather than an outsider. Before you set off to ‘fix’ this problem, you have to remember what works about ad-hoc requests, too.”
Will Sweet furthers this idea by encouraging us to see that ad-hoc requests are actually a great way of building trust with the business.
“View each question as an opportunity to build more trust.” [1]
Guillaume Ansanay-Alex reminds us that ad-hoc requests actually contribute to a company’s bottom line due to what he calls the cross-fertilization between business topics and teams:
“Tech workers likely get requests from all around a business unit or even the whole business. Getting higher exposure to business contexts enable them to connect the dots between different requests. Like in, ‘We are talking about setting store pricing levels for various regions, but did you know that we worked on a multi-feature segmentation of stores for the product folks? Wouldn’t it be an even better approach?’” [2]
In summary, banning ad-hoc requests entirely may eliminate the pain you feel, but it will also come at a significant cost and diminished impact on the larger impact. Our energy will be better spent focused on how to do them well.
A good first step, then, is being able to classify the ad-hoc requests we receive into different categories. Ryan Dolley, VP of Product Strategy at GoodData has defined two types of ad-hoc requests:
“High-value ad-hoc questions deliver insights that materially improve business outcomes. They may require new ways of thinking supported by new data pipelines or metrics. They lead people to take action. High presentation-purpose fit leads to high-value ad-hoc questions.”
“Low-value ad-hoc questions aren’t unimportant, but they don’t have a large impact on the business. These are often reconfigurations of existing knowledge to show a slightly different slice of the business. Think, ‘Can I see this metric by quarter?’ Low presentation-purpose fit leads to low-value ad-hoc questions.” [3]
Being able to distinguish between these requests can be difficult, and requires asking good questions (Tip #4), but the act of classifying high-value and low-value ad-hoc requests is a good way to start breaking down requests.
Much like how product teams have to leave room to bug fix, data teams should leave room to answer ad-hoc requests. Dinda Calista, Analytics Lead at Cleo tells us that there analysts there are expected to manage all of their tasks, including ad-hoc requests, in their squad backlog. This effectively puts ad-hoc requests and scheduled projects on equal footing, rather than secretive things analysts do in their spare time that cause their core work to be late.
The advantages here are:
Olivia Tanuwidjaja furthers this idea by suggesting implementing an ad-hoc roster schedule on your team:
“The way it works is for every week (or 3 days, or 2 weeks, etc — you decide), one team member is in charge to handle the ad-hoc requests coming in that week. This ensures you’re only working on ad-hoc requests for your roster week and can focus on other projects when you’re not on duty.” [4]
Getting the right information for each ad-hoc request is crucial to deciding what to do about it. We received many suggestions about what the right questions to ask are, and what your questions should be achieving, but here are the three highlights that emerged:
[6.1] Why now?
Ad-hoc requests are often stressful because they come with an implicit urgency. Everything feels like it was needed yesterday. It is therefore an imperative question to ask not just why this request is needed, but why is this needed now.
[6.2] Make sure your questions help you find alignment.
Will Sweet reminds us that asking questions is not purely for gathering requirements, but is also a crucial step in establishing alignment.
“Finding common ground builds empathy, and there’s no better way to communicate with someone then to align your goals and theirs." [1]
Making sure your questions help reveal not just the requestor’s motivations, but also your own, is fundamental to establishing alignment, and thus, trust.
[6.3] Make it about actions
Cassie Kozyrkov, Chief Decision Scientist at Google, encourages an action-orientated approach when it comes to acting on an ad-hoc request (or indeed any data request). This emphasis helps remove emotion and bias from a request and ensures that you and your team are only working on things that will affect the actions the business will or will not take.
“If you don’t have a default [action], you don’t need fancy statistics.” [5]
Asking the right questions is only helpful if you can use them to act, and sometimes the action you will need to take is to say No to an ad-hoc request.
Will Sweet offers several tips on how to say ‘No’ to your business, most of which do not actually involve saying ‘No’, but maybe ‘not right now’ or ‘not quite in that way.’ Regardless, applying friction to these requests is essential in ensuring your team is working on the right requests at the right time. [1]
Amanda Sjöstrand Henriksson echos this sentiment, however focusing on the need for data teams to lead business requestors into the right approach to their problem.
“Your stakeholders probably do not know ways of working in an analytics team and what is an ad hoc request or not. It is the analysts job to guide the stakeholder - they only come to you with a problem. And if you see it fitting the ad hoc framework or if it rather should be a big project - that is your decision and responsibility. So guide the stakeholders!”
Once you have a clear understanding of what is required, the urgency, and the outcomes of the request, it is important to choose the right approach.
Ryan Dolley refers to this step as ‘presentation-purpose fit’, which means are you choosing the best tool for what you need to communicate.
“If you fail to build a delightful front end, all of your backend work may be wasted.” [3]
Phuong Nguyen, Growth, BI at ZaloPay furthers this by reminding us that:
Sharing the results of an ad-hoc request is about more than the tool and output you choose; it is also about how you communicate what you’ve found.
Will Sweet breaks down the three components of effective communication [1]:
Depending on the request, you may need to emphasize some of these more than others. Big decisions, or actions, for example, may require more methodology as they require more trust to be established before moving forward.
This is one of the most common problems faced by modern data teams. If there were a quick fix, we would know it. Ad-hoc requests are a messy and complex problem, so don’t get discouraged if this doesn’t go away overnight. Keep an open mind, keep iterating, and things will improve.
We all know the importance of addressing ad-hoc requests quickly, but also not wasting time on answering the wrong questions. Count is designed to help data teams deliver the value of data at scale, and that extends to ad-hoc requests by:
[1] The why and how of saying no ft. Will Sweet, MDS Fest 2023. August 24, 2023. Video recording.
[2] "Please, no ad hoc data requests anymore". Guillaume Ansanay-Alex. 5 Sept 2023. Data Partners. Article link.
[3] How to Build an Analytics Front End that Doesn't Suck: Presentation-Purpose Fit. Ryan Dolley. 7 March 2023. Super Data Blog. Article link.
[4] Data Analysts's guide in handling flooding data ad-hoc requests. Olivia Tanuwidjaja. Aug 15, 2023. Towards Data Science. Article link.
[5] Never start with a hypothesis. Cassie Kozyrkov. Nov 30, 2018. Towards Data Science. Article link.
[6] Tips to survive with ad-hoc questions. PhuongNDC. November 5, 2022. Medium. Article link.
.png)
.png)