Ollie Demo - "The AI Analytics Dilemma" - ChatGPT vs. Count

So the agent isn't, in this case, isn't just building text and giving you an answer. It's actually building objects that are editable by human, auditable by you, can be checked that it's right. Let's talk a bit about the challenges of rolling out AgenTix. And I'm going to do that with what I would call an anti demo. Now I'm going to show you actually a kind of an analysis of a CSV file using ChatGPT. There's nothing against ChatGPT. We obviously, we use in sometimes we even use the underlying models. But I wanted to show you what we know is challenging about rolling out agentic analytics by just making it a very clear demo. So here we have a CSV of customers. And I'm gonna ask Chachibi to help me build an ideal customer profile off this. So I'll just ask her the question. Hey, could you take the CSV file and help me build out from it what could be our ideal customer profile based on all the contracts you can see here? Great, okay. A relatively meaningful exercise. So this is basically as you can see here, a CSV of lots of very sensible contracts, but with customers, their size, their touch date. And now we've got Chatbot bringing back its answer. And it's done it pretty quickly, as you can see it's building out. It's given us a bit of a sense of criteria, which is very helpful. And it's given us an answer. It's suggesting a company size of fifty to four hundred, sweet spot, funding stage, industries, something about their setup as well, which is very helpful. The challenge here is, how do we know this information is accurate? This is just simply text. I have no way to understand if this information is correct, if that stat is even right. Haven't got an ability to really interrogate this methodology. It's given me a sense of what the answer is and some detail, but hasn't told me how it actually got about doing it. And also more importantly, how do we govern this? How do we define concepts like ICP in a larger scale or even what IRR is? And then the other part of this is, this is obviously just my chat. This is just what I have been engaging with. I have built this with a CSV file and the results of this, where does it go? How do we make this into a thing that the whole company can work with and get value from that we can align the business around? And this is what makes it quite challenging to roll out Agentyx across the organisation is it doesn't necessarily help us, there's a magic box here, but how we implement that magic box really matters to get the maximum value and to build the maximum trust. What we don't want is something like this, where we have everyone working in their own little cubicle, own chat thread, getting answers to their own personal questions, but there's no ability for them to align, to collaborate, to work together, to reach consensus and actually make value out of the decisions other than what's relevant to just the individual. So let's move out. I'm gonna now bring you into, from the presentation into the count canvas properly where we can have our agent. And what I'm using here is actually the same CSV file we use in Chatcha BT. The same kind of list of customers and their contracts that we had in Chatcha BT. And I'm gonna ask exactly the same question of our agent in here as I did Chatcha BT. So I'm gonna ask our agent, which is this kind of dot here with this area it works in, to answer that same question. Hey, could you help me identify our ideal customer profile from this list of customer contracts? So here we can see that the agent is executing. This is its chain of thought just like in ChatGPT. And the power we've got here is that this CSV file and the answer it's giving here is fully auditable. So the agent isn't, in this case, isn't just building text and giving you an answer. It's actually building objects that are editable by human, auditable by you, can be checked that it's right. And that makes it very helpful, very easy to iterate with your stakeholder, the agent and an analyst checking and verifying its work. So it's doing an answer here, it's still working, but perhaps we've an ideal customer question here. But actually if you look at this dataset, it doesn't actually include anything about location. And so one of the really great things you can do very quickly, just to show how easy this is, you can bring in another data set which actually is for each customer, its location, country, and lat long if you wanted to plot it on a map. And I can then just add this to another agent very quickly and say, right, I want you to look at customers again, but I also want you to look at location as well. Could you please give me the ideal customer profile based on these dimensions of location, but also looking at the funding statistics, the demographics of the company and the contract size and expansion. So what I'm trying to show you here is like the ability to very quickly build queries with a stakeholder, getting them to tell you whether the outputs are useful, looking at the data with them, seeing if the data's answering the questions they want, checking the data's correct, checking that we trust the analysis that it's actionable, and then making quick edits, either with documentation or simply just by adding in new fields into the agent's context very early on, is an incredibly powerful way to make that embedding phase get going in a very quick way. You're basically building to demand of the stakeholders, rather than building up waterfall and hoping the final output you give them is useful. This way you're working with them, you're modeling better, they're feeding back information better, and you're both wrestling with the results it's providing back with the agent to make sure accuracy and trust is there. And you're doing that in a much more iterative approach, which gives both sides more accuracy. So as you can see here, our agent has finished off its analysis. I think it's actually realized a similar ish outcome to Chatuchi BT, but it's giving us a lot more data methodology about how it's got there. It's given us a full auditable approach and shown every single piece of analysis it's done to reach the conclusion. So we have a much more information to verify or tweak or ask for follow ups if you wanted to go further. And we've got another agent doing the same piece of work here, but looking at geography, which suggests that the US is in a very important geography for this particular company's ICP.