Interviewer: Looking ahead to the next 12 months, what AI trends are you excited about?

Joel: One of the big trends we’re seeing, and I’m not sure how excited I am about it, is the focus on token yield or token efficiency. This is a shift from simply adopting AI or burning through tokens to thinking about how efficiently tokens are being used and the value they deliver.

It’s part of the broader transition from experimental AI environments to full production environments. Companies are moving from AI-specific budgets, which are often experimental, to operational budgets where there’s an expectation of measurable returns and impact on business metrics.

Token yield is about understanding how much you’re spending on tokens and the value you’re getting in return. This is particularly relevant as many large organisations move from proof-of-concept stages to full AI deployments. Some are encountering unexpected costs, and I think the trend over the next year will be about using AI wisely and efficiently to achieve the best outcomes for organisations.

Interviewer: Beyond the financial challenges with tokens, are there other challenges that companies and leadership are facing? You mentioned building at scale earlier—what are the key challenges leaders are dealing with?

Joel: Leaders in IT who are tasked with deploying AI often become overly focused on choosing the right models. There’s a lot of excitement around the latest AI models from major providers, but this can overshadow a critical component: feeding those models the right information.

For AI to deliver value in an enterprise context, it needs access to the right data. This is a longstanding challenge—organisations have scattered information across SaaS tools, structured and unstructured data repositories, and systems of record. The first step is to unify this data.

Once that’s done, the next challenge is selecting and using the right large language model (LLM) wisely. Organisations need to keep their options open as the LLM landscape evolves, while also ensuring compliance with enterprise security and governance requirements. This includes having guardrails around change management and ensuring that AI deployments meet security standards.

Interviewer: How has the conversation around AI shifted in recent years, particularly at events like The AI Summit?

Joel: The conversation has shifted significantly. In the early days of GPT, AI was more of a desktop tool—something individuals would experiment with on their own. Now, the focus has moved to work AI, where the goal is to improve workflows for entire teams.

When AI is deployed at the team level, enterprises see an order of magnitude greater benefit. And when it’s rolled out consistently and governed across the entire organisation, the benefits multiply again.

Over the past year, we’ve seen a shift from individual experimentation to thinking about AI at the departmental or team level. The next wave will be about deploying AI across the entire organisation. This requires unified data and an intelligence layer that enables AI to assist across departments.

For example, connecting the sales team’s interactions with clients to product development in real time has long been a dream for many organisations. AI can make this a reality. Similarly, connecting feedback from support teams to product delivery can unlock new opportunities. This kind of cross-departmental integration takes productivity from the individual level to the team level and ultimately to the company-wide level.

Interviewer: What has your experience been like at The AI Summit London? Has it helped further your conversations? What’s your biggest takeaway?

Joel: My biggest takeaway from The AI Summit London is the scale at which large companies are deploying AI. It’s incredible to see the size of these organisations and the scale of their AI initiatives.

However, many of the tools being used weren’t originally designed for enterprise use. They’re being adapted or modified to meet enterprise requirements, which highlights a gap in the market. There’s a real need for AI tools that are purpose-built for enterprise environments, rather than repurposed from consumer or experimental tools.

Interviewer: If you could give me one key takeaway or focus area within AI, what would it be?

Joel: If I had to narrow it down to three key takeaways, they would be:

• Context: AI needs access to the right information to be effective. Just like a new hire or intern, an AI system can’t perform well without the necessary context about the organisation. 
• Intelligence: Be smart about your AI models. Choose the right model for the task, understand your model usage, and ensure you’re getting high-quality results for your investment. 
• Security and Governance: This is non-negotiable. AI must be deployed in a secure and compliant way, with guardrails around permissions and data access.

At Glean, we focus on these principles. By providing the right context to LLMs and selecting the appropriate models, we’ve been able to reduce token usage by 30–50% while delivering results that are 2.5 times more preferred by our clients. This translates to better outcomes for businesses, both in terms of efficiency and financial performance.

Conclusion

Joel McKelvey’s insights highlight the critical factors for successful AI adoption in enterprises: context, efficiency, and governance. As VP of AI Business Value at Glean, Joel helps organisations navigate the complexities of AI implementation, ensuring that their investments drive measurable business outcomes. 

With decades of experience in AI, machine learning, and data architectures, Joel is at the forefront of helping enterprises move from experimentation to execution, unlocking the full potential of AI to transform productivity, decision-making, and customer experiences.