Avi Perez, cofounder and CTO, Pyramid Analytics, says that in order to unlock the value of business intelligence AI, data teams need to first prepare their databases, their approaches to AI governance, and the way they interact with other stakeholders.
AI chatbots that deliver business intelligence capabilities are soaring in popularity right now. Every department is eager for more accurate and reliable forecasting, along with faster and richer insights to support their decision-making and strategic planning.
It’s almost an arms race among enterprises: whoever gains meaningful insights first gets a head start at mitigating emerging risks and seizing nascent opportunities.
There’s barely a use case that doesn’t benefit from them: sales teams track product demand trends, marketing teams access campaign performance metrics, customer support teams automate simpler customer inquiries, HR teams monitor employee engagement and productivity, and manufacturing teams optimize process efficiency.
According to research by BARC, AI-enhanced BI is expected to drive faster time to insight and a reduced workload, among other advantages. It’s not surprising that this shiny new tool is on everyone’s wish list.
But implementing an AI chatbot for business intelligence requires more thought than ordering socks from Amazon. If you rush into the project without laying the necessary groundwork first, you’ll risk encountering significant challenges.
We’re talking anything from unreliable outcomes that skew your business decision-making, to serious penalties for non-compliance with privacy regulations and industry standards. So before you implement an BI chatbot, check that you’ve covered these three critical bases.
Get your house in order in terms of data quality
Data quality is the single most important ingredient for successful generative BI. Because AI tools rely on vast volumes of complex data to generate answers, there’s a higher likelihood of poorly-defined data polluting the entire pool. What’s more, small errors in a training dataset can be amplified as they are copied across the system. In extreme cases, this can result in model collapse.
You need to ensure that data governance policies are not just in place but enforced. Implement robust rules controlling data collection and storage, data management, and data ownership and stewardship. Only clear data trails can assure data integrity and deliver reliable data that’s been cleaned correctly and processed consistently.
In a similar vein, keep an eye on inter-departmental differences around data enrichment and how they calculate the metrics they rely on. Different teams have different objectives, so it makes sense that they’ll take varying approaches to enrichment and metrics. They might start off with the same datasets, but apply different weights, labels, and sources as they process it, resulting in multiple versions of truth that eventually lead to inconsistent and unreliable outcomes.
Make sure that third-party AI services don’t have access to your full data
When you set up your infrastructure for a BI chatbot, it’s tempting to connect a third-party large language model (LLM) directly with your business databases. After all, if the LLM has access to the most up to date data, you’ll get more accurate insights, not to mention a seamless system that’s faster to respond.
But that would be a big mistake, primarily because of security. There’ve been many headlines about public GenAI models leaking proprietary data from prompts, and even if nothing links, it’s possible for unauthorized parties to draw conclusions based on sensitive information thanks to the queries that users put in.
Some LLM developers are open about utilizing user inputs to use for model training, which increases the risk of data breaches and leaks over time.
Even if your data uploads are siloed, giving third-party AI services access to your proprietary information is asking for trouble. Many regulations, including HIPAA, GDPR, and CCPA, have stringent requirements around cloud server usage and access to sensitive customer data. Simply connecting a database with an externally hosted AI tool would be considered non-compliance, inviting penalties and fines.
Prepare your data team to become enablers
Bear in mind that your systems, data, and infrastructure might all be ready for BI chatbots, but that your employees may not be. One of the big benefits of chatbots is that they enable self-service insights, but ordinary, line-of-business users who don’t have a data analytics background are unprepared for this step.
It’s not just that they don’t know how to code their queries, because a chatbot overcomes that by allowing them to ask natural language questions. It’s that they lack a data analytics mindset, and sometimes even lack basic data literacy. They may not know how to turn questions into queries that deliver useful answers, how to word them in ways that chatbots can understand, or which visualizations to request in order to gain the insights they need.
That’s why your data team has to be ready and waiting to support their line-of-business colleagues. They are going to have to step out of their usual role to serve as guides to BI thinking. Prepare them to educate your other employees, correct their mistakes, and explain how to get the results they’re looking for.
They’ll also need to supervise the chatbot for hallucinations and flaws and train their colleagues to be aware of the ways that inaccuracies and bias can creep into AI-powered BI insights.
BI chatbots require careful preparation
Implementing a BI chatbot can be a smart move for your company. It can speed up time to insights, support new products and services, and drive better business decision-making. But you’ll only be able to harvest those benefits if you lay the groundwork first. Making sure that your data and employees are ready, and maintaining strict standards for access, are vital for successful BI chatbot adoption.
