Why Gen AI Keeps Failing Consulting

Consulting has a long history of helping companies across every industry imaginable. Along the way, it has accumulated an enormous amount of knowledge. From regional banks to national retailers, every engagement adds another layer to the understanding of how businesses operate. That accumulation is the edge. It’s the reason clients call. The question is how to use that edge.

Many clients now expect complex questions to be answered at the speed of an AI chatbot. Meeting that expectation depends on how quickly you find and apply the company’s knowledge. Over 25 years, your team has built up engagement notes, playbooks, and hard-won judgment. However, most of it is buried across shared drives, old case files, and the minds of people who have since moved on. What firms need is a way to break down those silos and put their collective knowledge to work in time to meet the client’s request. 

There are two questions every company should consider. Do you buy off-the-shelf solutions and stitch them together in-house? Or do you build your own, knowing that means years of investment, new hires, and no guaranteed ROI? Both come with a bill, but neither feels right.

In this article, we’ll explore this buy vs build dilemma and see each one’s benefits and downsides. We’ll also introduce a win-win scenario where you can reap the benefits of both approaches.

The illusion of buying “high precision” AI

AI is overhyped right now and the pressure to act is real. Clients are asking, competitors are announcing new capabilities, and boards expect a plan by next quarter. Many firms have responded by buying enterprise access to Gen AI models/tools and rolling them out at scale. Then they waited for the productivity gains to follow.

Unfortunately, this approach has disappointed on five fronts:

• AI doesn’t know your firm. Generalist models/tools are trained on public data. Your methodologies, project history, and regulatory boundaries aren’t in there.
• Verifying every output defeats the purpose.  Generic AI hallucinates often, so your staff must carefully check each answer. This makes the efficiency gains rapidly disappear. Even then, if one fabricated claim slips through the quality check, it can cost the firm millions in fines and client relationships built over a decade, or both
• Your data leaves the building. Most third-party tools process prompts on external servers, and depending on the vendor, that data can end up in future model training
• There are hidden costs. Scaling a proprietary platform across thousands of employees increases the price, the technical maintenance, and the cost of switching later
• Your differentiation flattens. If every competitor runs the same off-the-shelf AI tool, the methods and terminology you actually sell get averaged out

So if buying isn’t the answer, the next natural idea is to build it yourself.

Building a company-native AI

There are real benefits to having your own model and your own team to run it. The AI is yours, the data is yours, the control is yours. In theory, this solves many of the problems that come with off-the-shelf tools.

However, building a system that holds up in front of clients takes significant time, effort, and a lot of resources. It also requires a specific mix of talent that must be found, hired, onboarded, and aligned around a clear goal by experienced leadership. You’d need experts such as:

• Knowledge Management Experts
• Data Scientists / AI Engineers
• Otologists / Taxonomists
• Knowledge Stewards / Domain Experts
• Data Engineers
• Project Manager

Having so many people on board takes resources away from your main operations.

Then there’s the foundation itself. In most firms, data sits in silos, scattered across shared drives, different systems, and the unwritten expertise of senior staff. Before any AI can be useful, you have to pay this “bad data” tax and clean it up. 

Ultimately, you should build the foundational infrastructure that connects all your knowledge that links all those fragments together. Without it, the model isn’t reasoning but guessing from a void. Also, unlike a bought tool, there’s no vendor sharing the burden. Every guardrail for privacy, bias, and compliance is yours to build, monitor, and answer for.

The last problem is staying competitive. Technology becomes outdated in months. Custom pipelines need constant manual updates as changes to the models, data sources, and regulations happen constantly. Without an ongoing R&D effort, the system you built will decay and fall behind the ones your competitors have.

Both buying and building promise to utilize your company’s knowledge more efficiently. Buying tools off-the-shelf seems fast but it hallucinates regularly. Building your system also however requires vast amounts of resources and time with no promise of ROI. 

The good news is that there’s a third way that combines the best of both worlds.

Buy the building

There’s a third approach that borrows the best of both without their worst tradeoffs. 

The reason why AI doesn’t work well is because it does not use a proper infrastructure as a foundation to learn from. That’s the part that takes years to build and must be maintained. However, this has nothing to do with what makes your firm valuable.

Graphwise solves this problem. The graph database, the retrieval engine, the security and governance layer, are already built. Graphwise runs it, updates it, and keeps it current as models and regulations change. Your team doesn’t have to hire the specialists, build the infrastructure, or maintain it every quarter.

Your data is the only part you have to bring. This includes your documents, your methodologies, the way your firm actually thinks. The platform runs in your own private cloud or on-premises, and in many cases the data doesn’t have to move at all. Graphwise connects your fragmented silos into a contextually rich semantic layer without requiring data movement. Your context stays inside your walls. The AI simply learns how to interpret it.

The technical concerns you’d raise

Where does the data live? Wherever you want. Managed SaaS in an isolated environment, on-premises, or in your own private cloud on AWS, Azure, or GCP. In many cases you don’t even move the data. The platform links to your existing systems (SQL, Microsoft 365 systems, Adobe systems) in real time.

Who’s in control? You are. You define the access roles, the rules, the ontologies. Your data is never mixed with another client’s, and never accessible to 3rd parties. 

What about regulation? Graphwise is ISO/IEC 42001, GDPR, and NIS 2 compliant. Every answer comes with a traceability chain showing exactly which document or a fact came from.

What you actually get

Deployment in days or weeks – not the 12-18 months of a custom build.

95% accuracy – grounding the AI in a knowledge graph brings hallucinations close to zero.

80% cost savings – by cutting the bad data tax and reducing LLM token usage.

To wrap it up

Incorporating AI within your enterprise is a tricky business because so far you could either buy or build the infrastructure that supports it. 

If you buy it, you get a fast prototype but mediocre results. Gen AI is cheap, has a great UX and is maintained by somebody else. The price you pay is in the loss in control, uniqueness, data security. 

If you prefer to build your AI system you can expect dramatically high upfront costs plus the hurdle of managing a whole new department and technology. The price can be justified but only if you wait years to build the infrastructure, teach your AI, merge your siloed systems and most importantly keep your staff.

The best approach is to combine the two. When you buy the building you buy the expertise of people that have solved the hard problems already. You can balance between having the ultimate control of the system and owning your data but without having to manage a team of AI engineers and maintain the technology.

Do you want to explore how this is done in practice? See our White Paper