McKinsey Is Right: AI Needs Context. Almost No One Has It.

There is a moment in every technology cycle where the language settles before the understanding does. We are in that moment now with AI.

Everyone agrees on the direction. Even McKinsey & Company in their The AI revolution in software development article has made it explicit: the quality of an AI system is fundamentally tied to the quality of the context it operates on. That idea has moved quickly from insight to assumption, shaping strategy at the highest levels. But agreement on direction often hides a deeper problem.

We are using the same word “context” to describe fundamentally different things. And that gap between language and reality is now showing up in production systems. McKinsey pointed to the problem. They did not define the solution.

Context Became a Placeholder

In practice, “context” has become a placeholder. It absorbs meaning without requiring precision. When teams say they are improving context, what they are usually doing is expanding the surface area of information available to a model:

• more documents indexed 
• more embeddings generated 
• broader retrieval windows 
• richer prompt construction 

Each of these moves feels directionally correct. Each is measurable. Each can be presented as progress. But none of them answer the underlying question: What makes context useful to a system that is expected to produce decisions, not just outputs? Because usefulness is not a function of volume. Context is not volume. It is structure.

The Shift from Information to Meaning

There is a distinction most architectures blur. Information answers: What is there? Context answers: What does it mean in relation to everything else? A system can retrieve highly relevant information and still fail to produce a correct or consistent outcome, not because the data is wrong, but because the system has no reliable way to understand how pieces of information relate across multiple steps.

Retrieval systems are optimized to find proximity, semantic closeness, statistical similarity, and lexical overlap. They are very good at assembling fragments that look like they belong together. Retrieval finds proximity. It does not create understanding. Understanding emerges from relationships. And relationships require structure.

Where It Breaks

At small scale, this limitation is easy to miss. A model retrieves a handful of documents, produces a plausible answer, and the system appears to work. The underlying assumptions remain invisible. But as systems are pushed into real environments including multiple entities, multiple data sources, evolving relationships, real-time decisions, the cracks become structural.

You begin to see patterns:

• answers that shift depending on what was retrieved 
• reasoning that cannot be reproduced or audited 
• outputs that are locally coherent but globally inconsistent 
• cost increasing faster than accuracy 

These are not edge cases. They are signals. Most AI systems do not operate on context. They operate on fragments of information, loosely assembled and treated as if they were understanding.

What McKinsey Pointed Toward

McKinsey’s observation is directionally correct: context determines outcome quality. But improving context is not a retrieval problem. It is a representation and reasoning problem. It requires a system to do three things reliably:

1. Represent entities and their relationships explicitly 
2. Traverse those relationships across multiple steps 
3. Preserve the structure of those relationships as part of the reasoning process 

Without those capabilities, context remains fragmented, no matter how much information is added.

The Difference Between Knowing and Understanding

This is where most approaches collapse into a subtle failure. They assume that assembling the right pieces of information is equivalent to understanding. It isn’t. Two facts, on their own, are just facts. Connected through the right relationship, they become an explanation. Connected across multiple steps, they become a decision. Similarity can suggest relevance. But only relationships establish meaning. That transition from fact to explanation to decision is not driven by volume. It is driven by structure.

The Cost of Getting This Wrong

When context is treated as a retrieval problem, the natural response to uncertainty is expansion. More data. More documents. More tokens. More compute. Each addition feels like a step toward completeness. In reality, it increases the number of possible interpretations the system must evaluate. The system is doing more work to resolve ambiguity that should not exist. That is why cost does not scale linearly. It compounds. This is not just a scaling issue. It is a structural inefficiency. And it is already showing up in production systems.

Context as a System, Not an Input

To move beyond this, context must be treated differently. Not as something appended to a prompt or retrieved at query time, but as a first-class system layer that defines how information is organized and traversed. That layer must: make relationships explicit, enable those relationships to be navigated dynamically, and preserve the path taken so outcomes can be understood and not just generated  At that point, context stops being an approximation. It becomes the foundation for deterministic reasoning in systems that are otherwise probabilistic.

What Changes When Context Is Real

When context is structured and connected: the system evaluates fewer possibilities, not more, reasoning paths become traceable, outputs become consistent across similar conditions, and cost aligns with relevance instead of volume. The system does less work and produces better outcomes. This is not an optimization. It is a fundamentally different operating model.

The Real Takeaway

McKinsey is right. Context determines the outcome. But the real insight is not that context matters. It is that most systems today do not have it. They have proximity. They have fragments.
They have approximations. They do not have structured understanding. Context is not what you retrieve. It is the structure that determines how information connects and how those connections are used to reach a decision. Until that distinction is addressed, systems will continue to scale in cost while failing in consistency. And that is not a future risk. It is already happening.

The Missing Layer in Your AI Stack

Most AI stacks look complete. On paper, they have everything: 1. Applications; 2. Large language models (LLMs), 3. Vector databases, and 4. Enterprise data. It feels like a full system. It isn’t.

Because what looks complete is missing a critical form of understanding, one that no amount of data, retrieval, or model size can compensate for. Similarity finds data. Relationships explain it. And that distinction is where most AI systems break.

What’s Missing

 What’s missing is not more data. It’s not a better model. It’s not more retrieval. It’s a layer that understands how things connect, not just how they look in isolation. Call it what it is:

A Relationship Runtime.

A layer that computes connections directly, instead of asking the model to infer them on every request.

Why Similarity Isn’t Enough

 Vector systems retrieve based on similarity. That works for locating information that appears related to a query. It is effective for search. But enterprise decisions are not driven by similarity. They are driven by structure.

Similarity tells you what is near. Relationships tell you what matters.

A transaction is not risky because it looks like another transaction. It is risky because of how it connects to other accounts, to shared devices, and to coordinated behavior over time. Those are not similarity problems. They are relationship problems. In practice, most AI stacks look like this: data → vector search → model → application. What’s missing is the step in between, where relationships are computed, not guessed.

Where the Stack Breaks

When relationships are not explicitly represented, the system does not fail immediately. It compensates. The LLM is forced to: infer connections between entities, reconstruct context from fragments, and resolve ambiguity across loosely related documents,. This work is pushed upward into the model because there is nowhere else for it to happen. And that is where the inefficiency begins.

What the System Actually Does Without Relationships

Consider a simple question: “Is this transaction fraudulent?” A similarity-based system retrieves transactions that look alike, users with similar patterns, and documents describing related behavior. But similarity does not assemble a network. So, the model is forced to construct one. It has to: infer that two accounts are connected through a shared device, recognize that the same device appears across multiple identities, and identify that those identities form a coordinated pattern over time

None of that structure is given. It is reconstructed from fragments inside the model. On every request. That is not retrieval. That is repeated graph traversal happening in the most expensive layer of the system.

The Real Consequence

Without a relationship layer: context is fragmented because connections are implicit, signals are incomplete because paths are not surfaced, and decisions degrade because the system never sees the full structure. The system compensates in the only way it can: 1. It retrieves more, 2. Processes more, and 3. Guesses more.

More context doesn’t create clarity. It creates more work. And that work shows up during inference, where it is most expensive and least scalable.

The Hidden Cost of Missing Relationships

 When relationships are not resolved ahead of time, they are resolved during inference. That means more tokens to represent possible connections, more attention computation to evaluate them, more latency before an answer is generated, and more GPU time consumed per request. But more importantly, it means the same work is repeated. Every query reconstructs the same relationships. Every request pays for the same reasoning. Over and over again.

 If relationships aren’t resolved before the model runs, the model pays to figure them out every time.

Why This Becomes Inevitable

 This is not an optimization problem. It is an architectural one. As systems scale: more data is integrated, more workflows depend on AI, and more decisions are automated. The amount of relationship inference pushed into the model grows continuously. And with it: cost accelerates, latency becomes unstable, throughput declines, and infrastructure demand increases. Most teams don’t hit a model limit. They hit a context limit first.

Where TigerGraph Becomes Necessary

 This isn’t about adding a capability. It’s about moving a class of computation to where it belongs. Today, relationship traversal is happening inside the model, where it is: expensive, repeated, and opaque.

That is the mistake.

TigerGraph moves that computation into a system designed for it. By representing data as a graph, it resolves entities and relationships explicitly and can traverse multi-hop connections in real time. Instead of sending large volumes of loosely related data into the model, it returns a precise, connected structure of the relevant facts. The model no longer has to discover how things relate. It is given directly.

Stop asking the model to discover the graph. Give it the graph.

What disappears is not just excess context. It is entire classes of unnecessary computation: repeated relationship inference, attention over irrelevant data, and redundant reasoning cycles

From Missing Layer to System Efficiency

 This is why the Relationship Runtime matters. Without it, the system relies on expensive, repeated inference to reconstruct relationships. With it, relationships are resolved once efficiently and reused across queries. That shifts work out of the most expensive layer in the stack and into a system purpose-built for it. And that is what makes the entire system scalable.

The Real Takeaway

Most AI stacks are not incomplete because they lack models or data. They are incomplete because they lack a way to compute how data connects. Without that, every request rebuilds the same relationships from scratch inside the most expensive layer in the system. That doesn’t just increase cost. It defines it.

The systems that scale won’t be the ones with the largest models. They will be the ones that stop doing the same work twice.

Knowledge Graphs as the Missing Context Layer for AI

A growing number of enterprises are discovering the same problem at the same time. Their AI systems respond quickly, generate fluent output, and sound confident, but they’re confidently wrong. 

These systems struggle the moment a workflow depends on understanding rather than prediction. The model can retrieve information, summarize it, or generate an answer, but it cannot determine whether the answer reflects how the business actually works.

This gap becomes visible in every environment where context determines correctness. 

• Fraud patterns hinge on shared devices, merchants and accounts. 
• Supply chains depend on upstream and downstream dependencies. 
• Customer intelligence requires unifying fragmented records. 

AI cannot see the structure behind these signals to follow relationships or interpret what connects one event to the next.

A knowledge graph fills that gap, providing the structural context and logic that ties the signals together. 

This article explains how a knowledge graph informs graph-powered AI, and how the graph improves retrieval, reasoning and trust.  

Understanding the Role of a Knowledge Graph in AI

A knowledge graph models real-world structure. It creates an information foundation to use when generating answers, evaluating risk or supporting operational decisions.

Organizations adopt knowledge graphs because modern data is inherently interconnected. 

• Customers move across channels. 

• Devices share upstream dependencies. 
• Transactions form chains. 
• Risks propagate through multi-step patterns. 

These relationships cannot be expressed reliably in isolated tables. A graph data model captures these connections directly, preserving the meaning that AI systems require.

A knowledge graph is not an overlay. It is the structural source of truth that grounds AI behavior in the actual shape of the business.

A model predicts and approximates, and the knowledge graph explains and validates. 

This is why organizations are adding a graph layer beneath their AI stack. It replaces guesswork with connected, verifiable data.

Why AI Requires a Context Layer?

Large language models perform well in natural language tasks, but they are probabilistic engines. 

They generate plausible answers by identifying statistical similarity, but they do not validate these answers against authoritative data. This creates several operational gaps that appear immediately in enterprise systems:

• Limited understanding of multi-hop relationships
  • No transparent explanation of how an answer was formed
  • Susceptibility to hallucinated or unsupported claims
  • No mechanism for verifying retrieved information
  • Difficulty interpreting the meaning of connected events

AI without context behaves as an isolated predictor, operating without a mechanism to confirm whether its output aligns with enterprise reality. It can produce output, but it cannot confirm whether that output aligns with enterprise reality. And this is where a knowledge graph becomes essential.

The graph provides structure, logic and relationships. It gives the AI system a grounding mechanism for deductive reasoning, rather than relying solely on inductive prediction.

A context-aware AI system can justify its answers, trace the logic behind them, and operate with greater reliability. The knowledge graph provides the connective layer required to make this possible.

How Knowledge Graphs Enhance AI Retrieval?

Modern AI systems increasingly rely on hybrid retrieval, because no single retrieval method captures both meaning and structure.

Vector embeddings capture semantic similarity. A graph database provides structural precision. TigerGraph supports both approaches within one workflow so organizations do not have to choose between linguistic relevance and factual accuracy.

What Are Vector Embeddings?

Vector embeddings represent text, images, or other complex information as high-dimensional numerical arrays. Instead of matching exact keywords, a vector model measures meaning. It retrieves content that is conceptually related. 

For tasks involving language, summarization, or intent understanding, vectors provide an efficient first layer of recall.

However, semantic similarity alone is insufficient in enterprise environments. A vector may retrieve text that “sounds” relevant but is contextually unrelated, outdated, or inconsistent with the organization’s actual data. Vectors are useful but naive.

AI systems need a mechanism to confirm whether retrieved information matches real entities, relationships, and business logic. This is the role of the graph.

Deconstructing a Graph Database Workflow

A graph database helps determine whether the semantically similar results retrieved by an AI system make sense in the real world. It can show:

• how the entities involved are connected
  • whether the dependencies match the patterns the business knows to be true
  • and whether the implied relationships reflect how the system actually behaves

This step acts as a filter. It removes results that look relevant in language but fall apart once structure is considered. It keeps retrieval grounded in verified, authoritative data instead of statistical probability alone.

This workflow forms the basis of GraphRAG, where structured graph context is assembled before the model generates an answer.

Demystifying GraphRAG

GraphRAG is an extension of retrieval-augmented generation that incorporates graph structure into the retrieval process. Traditional RAG retrieves documents based on vector similarity. GraphRAG adds a layer of structural reasoning so the system retrieves not only linguistically relevant information, but information that is contextually and relationally accurate.

In a GraphRAG workflow, the knowledge graph becomes the source of truth for context assembly. 

The system begins by identifying the entities, attributes and relationships associated with a query. It then traverses multi-hop paths to understand how those elements connect, whether they share dependencies, and which subgraphs are relevant to the task.

This produces a structured context package. Instead of receiving unfiltered content, the LLM receives a graph-derived snapshot of the domain, including the entities involved, the relationships that bind them, the constraints that govern them, and the pathways that shape their behavior. This grounding enables the model to work with accurate context rather than probability alone.

GraphRAG is not a separate model or a different form of generation. It is a retrieval architecture that ensures the LLM is guided by authoritative structure. 

By integrating semantic reach from vectors with the structural precision of a graph database, GraphRAG offers retrieval that is more consistent, more explainable, and better aligned with how real systems operate.

Graph-powered AI System Operation

A graph-powered AI system goes beyond vector similarity alone. It works with the actual structured knowledge of the business and uses that structure to guide retrieval, validation, and reasoning. This enables it to perform tasks that conventional retrieval pipelines cannot.

• Follow explicit multi-step relationships.
  The system traces chains of connected entities. It could connect customers to accounts to devices to merchants, or suppliers to components to facilities. This mirrors how events occur in practice rather than how they appear in unstructured text.
• Evaluate dependencies spanning several hops.
  It can determine whether separate events share a cause, whether they influence one another, or whether a change in one area will have downstream effects. This provides context that statistical models cannot infer on their own.
• Identify relevant subgraphs for reasoning.
  Instead of pulling broad, generic content, the system extracts the precise slice of the enterprise graph needed for the task at hand. This keeps retrieval focused, efficient, and aligned with the organization’s domain.
• Validate retrieved facts against an authoritative structure.
  The graph acts as a guardrail. It checks that retrieved information matches known relationships before the model uses it. This helps the model reject contradictory, incomplete or irrelevant data.

Together, vectors provide semantic reach, while the knowledge graph supplies structural grounding. The combination creates a more accurate retrieval workflow, one that is less prone to hallucinations, and aligned with how the enterprise actually operates.

Knowledge Graphs Strengthen Explainability

A knowledge graph naturally strengthens explainability. It makes reasoning paths transparent and auditable. 

When an AI system produces an output, the graph can show which entities were involved and how they were connected. It also shows which relationships influenced the answer and how multi-hop logic contributed to the outcome.

This is essential for banking, healthcare, insurance, supply chain, and customer-facing environments.

A graph database ensures AI-driven decisions remain traceable, governed, and reviewable. It provides a direct, inspectable logic path that traditional AI systems cannot produce on their own.

Building Graph-Powered AI with TigerGraph

TigerGraph supports real-time context across large, complex enterprise environments by delivering the performance, scale, and clarity required for enterprise-grade graph-powered AI. Its architecture evaluates multi-hop relationships in real time, enforces schema consistency, and supports high-load analytics across large, connected datasets.

To summarize, TigerGraph strengthens the AI stack with:

• Real-time graph analytics for immediate insight
  • Schema-driven modeling for reliable interpretation
  • High-performance traversal for large subgraphs
  • Graph and vector search for hybrid retrieval
  • Consistent structure for various applications and business domains

TigerGraph enables contextual AI systems to reason over relationships instead of relying on probability-based predictions.

Best Practices for Implementing a Knowledge Graph

Here are several design principles for organizations build a knowledge graph:

1. Model entities and relationships that directly reflect business logic.
2. Maintain a schema that enforces clarity, consistency and long-term stability.
3. Validate relationship direction, type, and cardinality early in development.
4. Track lineage for attribute changes to support governance and versioning.
5. Integrate vector search only after structural accuracy has been established.

Why is this order crucial?
Because, as mentioned, vector search uses semantic similarity, which is powerful but also imprecise. If you add vector retrieval before you have a stable structural model, the system will retrieve items that feel relevant in language but do not fit your business logic. Without the guardrail of a well-defined graph, those errors spread quickly.

Once the graph structure is correct, vector search becomes the “semantic expansion layer.”
But the graph remains the source of truth. It’s the filter that prevents hallucinations and incorrect associations.

A well-designed knowledge graph is an enterprise asset that improves every analytical process. 

Summary

AI excels at language but struggles with structure. A knowledge graph provides the relationships, context, and multi-hop reasoning that modern AI systems require. TigerGraph supports this architecture with a high-performance graph database built for real-time insight, explainability, and connected decision-making.

By combining generative models with structural intelligence, organizations can deploy AI systems that reflect the actual shape of their business and deliver answers grounded in accuracy, clarity, and context.

If your organization is evaluating how to build contextual AI that performs reliably at enterprise scale, TigerGraph provides the structural foundation required for consistent reasoning and explainability. Reach out today to learn more.

Frequently Asked Questions

1. Why do AI systems fail when business decisions depend on understanding relationships?

Most AI models operate on probability rather than structure. They generate answers based on patterns in language, not on how entities, events, and dependencies are actually connected. Without a relationship-aware context layer, AI cannot reliably interpret cause, impact, or sequence across complex workflows.

2. How does a knowledge graph help AI distinguish between relevant and misleading information?

A knowledge graph validates retrieved information against real entities and known relationships. This prevents AI systems from using data that appears relevant linguistically but conflicts with how the business actually operates, reducing hallucinations and false assumptions.

3. What types of enterprise decisions benefit most from graph-powered AI?

Decisions involving risk propagation, dependency analysis, entity resolution, or multi-step causality benefit the most. Examples include fraud detection, supply chain disruption analysis, customer intelligence, compliance investigations, and operational planning.

4. How does combining graphs with vector search improve AI accuracy?

Vector search expands semantic reach by identifying conceptually related information, while the graph confirms whether that information is structurally valid. Together, they ensure AI retrieves content that is both meaningful and factually grounded in enterprise reality.

5. What makes a knowledge graph a long-term asset rather than a one-time AI enhancement?

Once established, a knowledge graph becomes a reusable source of truth that supports search, analytics, AI reasoning, and governance across many use cases. As the graph evolves with new data, it continuously improves the accuracy, explainability, and trustworthiness of AI systems built on top of it.