01 · Structured data
SQL execution
Use source metadata before generating a read query; execute against the configured database boundary and return rows plus query metadata.
- Inspect schema
- Choose joins + columns
- Generate query
- Execute + correct
Ask Titan Engineering Deep Dive · Part 1 of 7
How we built Ask Titan a governed enterprise AI assistant with LangGraph and MCP
Ask Titan routes business questions to governed capabilities for structured data, Power BI semantic models and enterprise documents. LangGraph coordinates the runtime; MCP provides a consistent capability boundary; application policy decides which tools reach the model.
LangGraphMCPDatabricks + SQLPower BI + DAXQdrant + RAG
Ask Titan runtime
governed capabilities
01
Question + identityResolve user and conversation
↓
02
Permitted capabilitiesDiscover, filter and bind tools
↓
03
Orchestrate + executeSQL · Power BI / DAX · documents / RAG
↓
04
Answer + execution evidencePersist state and source telemetry
The engineering problem
An enterprise AI assistant is not a ChatGPT-wrapper
A simple user → LLM → database → answer flow is enough to explain a prototype. It is not enough to run against enterprise systems. The application also has to determine which capabilities a user may access, which source should answer the question, what metadata the model may rely on, how failed queries are handled, how conversation state is maintained and how an answer can be traced back to the execution that produced it.
Prototype view
One model, direct access
User
→
LLM
→
Database
→
Answer
This hides authorization, source selection, query execution, state management and observability behind a single connection.
Ask Titan architecture
Separate responsibilities around the model
Identity + policy
Orchestration
Specialist execution
State + evidence
Ask Titan keeps these responsibilities in the application. The LLM receives only the capabilities and context required for the current request.
Design constraints
The architecture starts with constraints, not framework choices
We treated these as separate engineering problems rather than collapsing them into one agent prompt.
01
SQL ≠ DAX ≠ retrieval
Different sources require different metadata and execution semantics.
02
Available does not mean allowed
A capability can exist without being eligible for the current user or runtime mode.
03
Schema must precede query
Source metadata should be discovered rather than invented by the model.
04
Every loop needs an exit
Autonomy is useful only when deterministic runtime bounds surround it.
05
Context cannot grow forever
Persisted conversation and active LLM context are different concerns.
06
Answers need an execution trail
For data questions, the final text is not enough operational evidence.
Reference architecture
A governed multi-tool runtime around the LLM
Ask Titan is organized into five main layers: frontend, middleware, orchestration, MCP capabilities and enterprise data sources. Identity and capability policy determine what can be used for a request, while conversation state and execution observability are handled alongside the runtime.
One question end-to-end
Follow a question through Ask TItan's runtime
The diagram below follows one request through Ask Titan. The example asks: “Which customers caused our margin decline last month?” Ask Titan resolves the user’s access, routes the question to the Power BI capability, runs the required DAX against the semantic model and returns the result while retaining the relevant execution metadata.
Principle 1 · orchestration vs execution
The orchestrator decides what capability to use; the capability owns how the work is executed
Orchestrator responsibility
What should happen next?
- Choose among permitted capabilities.
- Maintain the active conversation state.
- Continue, call a tool or finish.
Capability responsibility
How is this source queried correctly?
- Discover source metadata.
- Generate source-specific operations.
- Execute, interpret errors and refine.
Simplified LangGraph pattern
graph = StateGraph(ConversationState)
graph.add_node("agent", agent_node)
graph.add_node("tools", tool_node)
graph.add_conditional_edges(
"agent",
route_next_step,
{"tools": "tools", "end": END},
)
graph.add_edge("tools", "agent")
graph = graph.compile(
checkpointer=conversation_memory,
)please note:The top-level model only receives the information needed for the current request. Schema details, DAX logic, retrieval behavior and connector-specific handling stay within the relevant capability.
Principle 2 · capabilities + authorization
Standardize the capability boundary. Then restrict what reaches the model
MCP gives the orchestrator a consistent way to discover and invoke capabilities. It does not decide whether a capability is appropriate for this user. Ask Titan resolves that application policy before tools are bound to the orchestrator.
Available catalog
MCP tools exist
Sales SQL · Production SQL · Finance Power BI · documents …
→
Application policy
Resolve permitted capabilities
Identity, entitlements and runtime configuration determine the eligible subset.
→
Bound tool set
Only verified tools reach the model
Unavailable capabilities are absent from the model's callable tool set.
Simplified capability filtering
available_tools = discover_capabilities()
policy = resolve_capability_policy(user)
permitted_tools = [
tool for tool in available_tools
if policy.allows(tool.name)
]
agent = build_orchestrator(
tools=permitted_tools,
state=conversation_state,
)Security boundary
Tool access is one layer
Identity
Who is making the request?
Capability authorization
Which Ask Titan tools may be bound?
Tool/source scope
Which configured tables, models or collections may the tool use?
Datasource authorization
What does the underlying source actually permit?
Important: application-level tool authorization is not equivalent to row-level or column-level security in a connected source.
Principle 3 · source semantics
One assistant. Three different execution paths
The architecture does not force every question through RAG. Structured databases, Power BI semantic models and enterprise documents are separate execution domains because they carry different metadata, business semantics and failure modes.
02 · Semantic data
Power BI + DAX
When measures, relationships and business definitions live in a semantic model, inspect that model and generate DAX rather than bypassing it with raw SQL.
- List model tables
- Load measure metadata
- Generate DAX
- Execute + refine
03 · Unstructured data
Documents + RAG
Rewrite the question for retrieval, fetch relevant document chunks from the configured collection and ground the answer in those sources.
- Rewrite question
- Retrieve chunks
- Add source context
- Return grounded answer
Architecture takeaway: RAG is a capability in Ask Titan, not the architecture of Ask Titan.
Runtime discipline
State, context and observability are related. But they are not the same thing
A durable conversation can be much larger than the context a model should receive on the next turn. Configuration also belongs outside the active agent loop, while execution evidence must survive the final response.
Context engineering
Persist more than you prompt
Thread checkpoints preserve conversation state. Older finished turns can collapse to user + final answer while the active tool interaction stays intact. Oversized tool payloads do not have to be replayed into the model.
Control plane
Configure outside the agent
Data sources, tool definitions, model/provider settings, prompts, semantic metadata, vector collections and user entitlements are durable configuration and not ad-hoc prompt content.
Execution observability
Keep the software trail
Retain tool calls and source telemetry around the answer. For database-backed capabilities that can include query IDs, SQL/DAX, duration, processed data and cost where available.
QUESTIONUser message IDanchor for the execution
→
AGENT + TOOLSIntermediate eventstool calls · arguments · source result metadata
→
ANSWERAssistant message IDlinks the response back to execution evidence
Observability is not chain-of-thought. The useful audit and debugging surface is the software execution path: tool calls, queries, timings and results. Private model reasoning is not part of that surface.
Failure handling and system boundaries
Failure handling is part of the architecture
Ask Titan has to handle more than successful tool calls. A capability can be selected incorrectly, a generated query can fail, a result can be too large for the active context and an agent loop can run longer than intended. These cases are handled in the application layer, while access to the underlying systems remains enforced by those systems themselves.
| Failure mode | Architectural response |
|---|---|
| Wrong capability selected | Limit the available tools to the capabilities permitted for the current request. |
| Invalid schema assumption or query | Read source metadata and return execution errors to the specialist capability for a bounded retry. |
| Agent loop does not finish | Count tool-planning iterations and stop execution when the configured limit is reached. |
| Tool result exceeds the active context | Store the complete result outside the model context and pass only a bounded representation back to the model. |
| Answer cannot be traced to execution | Link the answer to tool events and source-query metadata such as query ID, duration and result details. |
Tool authorization ≠ source authorization
Controlling which capability is exposed to the model does not replace permissions in the warehouse or semantic model.
A read-only prompt ≠ a security boundary
Read-only access must be enforced through source credentials, roles, grants and platform controls.
MCP ≠ automatic tool safety
MCP defines the tool interface. Authorization, input validation, network controls and source permissions are implemented separately.
RAG ≠ guaranteed correctness
Retrieval provides source context. The quality of the answer still depends on the documents retrieved and how well they match the question.
Engineering takeaway
The model is part of the runtime. Not the runtime itself
Ask Titan becomes easier to reason about when probabilistic model decisions sit inside deterministic software boundaries. The application owns capability exposure, graph control, durable state and execution evidence. Specialist capabilities own source semantics.
Seven principles we keep
Separate reasoning from execution.
Expose capabilities deliberately.
Use source-specific execution models.
Separate persisted state from model context.
Do not make the model the security boundary.
Bound every autonomous loop.
Keep an execution trail around enterprise answers.
Next in the series
Part 2 · How We Use LangGraph and MCP to Orchestrate Ask Titan
Next we go inside the orchestration layer: how Ask Titan resolves permitted tools, runs the LangGraph agent-tool loop, invokes specialist agents through MCP, persists conversation state and shapes the context sent to the model.
Read Part 2FAQ
Ask Titan enterprise AI architecture questions
Practical answers about LangGraph, MCP, capability authorization, SQL, Power BI, RAG, conversation state and observability.
What is an enterprise AI assistant architecture?
An enterprise AI assistant architecture surrounds the language model with explicit application controls for identity, capability access, orchestration, state, source-specific execution and observability. The model can decide which permitted capability to use, but it is not the security boundary or the control plane.
Why does Ask Titan use LangGraph?
[object Object]
What is MCP used for in Ask Titan?
The Model Context Protocol (MCP) is the standardized capability boundary between the top-level orchestrator and specialist execution tools. It lets the orchestrator discover and invoke capabilities through a consistent interface while source-specific implementation details remain behind each tool.
Does MCP decide which tools a user is authorized to use?
No. Ask Titan resolves application-level capability access before tools are bound to the LangGraph orchestrator. MCP exposes capabilities through a standard interface; authorization policy and datasource permissions remain separate controls.
How does Ask Titan decide between SQL, Power BI and RAG?
The top-level orchestrator chooses among the permitted specialist capabilities based on the user question and tool descriptions. Structured databases use a SQL path, Power BI semantic models use a DAX-oriented path, and enterprise documents use retrieval and source grounding.
Is Ask Titan a RAG application?
No. Retrieval-augmented generation is one capability in Ask Titan, not the overall architecture. Questions can also be answered through governed structured-data tools or Power BI semantic models when those sources carry the relevant business facts and definitions.
How does Ask Titan preserve conversation context?
Conversation state is persisted with thread-scoped checkpoints. When building the next model context, older completed turns can be collapsed to the user question and final answer while the active turn keeps the full tool interaction required for correct tool-call execution.
How can an Ask Titan answer be traced back to its execution?
Ask Titan stores execution metadata around answers, including intermediate tool calls and answer duration. Database-backed tools can additionally capture query identifiers, query text, duration, processed data and cost or consumption metrics where the underlying source exposes them.
Does Ask Titan give the top-level LLM direct database access?
No. The top-level orchestrator calls a specialist capability. The structured-data capability owns schema inspection, query generation, database execution and source-specific telemetry instead of embedding database connector logic directly in the orchestrator.
Does tool authorization automatically enforce row-level security?
No. Tool authorization controls which Ask Titan capabilities are exposed to the agent. Row-level, column-level and other datasource permissions are separate enforcement layers that must be implemented in the connected database, warehouse or semantic model.
From AI demo to governed runtime
Connect enterprise AI to governed data without making the LLM your control plane
Ask Titan is built for business questions that need controlled access to company data, semantic models and documents. The same architecture principles can also inform custom Data & AI platforms where explainability, permissions and source semantics matter.
A governed enterprise AI boundary
01Resolve permitted capabilities before model binding
02Keep source semantics inside specialist execution paths
03Bound tool-calling and active context
04Retain execution evidence around the final answer
Technical references
The Ask Titan repository is private. These public references document the open technologies and protocol concepts discussed in this article.