Ask Titan uses a top-level LangGraph runtime to choose among permitted specialist AI agents. MCP provides the tool interface. SQL, Power BI/DAX and document retrieval stay inside the agents that understand those sources.
The engineering problem
A top-level agent becomes difficult to control when routing, source-specific execution and runtime controls all accumulate in the same model call. Ask Titan separates those responsibilities: the orchestrator decides which permitted capability to use, the specialist agent executes the source-specific work, and the application runtime enforces the deterministic controls.
Top-level orchestrator
Work with the active conversation, see the permitted capabilities, select a tool when needed and decide when the turn is complete.
Specialist AI agent
Inspect source metadata, generate and execute the required operation, handle source-specific errors and return a bounded result.
Application runtime
Resolve capability access, persist conversation state, enforce runtime limits and retain execution evidence outside the model.
One turn through the runtime
The top-level graph has a small control surface. It receives the active conversation context and the tools permitted for that request. If specialist execution is needed, the request travels through the MCP capability interface to the selected specialist agent. The execution result returns through MCP to LangGraph, which either continues the agent-tool loop or produces the final answer for the user.
Step 1 · resolve the tool set
The MCP client can discover a larger capability catalog than the current request should receive. Ask Titan resolves the permitted tool set in application code and binds only those tools to the orchestrator model.
Load the current tool catalog exposed by the MCP capability service.
Determine which tools are permitted for the current identity and runtime context.
The model receives schemas for the permitted set rather than the complete capability catalog.
available_tools = await discover_mcp_tools()
allowed_names = await resolve_allowed_tools(user_id)
permitted_tools = [
tool
for tool in available_tools
if tool.name in allowed_names
]
agent = build_orchestrator(
tools=permitted_tools,
checkpointer=conversation_memory,
)This is application-level capability authorization. It does not replace permissions in Databricks, SQL Server, Power BI or any other connected source.
Step 2 · run the LangGraph loop
Ask Titan uses a ReAct-style LangGraph loop. The agent node calls the model with the active context and permitted tools. If the model emits a tool call, the graph routes execution to the tool node. The result is added to graph state and control returns to the agent. If the model emits no tool call, the graph can terminate the turn with the model response.
graph = StateGraph(AgentState)
graph.add_node("agent", agent_node)
graph.add_node("tools", ToolNode(permitted_tools))
graph.add_conditional_edges(
"agent",
route_next_step,
{
"tools": "tools",
"end": END,
},
)
graph.add_edge("tools", "agent")
graph = graph.compile(
checkpointer=conversation_memory,
)Step 3 · specialist execution
The top-level orchestrator sees a capability as a tool with a name, description and input schema. Behind that interface, the implementation can be a deterministic function or a source-specific agent graph. Ask Titan uses specialist agents where execution requires multiple source-specific steps.
Structured data
Works with the permitted data-source boundary and schema metadata, generates SQL, executes the query and can use execution feedback to refine the operation.
Semantic data
Works with semantic-model metadata, measures and relationships, then generates and executes DAX against the governed business model.
Unstructured data
Rewrites the active question for retrieval, retrieves relevant document chunks and generates an answer from the resulting source context.
Capability boundary
The orchestrator does not need to know how Databricks exposes execution metadata, how a Power BI semantic model represents measures or how document retrieval is implemented. It only needs enough information to select the permitted capability and work with the result returned by that capability.
@mcp_tool(
name="semantic_model_agent",
description="Answer questions using the governed semantic model",
)
async def semantic_model_agent(question: str):
metadata = await inspect_model()
dax = await generate_dax(question, metadata)
result = await execute_dax(dax)
if result.has_error:
result = await refine_and_retry(
question,
metadata,
result,
)
return bounded_tool_result(result)Step 4 · state and model context
The graph uses a checkpointer so thread state can persist across turns. That does not mean every stored message and historical tool result should be sent back to the model on every pass. Ask Titan keeps persistence and model-context shaping as separate concerns.
Persisted thread state
Thread-scoped checkpoints retain graph state outside the model prompt. This preserves conversation continuity and gives the runtime a durable state from which subsequent turns can continue.
Active LLM context
Completed turns can be collapsed to the user message and final answer, while the active turn retains the tool-call sequence needed for execution. Oversized tool results can be represented compactly in model context while the full result remains outside the prompt.
prefix, active_turn = split_at_latest_user_message(state.messages)
finished_turns = collapse_completed_turns(prefix)
recent_history = keep_recent_turns(finished_turns)
llm_context = [
*recent_history,
*active_turn,
]
llm_context = cap_oversized_tool_results(llm_context)MCP's role
Ask Titan uses MCP so the orchestrator can discover and invoke specialist capabilities through a consistent tool interface. The orchestration graph, conversation state, application policy, source permissions and specialist implementation remain separate responsibilities.
Interface boundary: MCP defines how capabilities are exposed to the orchestrator. It does not decide which capabilities a user may access, how a specialist executes its work or what the underlying data source allows.
Responsibility split
State, nodes, conditional routing, tool loops and checkpoint integration.
Capability discovery, tool schemas and invocation across the interface boundary.
Determines which discovered capabilities may be bound to the orchestrator for the current request.
Credentials, roles and source-level authorization, including RLS or CLS where supported.
Runtime boundaries
Agent loops need explicit failure and termination behavior. Ask Titan treats these as runtime concerns rather than relying on the model or a prompt to police its own execution.
| Runtime case | Runtime behavior |
|---|---|
| Model emits no tool call | End the graph turn with the model response and persist the resulting conversation state. |
| Model emits a permitted tool call | Route to the tool node, execute the selected capability and return its result to the agent for the next model pass. |
| Specialist execution fails | Keep source-specific error interpretation and correction inside the specialist flow, where the relevant source metadata and execution context are available. |
| Tool output is too large for useful model context | Keep the complete result outside the prompt and pass a bounded representation into the next model call. |
| The model continues emitting tool calls | Apply deterministic runtime limits so the loop cannot depend on the model eventually deciding to stop. |
Boundaries that remain separate
A discovered MCP tool is not automatically available to the current user or request.
Binding a tool does not replace credentials, roles or permissions enforced by the underlying data source.
Persisting a thread does not require replaying its complete history into every model call.
A tool result returns to graph state as evidence. The orchestrator can then continue the loop or finish the turn.
Engineering takeaway
The top-level Ask Titan graph stays easier to extend when it does not absorb every data-source implementation. New source-specific behavior can remain behind a specialist capability boundary while the orchestration structure stays largely unchanged.
Patterns we keep
Next in the series
The next article goes deeper into the authorization boundary introduced here: identity, application policy, capability filtering, tool binding and source-level permissions. It also explains why discovering an MCP capability is not the same as authorizing a user to invoke it.
Read Part 3FAQ
Practical answers about Ask Titan's top-level orchestrator, MCP capability discovery, specialist agents, graph state and model context.
LangGraph runs the top-level stateful agent-and-tool loop. The orchestrator receives the current conversation context and the permitted tool set, decides whether to answer or call a tool, receives tool results and continues until the turn is complete.
MCP provides a consistent interface for discovering and invoking specialist capabilities. Ask Titan can expose SQL, Power BI and document agents through the same tool boundary without embedding their source-specific implementation logic in the top-level orchestrator.
No. Ask Titan resolves application-level tool access before the tools are bound to the orchestrator. MCP provides the capability interface; user authorization and datasource permissions are separate controls.
No. The orchestrator selects a permitted specialist capability. The SQL agent owns structured-data execution, the Power BI agent owns semantic-model and DAX execution, and the document agent owns retrieval and source-grounded answering.
The result returns to the LangGraph runtime as a tool result. The top-level model can use that result to answer, invoke another permitted tool if required, or finish the turn.
Ask Titan compiles the graph with thread-scoped checkpoint persistence. The persisted graph state provides conversation continuity across turns, while the model context for each call is built separately from that stored state.
No. Older completed turns can be collapsed to the user message and final answer, while the active turn retains the tool calls and tool results needed for correct execution. Oversized tool results can also be represented more compactly in model context.
The explicit graph separates model planning, tool execution, persisted state and stop conditions. Source-specific behavior stays inside specialist agents rather than accumulating in one top-level prompt and one connector layer.
No. The orchestration design depends on a capability interface for tool discovery and invocation. Transport and protocol lifecycle details can evolve independently, which is important because MCP has changed materially across specification revisions.
Agent orchestration
Ask Titan separates top-level routing from source-specific execution. The same pattern can be used for custom enterprise AI systems that need multiple governed tools, persistent state and explicit application controls.
One turn, four boundaries
Technical references
The examples in this article are simplified architecture patterns. These public references document the LangGraph and MCP concepts used in the design.