From Router-Based Agents to Tool-Calling Agents
From Router-Based Agents to Tool-Calling Agents
As EVA's features and user requests grow, deciding which predefined route should handle each request becomes increasingly difficult. In EVA v3.1, we changed the chat architecture from a router-based agent to a tool-calling agent.
The previous architecture classified a request and sent it to a predefined processing route. The new architecture gives the LLM a registry of available capabilities, allowing it to select the tool required by the request and respond based on its execution result.
1. Why change the architecture?
A router-based architecture has a clear advantage: each request type has an explicit processing path. However, as the number of features and question types increases, several limitations become visible:
- Adding a feature often requires changes to both the router and the graph branches connected to it.
- Manual content embedded in prompts and code becomes difficult to update and maintain.
- Questions about the current runtime configuration can become mixed with general feature explanations in the same Q&A flow.
2. Router-based agents and tool-calling agents
The two architectures answer different questions.
| Architecture | Main question | Typical flow |
|---|---|---|
| Router-based agent | “Which path should handle this request?” | Request → classification → predefined Node or Subgraph → response |
| Tool-calling agent | “Which capability is needed to handle this request?” | Request → intent analysis → required tool selection → tool execution → response |
In the router-based design, the code defines the available branches and selects one of them. In the tool-calling design, the LLM receives the available tool schemas and determines the tool required by the request.
The distinction is not simply about replacing a router with an LLM. It changes where the processing decision is made and how capabilities are added to the system:
Router-based agent
User request
→ Request-type classification
→ One predefined route
→ Node or Subgraph execution
→ Response
Tool-calling agent
User request
→ Intent analysis
→ Required tool selected
→ Tool execution
→ Execution result reviewed
→ Response
3. The Prepare–Decide–Tool–Compose–Finalize flow
The new agent follows five conceptual stages:
User request
↓
Prepare
↓
Decide
↓
Tool
↓
Compose
↓
Finalize
| Stage | Role |
|---|---|
Prepare | Prepare the language, conversation history, current settings, and other context required for the request. |
Decide | Analyze the request and select the required tool. |
Tool | Execute the selected tool. |
Compose | Turn the tool execution result into a natural response. |
Finalize | Convert the result into the final response format. |
For example, the request “Change the AI inference interval to 30 seconds” can be processed as follows:
Decide
→ Select set_detection_interval
→ Validate the input value: 30 seconds
→ Change the setting
→ Return the execution result
→ Generate the final response
Not every request needs every stage to perform a separate operation. The important point is that the agent has a consistent lifecycle in which tool selection, execution, and response generation are explicit.
4. Moving manual Q&A from prompts to Knowledge RAG
The architectural change also affects how EVA answers questions about its manuals and features.
Prompt-based Chat
In the previous design, guides were selected by request type and inserted into the prompt before generating an answer.
User question
→ Classify the question
→ Select a related guide
→ Include the guide in the prompt
→ Generate the LLM response
For example, a question about object-detection sensitivity might be classified as a terminology or app-feature question. The code would then select TERM_GUIDE or APP_GUIDE and include its content in the prompt.
This approach has several limitations:
- There is no separate document-retrieval step.
- The answer is limited to the guide content included in the prompt.
- Updating a guide may require changes to a prompt or code.
- The code must select the guide that matches the question type.
Tool-calling Chat
In the new design, a manual question selects answer_eva_question. The tool embeds the question, searches the Knowledge store, and provides relevant pages as context for the answer model.
User question
→ Select answer_eva_question
→ Create a question embedding
→ Search related pages in Qdrant
→ Pass the retrieved page content as context
→ Generate the answer
For a question such as “How do I configure a detection scenario in EVA?”, the system searches for the manual page containing the relevant configuration procedure and generates an answer from that page.
This makes manuals an independently managed Knowledge source. Only the relevant pages can be retrieved, and metadata such as the file name and page number can be included in the result.
The trade-off is that retrieval quality now directly affects answer quality. The Knowledge pipeline and its evaluation become important parts of the chat system.
5. Page-level Knowledge Ingest
RAG search requires manuals to be stored in a searchable form before users ask questions. The current ingestion flow is:
PDF or Markdown
↓
Extract text, tables, and figure information
↓
Build page-level content
↓
Create embeddings
↓
Store page vectors in Qdrant
At query time, the flow is:
User question
→ Create a query embedding
→ Search Qdrant by similarity
→ Return related pages
→ Use the pages as answer context
Why search by page?
A page often contains information that is meaningful only when read together. For example, a single page may include:
- A feature description at the top.
- A configuration screen image in the middle.
- Important cautions at the bottom.
If the page is split into very small sentence-level chunks, these related pieces can be returned as separate search results. Page-level retrieval preserves the relationship between the description, configuration procedure, table, image explanation, and caution.
The goal is not simply to make chunks larger. It is to preserve information that is explained together within the same page.
The current structure finds related material at page granularity and passes the page text and metadata to the answer model as a ToolResult. This preserves the relationship between descriptions, configuration procedures, tables, and cautions while separating Knowledge retrieval from answer generation.
Extending Knowledge for customer environments
Knowledge can include not only EVA manuals but also documents required for each customer environment. Customer operating manuals, equipment standards, work procedures, and safety and environmental regulations can be ingested so that answers are grounded in the standards and rules of the relevant site, rather than limited to general feature explanations.
For example, adding a safety and environmental regulations handbook to Knowledge makes it possible to ask not only whether a detection alarm occurred, but also which safety or environmental regulations apply, what the site response procedure is, and what follow-up reporting is required. Customer-specific documents can be managed as separate Knowledge sources while using the same tool-calling flow to search and answer according to each site’s operating rules.
6. End-to-end architecture
The complete system has two related flows: an offline Knowledge ingestion flow and an online inference flow.
Offline Knowledge flow
PDF / Markdown
→ Extract text, tables, and figures
→ Create page-level text
→ Generate page embeddings
→ Store vectors in Qdrant
Online inference flow
User request
→ Prepare conversation history and current settings
→ Bind the available Tool Registry to the LLM
→ LLM creates the required tool call
→ Execute the selected tool
→ Return the ToolResult
→ Generate a natural-language answer when needed
→ Finalize the response
Tool-calling Chat
The Tool Registry can contain different kinds of capabilities:
- Action Tools for changing application state.
- Q&A Tools for Knowledge-backed answers.
- Conversation Tools for general dialogue.
- Meta Tools for reading runtime information or other system state.
When answer_eva_question is selected, it starts the query-embedding and Qdrant-search flow. The retrieved page content is returned as a ToolResult and can then be used by the composing model.
This keeps the responsibilities separate: tools perform concrete operations, Knowledge retrieval provides evidence, and the language model generates the final response.
7. What the structural change enables
The practical benefits of tool calling
The value of tool calling is not simply that an LLM can call a function. Its main benefit is that request interpretation, concrete operations, Knowledge retrieval, and final answer generation have separate responsibilities and clear roles.
- Easier feature expansion: Add a tool and its schema to the registry instead of wiring every new capability into router branches and graph paths.
- Current information: Retrieve manuals and runtime state from Knowledge or state-lookup tools when needed instead of fixing them inside a prompt.
- Safer execution: Keep natural-language interpretation separate from state changes by applying validation, permissions, and execution conditions to each Action Tool.
- Better observability: Record and evaluate which tool was selected, which inputs were passed, and which search results were returned.
For the evaluated requests, the average response speed was about 30% faster than the previous router-based flow. The improvement came from reducing unnecessary classification steps and executing the required tool directly.
Easier feature expansion
In the previous architecture, a new capability could require changes to router and graph branches. In the new architecture, the capability can be implemented as a tool and registered with its schema.
Before
→ Modify the router and graph branches
After
→ Implement a tool
→ Register its schema
Independent Knowledge management
Manual content no longer needs to live in prompts or application code. It can be managed as Knowledge documents and reflected through the ingestion pipeline.
Clearer separation of request purposes
The same user-facing topic can be connected to different backends depending on the request:
Current runtime value
→ Runtime setting lookup
Feature explanation
→ Knowledge search
Configuration change
→ Action Tool
Along with the response-speed improvement, the processing path becomes more flexible as the number of capabilities grows.
8. New operational considerations
The architecture is more flexible, but it also introduces new areas that require monitoring and evaluation.
| Area | Question to answer |
|---|---|
| Tool selection | Does the agent select the correct tool for the request? |
| Tool schema | Can the LLM correctly understand the tool's purpose and input values? |
| Action safety | Which state-changing actions can be executed automatically? |
| RAG retrieval | Does the search return the relevant page without missing key information? |
| Page-level retrieval | Does one page contain too many unrelated topics? |
| Document management | How are versions and duplicate Qdrant points managed? |
| Ingestion | Do the service settings and ingestion script remain consistent? |
| Evaluation | How are tool selection, action execution, and RAG answers tested? |
In particular, a successful tool-calling system needs to evaluate more than the final wording of the answer. It must also inspect whether the right tool was selected, whether its inputs were valid, whether the action was safe, and whether the answer was grounded in the right Knowledge page.
Closing
The move from a router-based agent to a tool-calling agent is not simply a replacement of one chat component. It is a change from a system where code determines the request's processing path to a system where the LLM selects the capabilities and Knowledge required to handle the request.
With this structure, EVA can manage manuals as an independent Knowledge source and extend functionality by adding tools to a registry. At the same time, tool schemas, action safety, retrieval quality, document versioning, and end-to-end evaluation become essential parts of operating the agent reliably.
