Observe, Judge, and Question: An Enhanced Thinking Mode
Observe, Judge, and Question: An Enhanced Thinking Mode
It is difficult to expect consistent decisions from a small VLM simply by asking it to “think deeply on its own.” A model may try to solve several problems within a single call, or revisit ambiguous evidence for too long without reaching a conclusion.
In this enhanced Thinking mode, we designed the reasoning flow itself as a set of explicit roles and structures instead of leaving every decision to the model’s internal reasoning. The pipeline establishes the criteria first, fixes the area to inspect, evaluates the scene from different perspectives, and leaves the final decision to code.
The goal is not to make the model think more. It is to make the order in which it should observe, judge, and question explicit.
1. Problems with the Previous Thinking Structure
The previous development version judged several hazardous situations in the incidents block with a single VLM call.
The structure is simple, but assigning different problems to one call creates two issues.
1) Multiple problems are judged at once
One call judges fall, smoke, and fire simultaneously. Even when some items are easy to determine, one ambiguous item can consume the entire reasoning budget.
2) The scope of reasoning is difficult to control
Reasoning happens inside the model as a black-box process. In practice, the only value the application can adjust is the budget number; it is difficult to control which criteria the model revisits or what it repeatedly checks.
| Aspect | Previous structure |
|---|---|
| Targets | Three fixed types: fall / smoke / fire |
| Decision criteria | Hard-coded in the code |
| Evidence | Not retained; only the final booleans are returned |
| Failure behavior | All results may quietly become False |
2. Limitations Revealed by Actual Thinking Traces
Consider a situation in which smoke must be judged in a facility room filled with vapor. A model “thinking deeply” does not necessarily mean that it is finding new evidence.
Facility scene used for the smoke decision
Qwen3.5VL-9B-Thinking Langfuse Trace
After finding the white vapor, the model repeatedly reread the same rules to determine whether it was steam or a gas leak. However, no new visual evidence was added during the repetition.
The repeated thought, “Wait, let’s look closer,” produced zero new visual evidence.
The main problems identified in this trace are:
| Symptom | Meaning |
|---|---|
| The same rule is reread multiple times | Rumination without new information |
| The model cycles without reaching a conclusion | It does not know how to stop thinking; the answer depends on where the budget runs out |
| The model revisits alternative interpretations from the dataset | It reinterprets the policy on the fly even though the policy was defined in the prompt |
| Easy items finish quickly while ambiguous items consume the entire process | One problem monopolizes the reasoning budget |
As a result, simply asking a small VLM to “think deeply on its own” is not enough. The system needs to guide what the model checks first, when it ends its judgment, and how it handles uncertainty.
3. Design Philosophy: Turn the Reasoning Flow into Structure
The enhanced Thinking mode is built around three principles.
| Principle | Description |
|---|---|
| 1. One call = one problem | Split the task into small, clear units so that each call does not need separate reasoning. Every call uses an Instruct model with enable_thinking: false. |
| 2. Reasoning flow = structure | Guide the flow through the order of establishing criteria → fixing the view → judging → looking for counterevidence. |
| 3. Code makes the decision | The final alert is determined by a deterministic Gate, not by a single LLM output. |
The new architecture first turns the scenario into explicit rules and then lets voters with different roles make independent judgments.
- The two voters run in parallel and independently; neither can see the other’s output.
- Every call uses an Instruct approach with
enable_thinking: false. - Each call receives only one decision problem.
- The final alert is triggered only when both voters are positive.
The LLM does not decide everything at once. Each stage focuses on its own role, while code controls the connections between stages and the final decision.
4. Core Mechanism: “I’m Looking Here Now”
Before making a judgment, the model is required to output evidence of where it is looking. The role responsible for this is focus_locator.
focus_locator does not make the decision. It only suggests the object and location to observe. vote_context then uses both the full frame and a crop of the region specified by the Locator, while vote_skeptic independently verifies the scene from the full frame without relying on the Locator’s description.
| Mechanism | Effect |
|---|---|
| Evidence declaration | Outputs what is visible and where it is located in text and coordinates before making a decision |
| Fixed focus | vote_context sees the full frame together with a crop containing a 20% margin around the region |
| Contamination prevention | vote_skeptic sees only the full frame without the crop, so an incorrect Locator does not contaminate its judgment |
Why does this improve performance?
- An 8B-class VLM can easily judge a scene from its overall impression. Providing evidence from an explicit region first helps anchor the judgment to that evidence through grounding.
- A 20% margin crop enlarges the target while preserving surrounding context. This reduces mistakes caused by looking only at part of the target.
- Because the observed location is retained as text, an error can be analyzed as either an incorrect focus or an incorrect judgment.
5. Solving Failed Traces with Structure
The new Pipeline roles and Gate address the problems found in the previous Thinking traces.
| Failure in the previous trace | Response in the new structure |
|---|---|
| Rules are renegotiated during the decision | Confirm the rules with scenario_rule_builder before making the decision |
| The inner monologue repeatedly asks, “Is it really smoke?” | Institutionalize counterevidence through an independent skeptic voter |
| True/False is forced when the state is ambiguous | A voter can return uncertain, and the Gate blocks the alert |
| One problem monopolizes the budget | Split calls according to the one call = one problem principle |
This changes the review process required for a decision into observable system stages instead of merely making the model’s internal reasoning longer.
6. Before / After
| Aspect | Previous (develop) | New (thinking_instruct) |
|---|---|---|
| Targets | Three fixed types | Arbitrary scenarios: intrusion, loitering, leakage, and more |
| Judgment | One VLM call judges all items simultaneously | Four role-separated calls, one problem per call |
| Reasoning | Happens inside the model and is difficult to control | An Instruct-based structure defines the flow |
| Final decision | Uses the model output directly | Deterministic AND Gate |
| Evidence | Not separately recorded | Each voter’s vote and reasoning are recorded |
| Decision criteria | Hard-coded in the code | Generated dynamically for each scenario |
The key change is not simply increasing the number of calls. It separates decision criteria, observation area, positive and counterevidence roles, and the final decision so that failure points can be traced.
7. More Flexible Scenario Expansion
In the previous structure, the targets had to be fixed to smoke, fire, and falls because the LLM could easily move outside the intended decision scope.
In the new structure, even when a user edits the incidents block, a rule guide first organizes the decision criteria. Each voter judges only according to the generated criteria.
For example, users can enter requirements such as:
- “Smoke detected” → Detect black smoke
- “Fire detected” → Detect large flames occupying most of the region
The edited text is used to generate the Rule Guide, and each voter then uses those criteria. This reduces the need to modify the Router or decision code whenever a new scenario is added.
8. Performance Results
Precision 0.8273: Suppressing False Positives
In the SINGLE evaluation, the new structure substantially improved Precision.
| Metric | F1 | Accuracy | Precision | Recall |
|---|---|---|---|---|
| Before | 0.8073 | 0.9536 | 0.7316 | 0.9006 |
| After | 0.8042 | 0.9518 | 0.8273 | 0.7823 |
Precision increased from 0.7316 to 0.8273, a gain of 0.0957 points. Recall decreased from 0.9006 to 0.7823. The AND Gate, which requires both voters to be positive before triggering an alert, suppresses false positives at the cost of conservatively blocking some true cases.
Category-level performance
| Category | Before Precision | Before Recall | Before F1 | After Precision | After Recall | After F1 |
|---|---|---|---|---|---|---|
| Smoke / flames | 1.0000 | 0.9714 | 0.9855 | 1.0000 | 1.0000 | 1.0000 |
| Fire | 0.7288 | 0.8600 | 0.7890 | 0.8085 | 0.7600 | 0.7835 |
| Fall | 0.5319 | 0.8621 | 0.6579 | 0.7222 | 0.6610 | 0.6903 |
Smoke and flames reached 1.0000 across all metrics, while Fire Precision improved to 0.8085. In particular, false fire alerts were reduced to nine cases, reducing unnecessary alerts in operations.
In fall scenarios such as the example above, the system checks step by step whether a person is lying horizontally on the floor rather than simply crouching or sitting, and whether there is sufficient direct visual evidence.
Conversely, a red glow is not immediately treated as a real flame. The system reviews the lighting, reflections, flame shape, combustion clues, and whether smoke is present; without clear evidence, it returns alert: false.
Summary
Instead of asking an 8B model to think more deeply, we laid out the path that its reasoning should follow — establish criteria, fix the focus, judge, and question.
