In the previous post, I described a proactive VLM as a system that watches a scene, predicts what may happen next, and warns a person when something important is about to happen. For example:

A baby is crawling toward a knife on the floor.

A normal VLM waits for someone to ask, "Is there a knife on the floor?" A proactive system notices the situation without being asked and warns the parent.

But this raises a bigger question: why should warning a human be the system's only option?

  • If a robot can move, it could pick up the knife.
  • If an autonomous car sees a pedestrian, it could slow down.
  • If a hospital robot detects that something is going wrong, it could call a nurse, bring equipment, or change what it is doing.

Once the system can choose and take actions, we are no longer talking only about a VLM — we are talking about a world model connected to an action policy.

A VLM understands. A world model predicts and acts.

A VLM helps a machine understand what it sees and connect it to language. It may recognize a baby, a knife, the direction the baby is moving, and the fact that knives can be dangerous. That's important, but it isn't enough — the system also needs to ask:

  • What will happen if nobody does anything?
  • What could I do?
  • What would happen after each possible action?
  • Which action would produce the best outcome?

That is the job of a world model. It doesn't only describe the current scene — it tries to predict how the world will change, and more importantly, how the future changes depending on what the system does. For the knife example, it might compare several futures:

  1. Do nothing: the baby may reach the knife.
  2. Warn the parent: the parent may remove it.
  3. Move toward the knife: the robot may block the baby.
  4. Pick up the knife: the danger disappears.

The system can then choose the best available action. That's stronger than simply detecting danger early — it's using a prediction of the future to prevent the dangerous situation from happening.

Proactivity is not just early detection

It's easy to describe proactivity as noticing something before a human notices it, but that's only one form of it. A security camera may see someone enter a restricted area and send an alert — since it can't physically intervene, alerting a person is the only useful action it has. A robot has more options: it can move, manipulate objects, block a path, call someone, ask a question, or change its current task.

So the deeper definition of proactivity is not "notice a problem early." It is:

Predict what is likely to happen and do something early enough to change the outcome.

The warning itself is simply one possible action.

The proactive loop

The basic loop is simple:

observe → understand the current situation → predict possible futures → compare possible actions → act → observe what happened

The world-model planning loop: observe, encode to latent state, roll the latent dynamics forward, evaluate rollouts under cost, pick and execute the best action, and feed the consequence back to the observation step.
The proactive loop is the standard world-model planning loop — nothing extra bolted on.

Consider a home robot watching a child near a stove. First it observes the room, then keeps track of the child, the stove, nearby adults, and how everyone is moving. Next it predicts several possible futures — the child walks away, the child touches the stove, an adult intervenes, the robot warns the adult, the robot moves between the child and the stove — and evaluates those futures to choose what to do. After acting, it watches the result and updates its understanding.

This is the same basic loop required for robots, autonomous vehicles, and other systems that act in the physical world. Proactivity is not a separate feature added at the end — it comes from continuously running this loop.

Where the VLM fits

The VLM still has an important role. It's simply not the entire system.

Understanding human instructions. A person may say, "Watch the child and do not let her get close to the stove." The VLM helps translate that instruction into something the system can use when planning its actions.

Understanding the meaning of the scene. The visual system may detect shapes, movement, and objects; a VLM connects them to broader concepts — this object is a knife, knives can cut people, this person is a child, the child may not understand the danger. This kind of general knowledge helps the world model decide which futures matter.

Communicating with people. The system's internal representation will not be understandable to a normal person. A VLM can turn its reasoning into a useful message:

The child is moving toward the hot stove. I am blocking the path. Please come to the kitchen.

Language is therefore the interface between the system and the human — one of the ways the system can act, but not the source of the entire proactive behavior. A simple way to think about it:

The world model predicts and plans. The VLM understands and communicates.

Passive and active proactivity

There are really two useful kinds of proactive systems, and the difference isn't intelligence — it's the set of actions available.

Class What it does Examples
Passive Observes, predicts, and alerts someone, but cannot physically change the environment. Security cameras, medical monitoring systems, workplace safety systems, home cameras, software agents watching computer activity.
Active Observes, predicts, and acts directly. Home robots, autonomous vehicles, industrial robots, medical robots, delivery robots.

A hospital monitoring system that predicts a patient is deteriorating and calls a nurse is still genuinely proactive, even though a person has to complete the intervention. A home robot that sees a dangerous object, moves toward it, and removes it before anyone asks is proactive in the fuller sense — it just has more actions to choose from.

Why a VLM alone is not enough

Suppose a VLM sees a cup near the edge of a table. It may correctly say, "The cup is close to the edge." But a proactive robot needs more — it must predict:

  • Is the cup likely to fall?
  • Is someone moving toward it?
  • Is there a laptop underneath it?
  • Would moving the cup help?
  • Could moving it interfere with the person using it?
  • Is the situation important enough to interrupt what the robot is currently doing?

This requires memory, prediction, planning, and an understanding of how actions change the world. The system cannot choose a useful action by recognizing objects alone — it needs an internal model of causes and consequences.

Language is one action among many

Speaking can itself be an action. Sometimes the best response is "Please move the knife away from the baby." Sometimes it's better to ask, "Is it okay if I move this?" Sometimes the system should remain silent and keep watching. And sometimes, if it's safe and authorized, it should act physically.

A proactive model must choose between all of these possibilities, which means it must consider not only what may happen in the world, but also what may happen because of its own response. An unnecessary warning may distract a driver. A robot moving too quickly may scare someone. Picking up an object may disrupt what a person is doing. The model therefore needs to ask:

Will my action actually improve the situation?

What this means for building proactive robots

If the goal is only proactive monitoring, a VLM combined with memory and future prediction may be enough — the system watches what's happening, identifies a potentially important future, and tells a human.

But if the goal is a robot that independently helps people, the system needs more than a notifier. It needs to:

  • understand the current world;
  • predict how it may change;
  • understand what the human is trying to do;
  • consider different actions;
  • predict the result of those actions;
  • choose whether to speak, move, wait, or intervene.

This is fundamentally a world-model problem. The VLM remains important because the world is full of objects, people, language, and social context — but it's one component inside a larger decision-making system.

The takeaway

A proactive VLM can watch the world and tell a person that something important may happen. A proactive robot must go further: it must predict how the future changes under different actions and choose the action most likely to help.

So the complete proactive system is not just a VLM. It is:

a world model that predicts what may happen, a policy that decides what to do, and a VLM that helps the system understand and communicate with people.

The goal is not simply to warn someone before a bad event happens. The goal is to act early enough that the bad event never happens at all.