Alpha Feature — not recommended for production use.

deploy

-p search

• "Find all instances of forklifts"
• "When did someone enter the restricted area?"
• "Show me people near the loading dock"
• "Search for vehicles between 8am and noon"
• Any natural-language search across video archives

$HOST_IP

1. Probe the stack:
   bash

   curl -sf --max-time 5 "http://${HOST_IP}:8000/docs" >/dev/null \
     && curl -sf --max-time 5 "http://${HOST_IP}:9200/" >/dev/null

   (The second check confirms Elasticsearch is up — unique to the search profile.)
2. If the probe fails, ask the user:

   "The VSS

   search

   profile isn't running on

   $HOST_IP

   . Shall I deploy it now using the

   /deploy

   skill with

   -p search

   ?"

   • If yes → hand off to the

     /deploy

     skill. Return here once it succeeds.
   • If no → stop. Do not run this skill against a missing or wrong-profile stack.
   (If your caller has granted explicit pre-authorization to deploy autonomously — e.g. the request says "pre-authorized to deploy prerequisites", or you are running in a non-interactive evaluation harness with that permission — skip the confirmation and invoke

   /deploy

   directly.)
3. If the probe passes, proceed.

1. Ingest — Videos are uploaded or streamed via VIOS. The RTVI-Embed service (Cosmos Embed1) generates vector embeddings for video segments.
2. Index — Embeddings are stored in Elasticsearch via the Kafka pipeline.
3. Query — Natural-language queries are embedded and matched against stored vectors by similarity.
4. Results — Timestamped video segments ranked by relevance, with clip playback links.

• Attribute-only: when the LLM decomposes the query and finds only appearance attributes with no action (e.g. "person wearing red jacket")
• Embed-only: when the query has no extractable attributes (e.g. "show me forklifts")
• Fusion: when the query has both an action and attributes (e.g., "person in red jacket running"), it runs embed search first, then reranks using attribute search

1. Resolve inputs from user instructions — HARD STOP if

   $HOST_IP

   is not explicitly provided. See § Input resolution below. Do NOT default to

   localhost

   ,

   127.0.0.1

   , the host the agent itself is running on, or any other guess. Do NOT issue a

   POST http://.../generate

   request until the user has supplied an endpoint. Respond to the user with a single question asking for

   HOST_IP

   / the VSS agent endpoint and wait.
2. Run the search(es) via approach chosen
3. Present the results to the user query. Format response as a professional inspection report but name it

   Video Search Results

   : — Use clear section headers
   • Organize findings individually with supporting detail, and close with a summary
   • Use tables where comparisons help. Write like a technical report, not a chat message.
4. CRITICAL: Verify the results and explain this to the user concisely. If search fails, or returns unexpected results (i.e. videos that do not appear to match user query, zero matches, zero videos returned, error etc.), STOP. Do not proceed without reading troubleshooting.md to iterate with feedback loops until proper results are found and presented like a professional inspection report.
5. Final verifications:
   • ALWAYS inform user that final and further verifications can be run. Present this as a

     Verification Step

   • ONLY IF user agrees, download screenshots using the

     screenshot_url

     of the best candidates (highest similarity scores) from the search hits (JSON results) to

     /tmp

     . Read them and verify if they correspond to the user query

• $HOST_IP: where the VSS agent backend runs

• ALWAYS step into the troubleshooting step of the workflow immediately if anything unexpected happens, read troubleshooting.md
• Queries work best with concrete visual descriptions (objects, actions, locations). Augment user queries if needed to enhance the quality of the questions, expanding potential details
• User queries to do video search supposes video sources are already ingested. No need to search for them locally. Assume this unless the findings show the video source is not ingested yet
• Use

  video-analytics

  skill to cross-reference search results with incident/alert data