Event and Ladder planning

Real-Time Video Agents Hack — SF

App deployment expectation — checked October 2, 2026

The starter repo at commit 4987d8e explicitly directs custom hackathon apps onto the supplied Kubernetes cluster, inside the assigned team namespace, exposed at the existing team host under /app. Its custom-app deployment guide labels on-cluster deployment and that ingress path non-negotiable. The deployment overview repeats this route. The already-running VAST ingestion stack stays in place; the custom app calls the team's VSS APIs. “Within VAST” here means a web service in their supplied cluster using the VAST stack, not a native OS plugin or a replacement ingestion deployment.

The submission guide separately makes a lasting reachable live-app link optional because some apps will not remain up after the event. It requires runnable code that judges can open. This distinction supports treating cluster /app as the documented demo target; the optional link is not evidence that a Vercel-only app fulfills that deployment guide. No disqualification rule for external hosting was established by this check. Older submission-channel claims below are historical: the current starter says to wait for submission instructions.

This was a documentation check requested by Pablo, not a deployment or a verification of MATCHA's running state.

Publication note, September 28, 2026: imported from the existing conversation’s event and Ladder planning notes. This is a dated planning record, not fresh verification of the listing or repositories.

Official hackathon page (tokensand.com/vastsf, read October 1)

• Times (newer than the listing below): doors and keynotes 9:30, build 10:00, submission deadline 4:30 PM PT, demos 5:00, awards 7:00. The earlier listing said "build 10am-5pm".
• Build environment for the first 100 attendees to arrive only. Others may join "while space is available".
• Submit: a public GitHub repository, a short demo video with a shareable link, what you built and the tools used, team names and emails. A working website and a screenshot are welcome. "Build your project during the event." One project per team, up to 4 people.
• Organizers' advice: pick a specific use case ("flag someone missing a hard hat", not "watch for safety issues"); adapt the idea to the footage on build day; "Search depends on what the model's prompt captures in each video description. You can ingest video with a different prompt on build day."
• Prizes: 1st NVIDIA DGX Spark, 2nd Hugging Face Microduck; two teams win overall.
• Resources named: a "Video search overview" video (below), YOLO11 docs, CoreWeave, W&B ARIA research agent, W&B serverless inference and Weave, VAST AI OS, VSS Blueprint docs, Cosmos. Project gallery empty on October 1.

What the organizers' overview video shows (the baseline judges know)

The 2:13 video (vss2-blurred-3.mp4) walks through the VAST VSS Blueprint app:

• DataEngine pipeline manager and visual builder (events → chunks-segments → detector → reasoner → embedder), with Deploy.
• Search with filters (time, camera ID, capture type, location, object class) and "Reveal Similarity Query" (shows the SQL).
• Cosmos Reason "Top 3 clips summary": answer, notable moments with times, and a "Gaps" line.
• Segment timeline: one 5 s card per piece with caption, YOLO class chips with counts ("29 bench", "4 bus"), relevance %, boxes on/off.
• Explore by upload date and location.
• Dashboard: ingest quality (structured JSON parsed, detector coverage, rows with object classes, all 100%), YOLO class heatmap per segment, upload metadata counts, key events list.

Implication: search, 5 s timelines, YOLO chips and an ingest dashboard are the baseline, not a contribution. The dashboard measures pipeline coverage, not correctness: "100% of rows have object classes" says nothing about whether the classes are right.

Source: listing pasted by Pablo on September 28, 2026; no URL supplied or independent verification.

• VAST Builders Challenge: Friday October 2, 9am–7pm, San Francisco; address requires registration. Event timezone not explicit in the pasted text.
• Approval required. Teams up to four prioritized; individuals welcome. Use the email associated with Cursor credits and prerequisites.
• Deadline conflict: explicit September 29 versus generic “one week prior” (September 25). Current acceptance unconfirmed.
• Build 10am–5pm; demos 5pm; awards 7pm.
• Stack: NVIDIA Cosmos, semantic video search, YOLO, Weights & Biases LLMs, VAST AI OS, CoreWeave GPUs, Cursor. Starter videos supplied.
• First prize NVIDIA DGX Spark; also HuggingFace Microduck, Cursor credits, cash gift cards.
• Hosts: tokens&, Alessandro Amenta, Syerra Laurusaitis, Jacopo Piazza. Partners: VAST Data, NVIDIA, SpaceXAI, CoreWeave, Weights & Biases.
• Sponsor descriptions in the September 28 pasted continuation: NVIDIA supplies accelerated computing and AI software; Weights & Biases is described as a CoreWeave company providing experiment tracking, model evaluation and collaborative ML development tools. These are listing claims, not independently verified. The footer reiterates that the exact San Francisco location requires registration.
• Judges: Hassan Moustafa, Adam Ryason, Ram B., Brian Verkley, Anushrav V., Arnav Verma.
• Entry: 18+, physical government photo ID, photography/filming consent. No bikes/scooters in Amazon buildings; no onsite parking.

No registration, attendance decision or team commitment recorded. People listed are not established personal contacts. Saved as a free CRM research note. Historical state: no shared project destination had been identified at initial capture. On September 28, Pablo selected this workspace for these notes.

Update — September 29 (live page and starter repo checked)

• Registration: the live SF page says "Registration closes on September 30th" and still carries the generic "one week prior" line. Pablo's September 28 paste said September 29. The page is the newer source. Approval required; teams of up to four prioritized. The application asks "What video agent do you or your team want to build?" Pablo told us on September 29 that he has signed up. Approval arrived by email at 6:53 pm PT the same day. A generic "registered solo? bring a teammate" email came 9 minutes earlier. The approval email asks for a laptop and charger, a Cursor account on the application email, and optionally your own clips.
• Series: SF is the first of three (NYC October 9, London October 17). It is the inaugural challenge, so there are no past winners.
• Venue [I]: co-host AWS Builder Loft and "Amazon building" point to 525 Market St. Not confirmed.
• SpaceXAI owns Cursor (deal closed August 14, 2026). Its judge is a Cursor field engineer.
• Judging: "creativity, technical implementation, and the real-world impact" (VAST blog). The starter guide asks for a clear use case, clever use of search and metadata, and something that runs. Submission is a SUBMISSION.md posted in the event Slack, with a demo video and a code link.
• Starter stack and videos: see builder stack and plan. The corpus is pre-indexed CCTV, dashcam and human first-person video, with no labels.

Planning direction — September 28

Pablo explicitly designated this conversation for planning, then selected publu/ladder as the project to build up. This is direction for planning, not authorization to implement, deploy, register, or contact others. At initial capture no shared Ladder wiki destination had been selected. Pablo subsequently selected this hackathon workspace on September 28; other workspace records are outside this import.

Repository review at main commit 81631f9ee8da13f1337ebfa8e49715af9624b0ae:

• README describes egocentric manipulation-video capture-quality triage using cheap checks and an expensive judge, cached frames, versioned results and a viewer. Reported 132,576-clip outcomes are historical repository claims, not a fresh benchmark or independently verified accuracy result; only a 43-clip judge sample is described.
• Core implements file-based cached-keyframe checks and a six-keyframe judge. _vlm defaults missing output to PASS and returns PASS on exceptions. assemble_verdict can return PASS with missing stages. These are static code findings, not executed reproductions; unknown/error/incomplete states need explicit handling before trusting acceptance metrics.
• Evaluation supports human-corrected gold drafts and scoring, but still contains older category names. Rubric/evaluation alignment and a frozen human-reviewed holdout should precede quality claims.
• Open issues describe teacher-based calibration, declarative workflow configuration, and measured specialist/temporal capacity increases. Issue descriptions are proposals/status claims; they do not prove that every described stage exists in current main. Preserve the distinction between teacher agreement and human-verified accuracy.

Earlier assistant proposal, superseded by Pablo's broader direction below: retain the capture-quality focus and demonstrate a recording-quality agent using rolling windows, immediate actionable feedback, uncertain-case escalation, and a reviewed clip/segment export. First narrow prototype: one tabletop manipulation task; blocked lens, hands out of view and lost workspace. This proposal was not selected.

Current synthesis — robot learning and control

Pablo wants something more ambitious and current: gather intelligence on MolmoAct, JEPA, existing RoboRun work, better robot post-training/control, and a striking demonstration for the judges. The session remains for planning. See the sourced robot post-training research and its immutable source manifest.

Research identifies MolmoAct2 as an open manipulation-policy candidate; V-JEPA 2.1 as a potential temporal representation for failure analysis, distinct from an action-conditioned planner; Cosmos 3 policies as candidates for joint action/future-video demonstrations; GR00T N1.7 and openpi as alternative compatible baselines. RECAP and RL Tokens supply relevant post-training precedents. None has been benchmarked locally in this session.

The earlier assistant recommendation was a failure-to-correction-to-policy-update loop. Pablo subsequently clarified the priority: preserve and extend Ladder's minimal-compute, fast-path approach, increasing compute as cases get harder and emphasizing a 95% accuracy target. Adaptive compute under a measured quality constraint is now the primary planning direction; a standalone post-training demonstration is secondary. The target is unachieved and the task/accuracy denominator is unresolved.

Proposed evaluation: compare cheap-only, full-model and adaptive-cascade baselines on the same held-out workload; measure independent correctness, coverage/abstention, difficult-case results, full-task success, latency and compute. A further distillation experiment would teach a cheap specialist to handle previously escalated cases. Model confidence and model size cannot establish the quality threshold by themselves. This is a proposal, not a benchmark result.

Reuse RoboRun recording and scenario infrastructure. Public feat/fleet-lab source contains additional useful components, but its VLA adapter only maps velocity-style motion commands and its export tests do not establish synchronized manipulation-training episodes. A real joint/gripper policy interface and action/state dataset contract remain work.

An imagined-future-versus-execution view is an optional demonstration layer if the chosen action-conditioned model supports it; it is not a substitute for real success measurements. Ordinary human video is not automatically robot action supervision. Available hardware, GPU budget, strongest prior run, event preparation rules and exact sponsor endpoints remain unresolved. No implementation or training has occurred. These planning notes were subsequently published here on September 28 at Pablo’s request.

Planning index · Event and direction · Builder stack and plan · Models and existing assets · Experiments · Source provenance