GGrounded-USA

Documentation

Help & testing guide

Use this page to learn the product flow, exercise the video intelligence pipeline, and run the same automated checks used during development.

HomeNew daily logVideo intelligence lab

Quick start

  1. Install dependencies: npm install
  2. Copy environment variables from .env.example into .env.local (MongoDB, auth, AI gateway, optional queue and blob tokens as described in the project README).
  3. Start the app: npm run dev then open http://localhost:3000.
  4. Confirm the API can reach MongoDB: GET /api/health/mongodb (returns JSON without exposing secrets).

Daily log wizard

From New daily log, pick a template, capture evidence (video first, then optional photos and voice notes), run the AI draft when ready, and complete structured fields before saving. Saved logs appear under Daily logs.

Video walkthrough → daily log (AI)

The wizard posts your selected walkthrough to /api/daily-log-video/process with streaming NDJSON and (in production-style setups) a queued job so processing can finish in a worker. The server pipeline lives in lib/video-intelligence.ts: it probes duration, extracts audio with ffmpeg, transcribes through the Vercel AI SDK, samples JPEG frames on an interval from your pipeline config, then runs multimodal analysis that pairs each frame with nearby transcript text before streaming a structured daily log draft.

When using the external Python video processor (recommended for Vercel deployments), the pipeline can also compute additional “video intelligence trace” signals per frame: OCR text (signage/boards), brightness/blur quality hints, scene-change scores, and optional hosted vision signals (objects, PPE, and hazards). These signals are attached to the frame trace and injected into the final daily log prompt to improve grounding.

  • Understand the flow visually: /video-intelligence/algorithm — interactive diagram aligned with the TypeScript implementation.
  • Tune and dry-run pipelines: /video-intelligence — select a pipeline variation, edit sampling, prompts, and CV-signal toggles, and upload a test clip (same API as the wizard).
  • Inspect jobs: /video-intelligence/queue lists queued runs; each job page shows status events, transcript, frames, and raw model output when the processor persists them.
  • After a queued wizard run: on the capture step, open the Video intelligence trace block for audio, transcript, frames, and a link to the matching queue job.

Server requirements for real (non–dry-run) processing include ffmpeg / ffprobe on the host running Next.js, plus AI gateway credentials. Optional Python backup is documented in README.md under VIDEO_PROCESSOR_URL.

Automated tests in this repo

Unit tests (Vitest)

Fast, Node-only tests live next to library code as *.test.ts files (for example job URL parsing and error types). The npm run test script runs vitest run using vitest.config.ts.

npm run test

During development you can use npm run test:watch (interactive vitest).

End-to-end tests (Playwright)

Browser tests live in e2e/. The Playwright config can start npm run dev automatically when no server is listening (set CI=1 in automation to enforce a fresh server). First-time setup on a machine may require browser binaries:

npx playwright install

Then run npm run test:e2e (wraps playwright test):

npm run test:e2e

To point tests at an already-running instance (for example another port), set PLAYWRIGHT_BASE_URL before invoking Playwright.

Manual QA checklist (video AI)

  1. Open the lab, confirm pipeline configs load without error.
  2. Upload a short narrated MP4, run the pipeline, and verify a draft object returns or a clear error.
  3. From the daily log wizard capture step, queue a run and confirm status messages advance, the draft merges into fields, and the trace section links resolve.
  4. Open the algorithm page and confirm the diagram and frame estimate controls respond.