← Back to home

LingBot house scan

Turn a phone video into a shareable house scan.

This is a product-shaped proof of concept around LingBot-Map: record or upload a room-to-room walkthrough, send it to a cloud GPU, reconstruct the scene, and open the result as a lightweight 3D viewer.

The research base is Robbyant's Apache-2.0 LingBot-Map repo. This page turns that streaming reconstruction work into a consumer scan flow, plus a GPU API scaffold that can be deployed behind the demo.

Phone walkthrough Cloud GPU worker GLB viewer target Indoor reconstruction
HouseScan Local demo
1Record video 2Generate model 3View scan
SettingsEndpoint, rooms, retention
Scan profile
10 FPS / 128 window / 8 overlap
Rooms
Living, kitchen, hall, bedroom
Capture route 4 rooms / 3 doorways / 28s target
Keep scan
Expires after 7 days
Budget cap
Run only if estimate stays under $3
Quality gate
Worker holds tiny or sparse uploads
Ready to record Living, kitchen, hall, bedroom
No video selected
More capture optionsBrowser recorder and dry run
DetailsReadiness, progress, capture checks
Frames 1,280 GPU queued Mode indoor Output pending
Scan readiness Waiting for walkthrough
InputNo video selected CoverageLength unknown FramesResolution unknown EndpointLocal demo mode
No model generated yet.
Why this repo

LingBot-Map is close to the right primitive.

A house scan product needs streaming reconstruction, camera pose stability, and a way to keep long indoor walks from falling apart. LingBot-Map's feed-forward reconstruction model is built around that shape: image sequences go in, point clouds and camera motion come out.

The upstream demos are research-entry commands. The product layer should hide those flags behind capture quality checks, GPU job status, output conversion, and a viewer that feels normal to open on a phone or laptop.

Consumer loop

Make scanning feel like one job, not a reconstruction notebook.

01

Capture

Record in the page or choose a continuous walkthrough against an ordered room route with local draft recovery.

02

Stage

Upload with visible progress, sample frames, and keep the raw input tied to one scan receipt.

03

Reconstruct

Run LingBot-Map in windowed indoor mode on a cloud GPU and persist predictions.

04

View

Return a depth-exported GLB point cloud with camera presets and a saved inspection view.

GPU service

A thin Modal worker wraps the LingBot runner.

The scaffold in `gpu/lingbot-map-modal/app.py` exposes `/scans` for browser uploads, publishes `/health` with the source/model contract, validates scan input, spawns a GPU reconstruction job, records a receipt timeline, caches the public LingBot checkpoint, can require a bearer token for expensive routes, serves finished artifacts from a persistent volume, lets active jobs be cancelled, lets completed scan data be deleted, can prewarm the checkpoint before a first scan, stores room coverage metadata with each receipt, returns rough GPU minute and cost estimates, enforces a selected budget cap before GPU work starts, stores a selected 24-hour, 7-day, or 30-day retention window, and gives the browser a shareable viewer URL with a per-scan read-only token in the fragment plus a local QR, so a completed scan can be opened without the expensive API token. Each run also publishes a worker log link, and failed runs return the latest log tail in the receipt. Expired receipts remain readable, but staged input and generated artifacts are removed by the worker. The browser can also run a protected runtime check against the worker image before spending GPU time on a real house scan. Dry-run Preflight can run first, but Generate 3D model waits for `Check runtime` to pass. A phone capture link carries endpoint setup through the URL fragment, with a local QR code so desktop configuration can move to the phone without sending the token to the website server or a QR image service. The Preflight button sends the same recorded walkthrough with `dry_run=true`, so endpoint auth, upload staging, and rough cost can be tested before a real GPU run; when the setup still matches, Generate 3D model reuses that staged upload instead of sending the video a second time. A real receipt is only marked complete after the worker exports a browser-viewable GLB from saved depth predictions and confirms `batch_results.json` did not report failed scenes. The browser also sends an ordered capture route so the receipt can preserve room steps, doorway count, and minimum capture time; the route chips can be reordered before recording so the path matches the real floor plan. In-page recordings are saved as a browser-local 24-hour draft, capped at 256 MB, so a phone refresh can recover the video before upload without storing endpoint bearer tokens in the draft. Completed GLB scans open with room context, camera presets, and a locally saved viewpoint for returning to the same inspection angle. The browser keeps a local recent-scan list with read-only viewer tokens so active, completed, failed, cancelled, and held scan receipts can be reopened later without saving the API token. The page and terminal preflight/verify commands require the endpoint `/health.source.ref` to match the pinned LingBot-Map ref before uploads start.

npm run lingbot:gpu:auth
npm run lingbot:gpu:deploy
LINGBOT_ENDPOINT=https://YOUR-ENDPOINT.modal.run \
  LINGBOT_API_TOKEN=YOUR_TOKEN \
  npm run lingbot:gpu:accept -- \
  --file ./walkthrough.mp4 \
  --rooms entry,living,kitchen \
  --route entry,living,kitchen \
  --allow-low-quality
npm run lingbot:smoke -- --url https://www.matthoffner.com/lingbot-house-scan.html
input GET /health, optional GET /auth/check, optional GET /diagnostics/runtime, optional POST /models/prewarm, then POST /scans with one video or ordered image frames, optional POST /scans/{job_id}/run after dry-run preflight
model cache robbyant/lingbot-map into /models
command python demo_render/batch_demo.py --mode windowed --config demo_render/config/indoor.yaml --save_predictions --no_render
output phone capture handoff, route checklist, runtime diagnostics, recent scan history, quality gate, rough GPU estimate, budget cap, output validation, retention expiry, read-only viewer token, primary_artifact manifest, job receipt timeline, inline GLB viewer with saved viewpoints, delete action, depth GLB export, batch_results.json, worker log
LingBot-Map teaser showing streaming 3D reconstruction point clouds and camera trajectories.
LingBot-Map already demonstrates streaming reconstruction and long video rendering. This POC narrows the interface to a house scan workflow.
Product hardening

The hosted page can point at a real upload-to-GPU loop.

The useful product boundary is clear now: the page owns capture and viewing, while the deployed GPU worker owns reconstruction artifacts. The remaining work is testing a real room-to-room walkthrough, then hardening billing controls and artifact retention around that house-scale path.

Capture QA

Coach for light, pace, route coverage, recording length, and overlapping doorway views.

Cost controls

Add signed uploads, per-user quotas, account billing, and account-level retention defaults.

Viewer polish

Add clipping controls, room filters, and named viewpoints on top of the current camera presets.