← Back to home

LingBot house scan

Turn a phone video into a shareable house scan.

This is a product-shaped proof of concept around LingBot-Map: record or upload a room-to-room walkthrough, send it to a cloud GPU, reconstruct the scene, and open the result as a lightweight 3D viewer.

The research base is Robbyant's Apache-2.0 LingBot-Map repo. This page turns that streaming reconstruction work into a consumer scan flow, plus a GPU API scaffold that can be deployed behind the demo.

Phone walkthrough Cloud GPU worker GLB viewer target Indoor reconstruction
HouseScan / cloud GPU preview endpoint optional
Local demo
Scan profile
10 FPS / 128 window / 8 overlap
Rooms
Living, kitchen, hall, bedroom
Capture route 4 rooms / 3 doorways / 28s target
Keep scan
Expires after 7 days
Budget cap
Run only if estimate stays under $3
Quality gate
Worker holds tiny or sparse uploads
Walkthrough captured Living, kitchen, hall, bedroom
No walkthrough selected
Frames 1,280 GPU queued Mode indoor Output pending
Scan readiness Waiting for walkthrough
InputNo walkthrough selected CoverageLength unknown FramesResolution unknown EndpointLocal demo mode
Sample and cloud artifacts appear here after a scan completes.
Why this repo

LingBot-Map is close to the right primitive.

A house scan product needs streaming reconstruction, camera pose stability, and a way to keep long indoor walks from falling apart. LingBot-Map's feed-forward reconstruction model is built around that shape: image sequences go in, point clouds and camera motion come out.

The upstream demos are research-entry commands. The product layer should hide those flags behind capture quality checks, GPU job status, output conversion, and a viewer that feels normal to open on a phone or laptop.

Consumer loop

Make scanning feel like one job, not a reconstruction notebook.

01

Capture

Record in the page or choose a continuous walkthrough against an ordered room route.

02

Stage

Upload with visible progress, sample frames, and keep the raw input tied to one scan receipt.

03

Reconstruct

Run LingBot-Map in windowed indoor mode on a cloud GPU and persist predictions.

04

View

Return a flythrough MP4 plus GLB point cloud with camera presets and a saved inspection view.

GPU service

A thin Modal worker wraps the LingBot runner.

The scaffold in `gpu/lingbot-map-modal/app.py` exposes `/scans` for browser uploads, publishes `/health` with the source/model contract, validates scan input, spawns a GPU reconstruction job, records a receipt timeline, caches the public LingBot checkpoint, can require a bearer token for expensive routes, serves finished artifacts from a persistent volume, lets active jobs be cancelled, lets completed scan data be deleted, can prewarm the checkpoint before a first scan, stores room coverage metadata with each receipt, returns rough GPU minute and cost estimates, enforces a selected budget cap before GPU work starts, stores a selected 24-hour, 7-day, or 30-day retention window, and gives the browser a shareable viewer URL with a per-scan read-only token in the fragment plus a local QR, so a completed scan can be opened without the expensive API token. Each run also publishes a worker log link, and failed runs return the latest log tail in the receipt. Expired receipts remain readable, but staged input and generated artifacts are removed by the worker. The browser can also run a protected runtime check against the worker image before spending GPU time on a real house scan. Dry-run Preflight can run first, but Run cloud scan waits for `Check runtime` to pass. A phone capture link carries endpoint setup through the URL fragment, with a local QR code so desktop configuration can move to the phone without sending the token to the website server or a QR image service. The Preflight button sends the same recorded walkthrough with `dry_run=true`, so endpoint auth, upload staging, and rough cost can be tested before a real GPU run; when the setup still matches, Run cloud scan reuses that staged upload instead of sending the video a second time. A real receipt is only marked complete after the worker finds a browser-viewable GLB or MP4 artifact and confirms `batch_results.json` did not report failed scenes. The browser also sends an ordered capture route so the receipt can preserve room steps, doorway count, and minimum capture time; the route chips can be reordered before recording so the path matches the real floor plan. Completed GLB scans open with room context, camera presets, and a locally saved viewpoint for returning to the same inspection angle. The browser keeps a local recent-scan list with read-only viewer tokens so completed scans can be reopened later without saving the API token. The page and terminal preflight/verify commands require the endpoint `/health.source.ref` to match the pinned LingBot-Map ref before uploads start.

npm run lingbot:gpu:deploy
LINGBOT_ENDPOINT=https://YOUR-ENDPOINT.modal.run \
  LINGBOT_API_TOKEN=YOUR_TOKEN \
  npm run lingbot:cloud:preflight -- --no-upload --runtime-diagnostics
LINGBOT_ENDPOINT=https://YOUR-ENDPOINT.modal.run \
  LINGBOT_API_TOKEN=YOUR_TOKEN \
  npm run lingbot:capture:url -- --profile preview --rooms entry,living,kitchen
LINGBOT_ENDPOINT=https://YOUR-ENDPOINT.modal.run \
  LINGBOT_API_TOKEN=YOUR_TOKEN \
  npm run lingbot:cloud:verify -- \
  --file ./walkthrough.mp4 \
  --rooms entry,living,kitchen \
  --allow-low-quality
npm run lingbot:smoke -- --url https://www.matthoffner.com/lingbot-house-scan.html
input GET /health, optional GET /auth/check, optional GET /diagnostics/runtime, optional POST /models/prewarm, then POST /scans with one video or ordered image frames, optional POST /scans/{job_id}/run after dry-run preflight
model cache robbyant/lingbot-map into /models
command python demo_render/batch_demo.py --mode windowed --config demo_render/config/indoor.yaml --save_predictions --save_glb
output phone capture handoff, route checklist, runtime diagnostics, recent scan history, quality gate, rough GPU estimate, budget cap, output validation, retention expiry, read-only viewer token, primary_artifact manifest, job receipt timeline, inline GLB viewer with saved viewpoints, delete action, MP4 flythrough, batch_results.json, worker log
LingBot-Map teaser showing streaming 3D reconstruction point clouds and camera trajectories.
LingBot-Map already demonstrates streaming reconstruction and long video rendering. This POC narrows the interface to a house scan workflow.
Product hardening

The hosted page can point at a real upload-to-GPU loop.

The useful product boundary is clear now: the page owns capture and viewing, while the GPU worker owns reconstruction artifacts. The remaining work is deploying the endpoint with model weights, adding billing controls, artifact retention tuning, and testing a real room-to-room walkthrough.

Capture QA

Coach for light, pace, route coverage, recording length, and overlapping doorway views.

Cost controls

Add signed uploads, per-user quotas, account billing, and account-level retention defaults.

Viewer polish

Add clipping controls, room filters, and named viewpoints on top of the current camera presets.