# Satyajit Ghana — full content corpus > Head of Engineering @ Inkers Technology. I build deep-learning systems, 3D perception, and high-performance infra. > Generated from the content layer. Index: /llms.txt --- # About Satyajit Ghana > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/about I build deep-learning systems, 3D perception, and high-performance infra. Head of Engineering at Inkers, where I work on industrial AI — 3D perception, deep learning, and the systems that ship it to production. I write custom neural networks, CUDA kernels, LiDAR/point-cloud pipelines, and high-performance C++/gRPC services. Previously taught MLOps (EMLO 2.0) and computer vision (EVA 4.0) at The School of AI. ## Links - github: https://github.com/satyajitghana - linkedin: https://www.linkedin.com/in/satyajitghana/ - x: https://x.com/thesudoer_ - medium: https://satyajitghana.medium.com/ - scholar: https://scholar.google.com/citations?user=rZCRakQAAAAJ&hl=en - website: https://satyajit.si/ - email: satyajitghana7@gmail.com --- # Resume — Satyajit Ghana > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/resume > pdf: https://ai.satyajit.si/satyajit-ghana-resume.pdf > json: https://ai.satyajit.si/resume.json ```json { "contact": { "name": "Satyajit Ghana", "title": "Head of Engineering", "company": { "name": "Inkers Technology", "url": "https://inkers.ai" }, "location": "Bengaluru, India", "email": "satyajitghana7@gmail.com", "github": "https://github.com/satyajitghana", "linkedin": "https://www.linkedin.com/in/satyajitghana/", "website": "https://ai.satyajit.si" }, "summary": "Engineering leader and deep-learning systems builder. I lead engineering for industrial-AI products — 3D perception, LiDAR/point-cloud pipelines, and structural-defect analysis — and ship them as high-performance C++/CUDA/gRPC services. Two USPTO patents pending. Previously taught MLOps and computer vision.", "experience": [ { "organization": "Inkers Technology", "url": "https://inkers.ai", "location": "Bengaluru, India", "roles": [ { "title": "Head of Engineering", "start": "2024-01" }, { "title": "Deep Learning Software Engineer", "start": "2022-06", "end": "2024-01" }, { "title": "Deep Learning Associate", "start": "2021-07", "end": "2022-06" } ], "summary": "Lead engineering for industrial-AI products across 3D perception, LiDAR/point-cloud pipelines, and structural-defect analysis.", "highlights": [ "Lead engineering for industrial-AI products: 3D perception, LiDAR/point-cloud pipelines, and structural-defect analysis.", "Design custom neural networks, CUDA kernels, and high-performance C++/gRPC services for production deployment.", "Named inventor on 2 USPTO patents pending (assigned to Inkers); grew from Deep Learning Associate to Head of Engineering." ] }, { "organization": "The School of A.I.", "location": "Remote", "roles": [ { "title": "MLOps Instructor", "start": "2022-01", "end": "2023-01" } ], "summary": "Designed and taught MLOps and contributed to computer-vision curriculum.", "highlights": [ "Designed and taught EMLO 2.0 — an end-to-end MLOps course (training, packaging, deployment, monitoring).", "Contributed to EVA 4.0, the deep computer-vision program." ] } ], "education": [ { "institution": "M.S. Ramaiah University of Applied Sciences", "location": "Bangalore, India", "degree": "B.Tech", "end": "2021", "highlights": [ "CGPA 9.78/10", "Silver Medalist" ] } ], "skills": [ { "name": "Deep Learning", "skills": [ "PyTorch", "TensorFlow", "Computer Vision", "3D / Point Clouds / LiDAR", "GenAI (SDXL, LLMs)" ] }, { "name": "Systems", "skills": [ "C++", "C", "Rust", "CUDA", "gRPC", "MongoDB" ] }, { "name": "MLOps", "skills": [ "Kubernetes", "AWS", "GCP", "Docker" ] }, { "name": "Web", "skills": [ "TypeScript", "React", "Next.js" ] } ], "projects": [ { "name": "torch-point-ops", "description": "High-performance PyTorch operators for point-cloud and 3D geometry processing.", "url": "https://github.com/satyajitghana/torch-point-ops" }, { "name": "PV-LIO-for-HBA", "description": "Point-to-voxel LiDAR-inertial odometry adapted for hierarchical bundle adjustment.", "url": "https://github.com/satyajitghana/PV-LIO-for-HBA" }, { "name": "ige_lio", "description": "Iterated error-state Kalman-filter LiDAR-inertial odometry experiments.", "url": "https://github.com/satyajitghana/ige_lio" }, { "name": "sdxl-dreambooth-finetune", "description": "DreamBooth fine-tuning pipeline for Stable Diffusion XL subject personalization.", "url": "https://github.com/satyajitghana/sdxl-dreambooth-finetune" }, { "name": "TSAI-DeepVision-EVA4.0", "description": "Coursework and experiments from The School of AI's EVA 4.0 computer-vision program.", "url": "https://github.com/satyajitghana/TSAI-DeepVision-EVA4.0" }, { "name": "PadhAI-Course", "description": "Implementations and notes from the PadhAI deep-learning course.", "url": "https://github.com/satyajitghana/PadhAI-Course" } ], "publications": [ { "title": "Adaptive Visual Learning Using Augmented Reality and Machine Learning Techniques", "publisher": "Journal of Computational and Theoretical Nanoscience", "date": "2020-01-01", "details": "Vol. 17, No. 11, pp. 4952–4956", "doi": "10.1166/jctn.2020.8982", "url": "https://doi.org/10.1166/jctn.2020.8982" } ], "patents": [ { "title": "Method and System for Performing Structural Defect Analysis in a Structural Environment", "status": "Pending", "applicationNumber": "US 19/634,310", "filed": "2026-03-31", "assignee": "Inkers Technology" }, { "title": "Data Acquisition Device", "status": "Pending", "applicationNumber": "US 19/634,339", "filed": "2026-03-31", "assignee": "Inkers Technology" } ] } ``` --- # Health — biomarker panel > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/health > panel date: 2026-05-15 ```json { "panelDate": "2026-05-15", "stack": [ { "type": "device", "name": "Apple Watch", "url": "https://www.apple.com/watch/" }, { "type": "device", "name": "Withings Body+" }, { "type": "supplement", "name": "Vitamin D3" }, { "type": "supplement", "name": "Omega-3" }, { "type": "supplement", "name": "Creatine" }, { "type": "supplement", "name": "Magnesium Glycinate" } ], "biomarkers": [ { "key": "ldl-c", "label": "LDL-C", "value": 128, "unit": "mg/dL", "category": "cardiovascular", "weight": 2, "optimalRange": { "max": 100 }, "note": "Elevated — primary driver of atherosclerotic risk.", "status": "elevated" }, { "key": "hdl-c", "label": "HDL-C", "value": 52, "unit": "mg/dL", "category": "cardiovascular", "optimalRange": { "min": 40 }, "status": "optimal" }, { "key": "apob", "label": "ApoB", "value": 105, "unit": "mg/dL", "category": "cardiovascular", "weight": 1.5, "optimalRange": { "max": 90 }, "note": "Elevated — atherogenic particle count above target.", "status": "elevated" }, { "key": "triglycerides", "label": "Triglycerides", "value": 88, "unit": "mg/dL", "category": "cardiovascular", "optimalRange": { "max": 150 }, "status": "optimal" }, { "key": "hba1c", "label": "HbA1c", "value": 5.4, "unit": "%", "category": "metabolic", "weight": 1.5, "optimalRange": { "max": 5.7 }, "status": "optimal" }, { "key": "fasting-glucose", "label": "Fasting Glucose", "value": 96, "unit": "mg/dL", "category": "metabolic", "optimalRange": { "min": 70, "max": 99 }, "status": "optimal" }, { "key": "fasting-insulin", "label": "Fasting Insulin", "value": 8.1, "unit": "µIU/mL", "category": "metabolic", "optimalRange": { "max": 8 }, "note": "Borderline — marginally above optimal fasting insulin.", "status": "borderline" }, { "key": "alt", "label": "ALT", "value": 22, "unit": "U/L", "category": "liver_kidney", "optimalRange": { "max": 40 }, "status": "optimal" }, { "key": "ast", "label": "AST", "value": 24, "unit": "U/L", "category": "liver_kidney", "optimalRange": { "max": 40 }, "status": "optimal" }, { "key": "creatinine", "label": "Creatinine", "value": 0.95, "unit": "mg/dL", "category": "liver_kidney", "optimalRange": { "min": 0.7, "max": 1.3 }, "status": "optimal" }, { "key": "egfr", "label": "eGFR", "value": 99, "unit": "mL/min", "category": "liver_kidney", "optimalRange": { "min": 90 }, "status": "optimal" }, { "key": "tsh", "label": "TSH", "value": 2.1, "unit": "mIU/L", "category": "hormonal", "optimalRange": { "min": 0.5, "max": 4 }, "status": "optimal" }, { "key": "testosterone", "label": "Testosterone", "value": 610, "unit": "ng/dL", "category": "hormonal", "optimalRange": { "min": 300, "max": 1000 }, "status": "optimal" }, { "key": "vitamin-d", "label": "Vitamin D", "value": 24, "unit": "ng/mL", "category": "nutritional", "weight": 1.5, "optimalRange": { "min": 30, "max": 100 }, "note": "Low — below the optimal 25-OH vitamin D range.", "status": "low" }, { "key": "vitamin-b12", "label": "Vitamin B12", "value": 540, "unit": "pg/mL", "category": "nutritional", "optimalRange": { "min": 400, "max": 1000 }, "status": "optimal" }, { "key": "ferritin", "label": "Ferritin", "value": 38, "unit": "ng/mL", "category": "nutritional", "optimalRange": { "min": 50, "max": 300 }, "note": "Low — iron stores below the optimal floor.", "status": "low" }, { "key": "hemoglobin", "label": "Hemoglobin", "value": 15.1, "unit": "g/dL", "category": "blood_panel", "optimalRange": { "min": 13.5, "max": 17.5 }, "status": "optimal" }, { "key": "wbc", "label": "WBC", "value": 6.2, "unit": "10³/µL", "category": "blood_panel", "optimalRange": { "min": 4, "max": 11 }, "status": "optimal" }, { "key": "platelets", "label": "Platelets", "value": 245, "unit": "10³/µL", "category": "blood_panel", "optimalRange": { "min": 150, "max": 400 }, "status": "optimal" }, { "key": "resting-hr", "label": "Resting HR", "value": 58, "unit": "bpm", "category": "vitals", "optimalRange": { "min": 40, "max": 60 }, "status": "optimal" }, { "key": "bp-systolic", "label": "BP Systolic", "value": 132, "unit": "mmHg", "category": "vitals", "weight": 1.5, "optimalRange": { "max": 120 }, "note": "Elevated — systolic pressure above the optimal ceiling.", "status": "borderline" }, { "key": "vo2max", "label": "VO₂max", "value": 47, "unit": "mL/kg/min", "category": "vitals", "optimalRange": { "min": 42 }, "status": "optimal" } ] } ``` --- # Now > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/now ```json { "updated": "2026-06-04", "items": [ "Building ai.satyajit.si — a dual-native personal site maintained by a crew of Claude agents that ship every change as a reviewed PR.", "Leading industrial-AI 3D perception at Inkers: turning LiDAR and camera capture into structural-defect analysis that runs in production.", "Writing CUDA point-cloud kernels — fused nearest-neighbour, sampling, and grouping ops to take the Python side out of the hot path.", "Reading deeply on LiDAR odometry and point-cloud registration, with an eye toward tighter, faster SLAM front-ends." ] } ``` --- # Uses > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/uses ```json [ { "section": "Workstation", "items": [ { "name": "Custom GPU workstation", "note": "NVIDIA RTX-class GPU for CUDA + training" }, { "name": "Linux", "note": "Ubuntu — primary dev OS" } ] }, { "section": "Editor", "items": [ { "name": "VS Code", "note": "daily driver" }, { "name": "Claude Code", "note": "agentic pair-programming in the terminal" }, { "name": "Fonts", "note": "JetBrains Mono today, migrating toward IBM Plex Mono" } ] }, { "section": "Terminal", "items": [ { "name": "zsh", "note": "shell" }, { "name": "tmux", "note": "session multiplexing" } ] }, { "section": "Stack highlights", "items": [ { "name": "PyTorch + CUDA", "note": "custom kernels for point-cloud ops" }, { "name": "C++ / gRPC", "note": "high-performance perception services" }, { "name": "Next.js + Tailwind", "note": "this site" } ] } ] ``` --- # Reading > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/reading ```json [ { "type": "paper", "title": "PV-LIO: A Probabilistic Voxel-based LiDAR-Inertial Odometry Framework", "status": "reading", "note": "PLACEHOLDER — voxel-probabilistic front-end ideas for tighter LIO." }, { "type": "book", "title": "Programming Massively Parallel Processors", "author": "Hwu, Kirk, El Hajj", "status": "reading", "note": "PLACEHOLDER — the CUDA reference I keep coming back to." }, { "type": "paper", "title": "FAST-LIO2: Fast Direct LiDAR-Inertial Odometry", "status": "read", "note": "PLACEHOLDER — iterated Kalman filter without feature extraction." }, { "type": "book", "title": "Multiple View Geometry in Computer Vision", "author": "Hartley & Zisserman", "status": "queued", "note": "PLACEHOLDER — geometry fundamentals refresher." }, { "type": "paper", "title": "3D Gaussian Splatting for Real-Time Radiance Field Rendering", "status": "queued", "note": "PLACEHOLDER — splatting as a reconstruction primitive." } ] ``` --- # Vritti UI > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/vritti > stack: React 19, Next.js 16, Tailwind v4, Motion, Three.js, TypeScript Components crafted for design engineers — animated components and blocks you copy and paste into your shadcn/ui project. No package installs, full ownership. ```bash npx shadcn@latest add "https://vritti.satyajit.si/r/shimmer-button" ``` **560+ components** across 14 categories — backgrounds, animations, text effects, buttons, cards, charts, layouts, navigation, carousels, cursors, inputs, media, shaders, and the genuinely weird ones. **240+ blocks** — pre-built page sections for real apps: auth, pricing, e-commerce, billing, dashboards, modals, testimonials, footers, FAQ, AI & Web3. Beyond components, it ships a set of **creative tools** that run online or install into your project: Art Studio (9 generative-art tools), Background Studio, Shape Studio (an SVG editor), Texture Studio (30+ stackable WebGL filters), Shader Studio (87 production-ready shaders), Dither Studio, a visual Theme Editor (42 presets, Google Fonts, contrast checker, CSS export), and native haptics. There's an interactive playground for tuning component props live, and AI components (chat streaming, image generation, voice, AI search) for LLM-native UIs. Everything is copy-paste, shadcn-style — you own the source. It's the most ambitious thing on this page: a full component ecosystem, a design-tool suite, and a theme system, built on React 19, Next.js 16, Tailwind v4, Motion, and Three.js. --- # torch-point-ops > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/torch-point-ops > stack: PyTorch, CUDA, C++, Python > repo: https://github.com/satyajitghana/torch-point-ops A PyTorch library for 3D point operations — nearest-neighbour, sampling, and grouping primitives — implemented as fused CUDA kernels for throughput on large point clouds. Built to back LiDAR and 3D-perception pipelines where the Python-side ops were the bottleneck. Exposes a clean `torch` autograd-compatible API over the native kernels. --- # fabrik > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/fabrik > stack: TypeScript, React, Vercel AI SDK, Zod, Motion > repo: https://github.com/satyajitghana/fabrik Most LLM apps are text in, text out. **fabrik** lets the model answer with *interface* — it picks and fills real React components (cards, forms, charts, pickers) instead of emitting a wall of prose. The AI decides what to show. It's provider-agnostic: wire it to any model through the Vercel AI SDK (Gemini, OpenAI, Anthropic, …), keep the API key server-side in a single route handler, and render the streamed component tree on the client. Components are validated with Zod schemas, so the model can only produce UIs you've defined — no arbitrary markup, no injection surface. The bet behind it: as models get better at tool use, the next interface isn't a chat bubble — it's a UI the model assembles on the fly for the task at hand. --- # rentree > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/rentree > stack: Next.js, React, TypeScript, Supabase, Drizzle ORM, Tailwind v4 > repo: https://github.com/satyajitghana/rentree Rentree connects people with organic farms across India. Instead of buying produce off a shelf, you rent a plot or adopt a tree: your farmer grows your food, sends real photo updates as it grows, and ships it when it's ready. Rent a plot for a season (tomatoes, herbs, rice — whatever grows), or adopt an apple, orange, or mango tree and get the whole harvest. Every tree carries a QR code, so you can scan and trace the full journey from planting to your plate. A clean Next.js 16 app on Supabase + Drizzle, with the provenance trail as the trust mechanism that makes the model work. --- # Quantum Orbital Visualizer > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/atoms > stack: Next.js, Three.js, Rust (WASM), C++23 (WASM), Tailwind v4, Zustand > repo: https://github.com/satyajitghana/atoms Each visualization is the probability density |ψ|² of a hydrogen-like electron, computed from the exact solutions to the Schrödinger equation and rendered as up to 500K glowing particles in real time. Drive the quantum numbers (n, l, m) and watch the probability cloud morph as you go. The fun engineering bit is the **dual WASM backends**: the same physics is implemented three ways — JavaScript, Rust→WASM (~41 KB), and C++23→WASM (~28 KB) — and you can switch the compute engine at runtime. Same orbitals, different implementations, side-by-side for benchmarking. There's also an Element Explorer: the full periodic table, where clicking any of the 118 elements parses its electron configuration into individual orbitals you can then visualize one by one. --- # BOTCHA > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/botcha > stack: Next.js, TypeScript, Redis, crypto (SHA-256 / HMAC / JWT) > repo: https://github.com/satyajitghana/botcha Only an autonomous agent with runtime access to HTTP, cryptography, and byte manipulation can pass — which is the whole joke, and the whole point. Each challenge is 256 random bytes plus 2–4 byte-level transformation steps written in randomized natural language, inside a 30-second window: fast enough for a machine, hopeless for a human copy-pasting into a REPL. The agent must decode the base64, execute each transform in order, concatenate the raw byte outputs, `SHA-256` the result, then `HMAC-SHA256(key=nonce, message=answer)` and submit both — proving it actually did the computation. On success it gets a short-lived JWT. It's a small, sharp take on a real question the agent era raises: if the web increasingly wants to *let bots in* and keep humans out, what does that gate look like? --- # penora > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/penora > stack: TypeScript, React, Canvas, npm package > repo: https://github.com/satyajitghana/penora Type a string and penora animates it into natural handwriting — not a font fade-in, but stroke-by-stroke drawing driven by the glyph contours, with pen physics layered on top: pressure tapering at stroke ends, seeded jitter so each render is subtly different, and micro-wobble for that hand-drawn quality. Export the result as video or GIF. Shipped as an npm package (`npm i penora`) and as a shadcn registry component you can drop straight into a project. It's the kind of small, self-contained library that's satisfying to build: a tight problem (make text look handwritten and *alive*) with a lot of room for craft in the details. --- # Hyr > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/hyr > stack: Next.js, TypeScript, Tailwind v4, Google Gemini > repo: https://github.com/satyajitghana/hyr Drop in any PDF resume and Hyr parses it into a structured, fully editable format — inline editing for every section, no forms or modals. From there it tailors the resume per job posting, aligning keywords and tone to the listing so it gets past ATS screening and reads as a real fit. Build, tailor, optimize, and track the whole job search in one place. The AI work runs on Gemini behind a server route; the parsing-to-structured-data step is the part that makes the rest of the editing experience feel clean rather than fighting a PDF. --- # MockLab > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/mocklab > stack: Next.js, Tailwind v4, shadcn/ui, Motion, html-to-image > repo: https://github.com/satyajitghana/mocklab MockLab generates pixel-perfect social mockups across 21+ platforms — social posts (X, LinkedIn, Instagram post/story, Reddit, Threads, YouTube, TikTok), chat messages (WhatsApp, Telegram, Slack, Discord, iMessage, Snapchat), and Gmail. Edit on the left, watch the live preview on the right, download as 2× PNG. Everything is configurable: verified badges, reactions, timestamps, read receipts, media uploads, and per-message editing for the chat mockups — all with dark/light theming for both the app and every mockup. It's a deceptively deep UI project: each platform is its own little pixel-accurate design system to reproduce. --- # tapcn > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/projects/tapcn > stack: React Native, Expo, TypeScript, CLI > repo: https://github.com/satyajitghana/tapcn tapcn brings the shadcn model to mobile: not an npm component library, but a CLI that copies beautifully-designed, accessible component *source* directly into your Expo project. No dependency lock-in, no version conflicts — just code you control and can edit. ```bash npx @tapcn/cli init npx @tapcn/cli add button card input text ``` Components work on iOS, Android, and Web from a single source. The whole appeal of the shadcn approach — own your components, theme them freely, no black-box library — has been missing on React Native; tapcn fills that gap. --- # My site's chatbot was stuffing 273k tokens into every message. I gave it tools instead. > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/blog/site-agent-dynamic-tools > date: 2026-07-20 > tags: agents, llm, tools, retrieval, systems The assistant embedded on this site — the `⌘K` console, `/api/ask`, and the `ask_satyajit` MCP tool — worked. It was also doing something faintly absurd. Every single request built its system prompt like this: ```ts // the old lib/chat.ts, abridged const sections = [] for (const page of ["about", "resume", "health", "now", "uses", "reading"]) sections.push(await dataPageMarkdown(page)) for (const item of getAllContent()) // every article, blog, log, digest… sections.push(contentMarkdown(item.kind, item.slug)) return [PERSONA, RULES, "=== CONTENT CORPUS ===", sections.join("\n---\n")].join("\n") ``` It pasted **the entire site** into the prompt and let the provider's context cache eat the cost. That's a defensible move when the corpus is a page or two. Mine isn't anymore: ```text old full-corpus system prompt: 1,090,854 chars ≈ 273,000 tokens ``` The code even carried a comment — *"if the corpus ever approaches ~100K tokens, switch to retrieval"* — that reality had quietly sailed past nearly 3× over. Two hundred seventy-three thousand tokens is past the input window of the model serving it. The chat was running on whatever survived truncation. Time to do the thing the comment said. ## The fix: retrieve, don't dump I rebuilt the assistant as a small **tool-using agent**. The system prompt now carries only a compact **catalog** — every page's `kind/slug`, title, and one-line description, about **5k tokens** — and the agent pulls the bodies it actually needs through tools. Three of them: ```ts // lib/chat.ts — the whole tool set search_content({ query, limit }) // BM25 over the site (ranked, returns section + snippet) get_content({ kind, slug }) // fetch one page's full markdown, on demand think({ thought }) // a reasoning scratchpad; records the thought, returns nothing ``` A question now flows: read the catalog → `search_content` (or jump straight to `get_content` if the catalog already names the page) → read one or two pages → optionally `think` to reconcile them → answer, with citations. The base prompt is fixed and small; the variable cost is the one-to-three pages it fetched, not the other sixty-five it didn't. ```text before: ~273k tokens, every request, whether relevant or not after: ~5k catalog + ~2–10k per page actually fetched ``` That's the same **dynamic loading** idea from [Kimi K3's tool-calling guide](https://platform.kimi.ai/docs/guide/kimi-k3-tool-calling-best-practice) — their point is that a big upfront payload "eats up context and makes the model more likely to pick the wrong thing." Kimi loads *tool schemas* on demand; my corpus is the payload, so I load *content* on demand. Same principle, one layer down. ## Three tools, on purpose The tool count is a decision, not an accident. Mario Zechner's [pi coding agent](https://mariozechner.at/posts/2025-11-30-pi-coding-agent/) makes the case that "four tools are all you need" — `read`, `write`, `edit`, `bash` — and that MCP servers which "dump their entire tool descriptions into your context on every session" are the anti-pattern. A read-only site agent needs fewer still: search, fetch, think. Each description is a couple of sentences. The combined tool surface is under a thousand tokens, so keeping it declared upfront (rather than lazily loading schemas, which only pays off with dozens of tools) is the right call here — the honest version of "dynamic tools" for a small surface is *don't have a big one*. The one genuinely new tool is `think`, from Anthropic's [think-tool post](https://www.anthropic.com/engineering/claude-think-tool). It does nothing — literally logs the thought and returns: ```ts think: tool({ description: "Think out loud: plan which pages to fetch, or check a draft answer against the sources you read. Records the thought and returns nothing new.", inputSchema: z.object({ thought: z.string() }), execute: async ({ thought }) => ({ ok: true, thought }), }), ``` That looks pointless until you watch a multi-step tool run. Between `search_content` and the final answer, the model has raw tool output sitting in context and no designated place to reason over it before committing. `think` is that place — a scratchpad that keeps the "what did I just read, and does it actually answer the question" step from being skipped. Anthropic reports it buying a large margin on multi-step tool tasks; the appeal for a retrieval agent is exactly that it makes the model *check the page it fetched* instead of answering from the snippet. ## The loop The agent loop itself is one call, courtesy of the Vercel AI SDK — `stopWhen` bounds how many tool round-trips it may take before it has to answer: ```ts const result = streamText({ model: chatModel("main"), system, // persona + rules + the 5k-token catalog messages, tools: agentTools(), // search_content, get_content, think stopWhen: stepCountIs(6), // search → read → think → answer is ~4; 6 is headroom }) ``` The same tool set backs all three surfaces — the streaming `⌘K` console (`/api/chat`), the one-shot `/api/ask`, and the `ask_satyajit` MCP tool — so there's one harness, not three. And `search_content` is the [Contextual BM25](/blog/site-search-contextual-bm25) engine I'd already built for the site's `/search`, now doing double duty as the agent's retriever. The pieces compose: the search work made the harness possible. Honest scope. This is retrieval over a small, single-author corpus, not a coding agent — the lessons transfer but the stakes are lower. `think`'s benefit is task-dependent (Anthropic saw big gains on policy-heavy multi-step tasks, near-zero on simple ones); on a two-hop lookup it mostly earns its keep by stopping the model from answering off the snippet. And the retriever is lexical BM25 — no embeddings yet, so a pure paraphrase with no shared terms can still miss the right page. The catalog is the safety net there: the model can see every title even when search comes up short. ## The take The old design wasn't wrong when it was written — it was wrong at 273k tokens. The rebuild is the [agent-harness](/articles/agent-harness) framing applied to my own site: the model barely changed, but the loop, the tool set, and the context policy around it changed completely. Give the agent a map and three tools and let it fetch, instead of force-feeding it the whole library and hoping the answer survives the truncation. Retrieve, don't dump. --- # Giving my own search the Contextual BM25 treatment > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/blog/site-search-contextual-bm25 > date: 2026-07-20 > tags: search, retrieval, bm25, rag, information-retrieval I recently wrote two pieces back to back: one on [BM25](/articles/bm25), the ranking function that refuses to die, and one on [Contextual Retrieval](/blog/contextual-retrieval), Anthropic's fix for chunks that forget where they came from. Then I opened `lib/search.ts` — the thing powering this site's own `/search` and the `/api/search` endpoint agents hit — and found this: ```ts // the old lib/search.ts, abridged for (const item of getAllContent()) { const fields = [item.title, item.description, item.tags, item.body] for (const text of fields) { if (text.toLowerCase().includes(q)) { // q = the whole lowercased query results.push({ /* … a ±120-char window around the hit */ }) break } } } ``` A substring `indexOf`. It has three problems, and the third is the one that actually bites: 1. **No ranking.** The first document that contains the substring wins, in date order. A tight, on-topic match and an incidental mention are indistinguishable. 2. **No notion of rarity or length.** Matching "the" counts the same as matching "kalman". 3. **Multi-word queries fall off a cliff.** `q` is the *entire* query string, so `includes(q)` needs that **exact phrase** somewhere in the text. Nobody searches in verbatim phrases. `"bm25 length normalization"` returns **zero results** — those three words are all in the BM25 article, just never contiguous. So I ate my own cooking. Here's the rebuild. ## BM25, the same formula I just wrote up The [BM25 walkthrough](/articles/bm25) has the whole derivation; the code is a direct transcription of it. Lucene's non-negative IDF, term-frequency saturation at `k1 = 1.2`, length normalization at `b = 0.75`: ```ts // lib/search.ts — the ranking core, straight from the article's formula const K1 = 1.2 const B = 0.75 function idf(term: string, ix: Index): number { const n = ix.df.get(term) ?? 0 return Math.log(1 + (ix.n - n + 0.5) / (n + 0.5)) } // per query term present in a chunk: const f = chunk.tf.get(term) ?? 0 const norm = K1 * (1 - B + (B * chunk.len) / ix.avgdl) score += idf(term, ix) * (f * (K1 + 1)) / (f + norm) ``` Two things fall out for free. The query is **tokenized** into terms and each is scored independently, so multi-word queries just work — no phrase has to exist. And rare terms dominate: `kalman` carries far more weight than `filter`, because `idf` collapses for common words. No stopword list, same as the article promised. I keep a **postings map** (`term → chunk indices`) so a query only touches chunks that actually share a term, instead of scanning the whole corpus. The corpus is one person's writing, so this is overkill — but it's the same inverted-index shape a real engine uses, and it's three lines. ## Contextual chunks, minus the Claude call BM25 alone still has the chunk-amnesia problem from the [Contextual Retrieval post](/blog/contextual-retrieval): if I split an article into passages and index each one alone, a passage that reads "it drops 41% at parity" has no idea it's *about the Harness Effect, in the section on the controlled swap*. A query for either misses it. Anthropic's fix is to have Claude write a one-line context for every chunk before indexing. Mine is cheaper and dumber: the context a chunk lost is sitting right there in the document's frontmatter and headings. So before indexing, every chunk inherits its document's **title, description, tags, and nearest heading** — deterministically, at request time, no model, no build step: ```ts // each chunk is packed with weighted terms: its own body, its heading, and the // document context it would otherwise have lost when the body was split. const contextTokens = tokenize([title, description, tags.join(" ")].join(" ")) for (const t of bodyTokens) add(t, 1) for (const t of headingTokens) add(t, 1) // section-local: full weight for (const t of contextTokens) add(t, CONTEXT_WEIGHT) // 0.5 — present, not dominant ``` The honest caveat: this is a **poor man's** Contextual Retrieval. Claude's per-chunk context can say things the frontmatter can't ("the previous quarter's revenue was \$314M"); my version can only replay the structural context the document already carries. And prepending the *same* title to every chunk of a document inflates those terms' document-frequency, which is exactly why the context terms get `CONTEXT_WEIGHT = 0.5` instead of full weight — present enough to make the chunk findable by its subject, quiet enough not to drown the passage's own words. It's the 80% of the win for 0% of the inference cost, which is the right trade for a static personal site. Each result also reports **which section** it matched — the nearest heading rides along as the result's `field`, so `/api/search` tells an agent not just *which* document but *where* in it. ## Before / after Same queries, old substring search vs. the new Contextual BM25, on the live corpus: ```text query substring indexOf contextual BM25 (top hit) -------------------------------------------------------------------------------------- "bm25 length normalization" 0 results (no verbatim articles/bm25 phrase) [Start with TF-IDF, and its two flaws] "revenue grew quarter" 0 results blog/contextual-retrieval [The problem: chunks forget …] "kalman filter" first dated doc that articles/kalman-filter (8.4) contains the phrase, + fast-lio2 (8.6, cites it) unranked ranked by relevance, not date "reciprocal rank fusion" 0 results blog/contextual-retrieval [A dependency-free repro] ``` The multi-word queries are the story. Under substring, anything that isn't a verbatim phrase — which is almost everything a person types — returned nothing. Under BM25 every term contributes, so the query finds the document even when its words are scattered across a paragraph, and the contextual prefix pulls in matches on the *subject* of a passage, not just its literal words. ## It's live Try it: [`/search?q=length+normalization`](/search?q=length+normalization). Agents get the ranked, scored JSON with the section label: ```bash curl -s "https://ai.satyajit.si/api/search?q=contextual%20retrieval&limit=3" | jq '.results[] | {slug, field, score}' ``` What this still isn't: it's the **lexical half** only. The [contextual retrieval post](/blog/contextual-retrieval) makes the case that the real win is *fusing* BM25 with a dense retriever, and that reranking the shortlist is what takes the failure rate the last mile. There are no embeddings here yet — a query has to share actual terms with a chunk, so a pure paraphrase with no lexical overlap can still miss. For a corpus this size, lexical BM25 over contextualized chunks is the honest 90% solution; the semantic half is the next commit, not this one. ## The takeaway The whole change is `lib/search.ts` — a few hundred lines, no dependencies, no index server, no model at request time. It's the two ideas I'd just written about, applied to the smallest possible target: my own search bar. BM25 for ranking, structural context for the chunk-amnesia problem, and an honest note about the half I haven't built yet. Writing about a technique is a good way to understand it; running it in your own site is a better one. --- # Contextual Retrieval, with a runnable repro and a browser playground > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/blog/contextual-retrieval > date: 2026-07-17 > tags: rag, retrieval, claude, embeddings, playground [Anthropic's Contextual Retrieval post](https://www.anthropic.com/engineering/contextual-retrieval) has been on my to-read list for a while. It targets the oldest bug in RAG, so I finally sat down, built a dependency-free reproduction, and wired up a browser playground. This is the write-up. ## The problem: chunks forget where they came from Standard RAG splits documents into chunks and indexes each chunk on its own. That destroys context. Anthropic's example is perfect: > `The company's revenue grew by 3% over the previous quarter.` Which company? Which quarter? The chunk can't say. A query like *"how did ACME's Q2 revenue change?"* has **no terms to match** against that chunk — the words "ACME" and "Q2 2023" live in the *document*, not the *chunk*. Both lexical (BM25) and semantic (embedding) retrieval miss it. ## The fix: let Claude situate each chunk Contextual Retrieval prepends a short, chunk-specific context to each chunk **before** you embed it and **before** you build the BM25 index — Anthropic calls these **Contextual Embeddings** and **Contextual BM25**. The context is generated by Claude, given the whole document. The same chunk becomes: > `This chunk is from an SEC filing on ACME corp's performance in Q2 2023; the previous quarter's revenue was $314 million. The company's revenue grew by 3% over the previous quarter.` Now the owner and the period are *in the chunk*, so retrieval can find it. ## Watch it happen Here is the whole pipeline in your browser — the document, its chunks, the chunks with context prepended, then retrieval. The scoring is a real BM25 and a TF-IDF cosine (a stand-in for an embedding model); the contextual prefixes stand in for Claude's output. Step to **4 · retrieve**, pick a query, and watch the right chunk climb from the standard column to the contextual one: Toggle between BM25, embeddings, and their fusion. The pattern holds across all three: the answer chunk is buried on standard chunks and near the top once each chunk carries its context. ## The one prompt that does the work This is the prompt from the post, verbatim. It runs once per chunk, with the full document supplied as context: ```text {{WHOLE_DOCUMENT}} Here is the chunk we want to situate within the whole document {{CHUNK_CONTENT}} Please give a short succinct context to situate this chunk within the overall document for the purposes of improving search retrieval of the chunk. Answer only with the succinct context and nothing else. ``` The obvious worry is cost: you re-send the whole document once per chunk. **Prompt caching** kills that. Cache the document once and every chunk in it reads from the cache, which is why Anthropic quotes a one-time **$1.02 per million document tokens**. The document is the stable prefix, so it takes the `cache_control` breakpoint; the chunk and instruction vary and come after it: ```python from anthropic import Anthropic client = Anthropic() CONTEXT_PROMPT = """Here is the chunk we want to situate within the whole document {chunk} Please give a short succinct context to situate this chunk within the overall document \ for the purposes of improving search retrieval of the chunk. Answer only with the \ succinct context and nothing else.""" def situate(doc: str, chunk: str) -> str: resp = client.messages.create( # One Claude call per chunk. For a large index you'd typically drop to a # cheaper model like claude-haiku-4-5 — which is what Anthropic's ~$1.02 / # 1M-doc-token estimate assumes — trading a little context quality for cost. model="claude-opus-4-8", max_tokens=200, messages=[{ "role": "user", "content": [ # The whole document is the stable prefix: cache it ONCE, then every # chunk of this document reads from the cache instead of re-paying for it. {"type": "text", "text": f"\n{doc}\n", "cache_control": {"type": "ephemeral"}}, {"type": "text", "text": CONTEXT_PROMPT.format(chunk=chunk)}, ], }], ) return "".join(b.text for b in resp.content if b.type == "text").strip() ``` Verify the cache is actually working: `resp.usage.cache_read_input_tokens` should be non-zero on every chunk after the first for a given document. If it's zero, something upstream is mutating the document bytes (a timestamp, non-deterministic JSON) and invalidating the prefix. ## A dependency-free repro To convince myself the lift was real and not marketing, I wrote a ~180-line pure-stdlib script: naive chunking, BM25 from scratch, a TF-IDF cosine as an embedding stand-in, and reciprocal rank fusion. The retrieval core is small — here is BM25 and the fusion step: ```python class BM25: def score(self, query, i): dl, tf, s = len(self.docs[i]), self.tf[i], 0.0 for t in query: if t not in tf: continue num = tf[t] * (self.k1 + 1) den = tf[t] + self.k1 * (1 - self.b + self.b * dl / self.avgdl) s += self.idf(t) * num / den return s def rrf(rankings, k=60): # reciprocal rank fusion of BM25 + embedding rankings scores = Counter() for ranking in rankings: for rank, item in enumerate(ranking): scores[item] += 1 / (k + rank + 1) return [i for i, _ in scores.most_common()] ``` Then I index the chunks two ways — plain, and with a one-line context prepended — and measure where the correct chunk lands for a couple of "which entity, which period" queries. The actual output: ```text query method plain rank ctx rank ------------------------------------------------------------------------------------ How did ACME Corp revenue change in Q2 2023? bm25 4 2 How did ACME Corp revenue change in Q2 2023? emb 4 1 How did ACME Corp revenue change in Q2 2023? hybrid 4 2 What happened to Beta Industries revenue in Q3 2023? bm25 5 2 What happened to Beta Industries revenue in Q3 2023? emb 5 1 What happened to Beta Industries revenue in Q3 2023? hybrid 5 2 recall@1 (fraction of queries where the right chunk ranks #1): bm25 plain 0/2 contextual 0/2 emb plain 0/2 contextual 2/2 ``` Contextualizing the chunks moves the answer from rank **4–5** to rank **1–2** across BM25, embeddings, and their fusion — and embeddings recall@1 goes from **0/2 to 2/2**. On this toy corpus fusion lands the answer at #2 rather than #1 (a small-N artifact of RRF), which is a good reminder that the fusion win is an *aggregate* effect — which is exactly what Anthropic's real evaluation measures. ## What the real evaluation found On Anthropic's benchmark (top-20 retrieval failure rate, i.e. `1 - recall@20`), stacking the techniques compounds: - **Contextual Embeddings** alone: `5.7% → 3.7%` — a **35%** cut in the failure rate. - **+ Contextual BM25**: `5.7% → 2.9%` — **49%**. - **+ reranking**: `5.7% → 1.9%` — **67%**. The lexical half matters more than you'd guess — BM25 nails exact identifiers (error codes, ticker symbols, function names) that embeddings smear together, so contextualizing *both* indexes and fusing them beats either alone. ## Things worth copying from the post - **Retrieve top-20, not top-5/10.** Anthropic found 20 the most performant cut for the final context. - **Rerank the shortlist.** Retrieve ~150 candidates, then rerank down to 20 for the answer prompt — that's the step that takes the failure rate from 2.9% to 1.9%. - **Embedding model matters.** Gemini and Voyage embeddings were the standouts in their tests. - **Chunking still matters.** Size, boundary, and overlap all move the numbers — Contextual Retrieval sits on top of good chunking, it doesn't replace it. - **A domain-tuned context prompt beats the generic one.** The template above is a floor, not a ceiling. Be honest about what this costs. Contextual Retrieval adds a Claude call **per chunk** at index time — cheap per token with caching, but real latency and spend when you're indexing millions of chunks, and it has to re-run when documents change. My repro is a minimal reproduction of the *core mechanism* (context lifts rank); the `35 / 49 / 67%` figures are Anthropic's, on their corpus. And my playground scores with a TF-IDF cosine, not a real embedding model — it shows the *shape* of the effect, not production numbers. ## The takeaway The move is almost embarrassingly simple: spend a cheap, cached Claude call per chunk to write down the context a human would need to make sense of it, then index that. It attacks the failure at its source instead of papering over it downstream, it helps lexical and semantic retrieval at the same time, and — as the little repro above shows — you can watch the right chunk climb the rankings the moment the context goes in. --- *Source: [Introducing Contextual Retrieval](https://www.anthropic.com/engineering/contextual-retrieval) (Anthropic). The prompt and the `35 / 49 / 67%` numbers are theirs; the repro and the browser playground are mine, and the playground's scoring runs entirely client-side.* --- # Qwen-BIM: a 14B model that matches 671B DeepSeek-R1 — by knowing its domain > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/blog/qwen-bim-domain-beats-scale > date: 2026-06-09 > tags: llm, fine-tuning, domain-models, bim, paper-notes Here's the headline from [Qwen-BIM](https://arxiv.org/abs/2602.20812) (Lin et al., Tsinghua, Feb 2026): a fine-tuned **14B** model scores **0.83 on G-Eval** for BIM-based design tasks — essentially tied with **DeepSeek-R1 at 671B** (0.84), and ahead of its own 72B sibling. A model ~48× smaller, matching the frontier on a specific domain. That result is not surprising on its own — "fine-tune a small model on your domain" is folklore by now. What makes the paper worth reading is the *anatomy*: where exactly general LLMs fall over on engineering work, and which one design choice did most of the lifting. I work on industrial AI at [Inkers](https://inkers.ai), so domain models over 3D/BIM data are close to home. These are my notes. ## The actual problem: a BIM model isn't text A Building Information Model is a structured graph of components — walls, slabs, beams, each with geometry, materials, and relationships. An LLM can't read it. So step one of *any* LLM-on-BIM pipeline is an unglamorous one the field mostly skips past: **turn the model into text.** The authors do exactly that, carefully. Revit models of five building types (malls, offices, dormitories, teaching buildings, museums) are sliced into spatial blocks of ~10–15 components each, defects are injected, and each block is serialized to plain text plus 22 templated questions with **hard-coded reference answers**. That last detail matters: the ground truth is computed by rules, not by another model, so the benchmark isn't measuring one LLM against another LLM's opinion. The questions ladder up in difficulty on purpose: from "list the wall IDs" (extraction) through "compute each slab's area" (calculation) to "is this wall thickness suspicious given residential norms?" (domain reasoning). It's a clean way to see *which rung* a model falls off. The whole pipeline, end to end, is just: project the structured model into text, turn it into supervised Q&A, add reasoning traces, and LoRA-fine-tune a small open model on it. {[ { x: 8, t1: "BIM model", t2: "Revit graph" }, { x: 132, t1: "textualize", t2: "+ 22 questions" }, { x: 256, t1: "BIM-QA", t2: "2,129 pairs" }, { x: 380, t1: "BIM-QRA", t2: "1,364 + reasoning" }, { x: 504, t1: "Qwen-BIM", t2: "LoRA · 14B" }, ].map((n, i) => ( {n.t1} {n.t2} {i < 4 ? : null} ))} structured 3D → text → supervised data → small fine-tuned model ## Where general LLMs actually fail They evaluated 11 general models (ChatGLM, Qwen, DeepSeek — including the 671B DeepSeek-V3/R1). The failure modes are specific and, honestly, familiar from any engineering-LLM project: - **Arithmetic.** Asked for a slab's planar area, Qwen-max picks the *right formula* and still returns the wrong number. The bottleneck isn't understanding — it's calculation. - **Natural-language literalism.** Models misread parentheses in the answer template, or a naming rule ("wall IDs start with Q"), and confidently apply the wrong transform. - **Missing domain knowledge.** Asked to infer a building's floor height, the 14B base model reasons that floor height ≈ slab thickness (120 mm) — coherent chain of thought, wrong mental model, because it was never taught what "floor height" means in practice. The pattern: general models clear extraction and counting, then degrade sharply on calculation, multi-step reasoning, and anything needing design common sense. On the domain-specific design-review tasks, G-Eval is mostly **below 0.8** — not reliable enough to trust. ## The one choice that mattered: reasoning supervision This is the part I'd underline. They built two datasets from the same BIM text: - **BIM-QA** — 2,129 plain question→answer pairs. - **BIM-QRA** — 1,364 question→**reasoning**→answer triples, where the intermediate steps are supervised, not just the final answer. Then they LoRA-fine-tuned Qwen2.5-14B on different mixes. The result is the kind of finding that should change how you build these datasets: | Fine-tuning data | Size | G-Eval | |---|---|---| | 100% QA | 2,129 | 0.69 | | 80% QA + 20% QRA | 2,661 | 0.77 | | 60% QA + 40% QRA | 2,500 | 0.77 | | **100% QRA** | **1,364** | **0.83** | The **smallest** dataset — pure reasoning triples — won, by a wide margin. More reasoning supervision monotonically improved G-Eval, and quality beat quantity outright. Teaching the model *how to get there*, on a third of the data, beat teaching it *what the answer is* on the full set. ## Bigger is not better (and the paper shows it twice) Two clean data points against scale-maximalism: 1. On the general benchmark, **QwQ-32B out-scored DeepSeek-R1 (671B)** on G-Eval. The giant model's verbose reasoning actually *hurt* — it padded answers with redundant text, tanking format and text-similarity scores without improving correctness. 2. After fine-tuning, **Qwen-BIM (14B) matched DeepSeek-R1 (671B)** and beat the 72B and 32B Qwen models on the domain G-Eval. {/* y gridlines at 0.5..0.9 — chart area y 20..200 maps score 0.9..0.5 */} {[0.5, 0.6, 0.7, 0.8, 0.9].map((v) => { const y = 200 - ((v - 0.5) / 0.4) * 180 return ( {v.toFixed(1)} ) })} {[ { label: "base 14B", sub: "Qwen2.5", score: 0.69, hl: false }, { label: "Qwen-BIM", sub: "14B · fine-tuned", score: 0.83, hl: true }, { label: "DeepSeek-R1", sub: "671B", score: 0.84, hl: false }, ].map((b, i) => { const x = 96 + i * 150 const y = 200 - ((b.score - 0.5) / 0.4) * 180 return ( {b.score.toFixed(2)} {b.label} {b.sub} ) })} The improvement from fine-tuning is also *targeted* exactly where you'd want it: | G-Eval | Base 14B | Qwen-BIM | Δ | |---|---|---|---| | General tasks | 0.810 | 0.874 | +0.06 | | Domain-specific tasks | 0.588 | 0.801 | **+0.21** | | Overall | 0.689 | 0.834 | +0.15 | General ability barely moved (it was already fine); the entire gain is concentrated in the domain tasks that were broken. That's the signature of fine-tuning doing the right thing — adding domain competence without trading away the base model's generality. ## What I'd flag It's a careful paper, but keep the scope honest: - **2D only.** Early tests showed the models couldn't do 3D geometry (collision detection), so those questions were cut. The hard part of real BIM reasoning is 3D. - **One narrow task family**, five building types, rule-generated Q&A. G-Eval is an LLM-as-judge metric — better-correlated with humans than BLEU/ROUGE here, but still a proxy. "Data available on request" rather than released. - The "matches 671B" comparison is on *this* benchmark. It's a domain-competence claim, not a general-capability one. ## Why it's the right playbook anyway Strip away the BIM specifics and this is a template for industrial domain models, the kind I think about constantly: you rarely need a frontier model. You need (1) a faithful **text projection of your structured/3D data**, (2) a benchmark with **rule-computed ground truth** so you're measuring competence, not vibes, and (3) **reasoning-supervised** fine-tuning data — quality and chain-of-thought over raw volume. Get those three right and a 14B model on two A6000s reaches the same place a 671B model does, at a fraction of the inference cost. For anyone shipping AI into a real engineering vertical, that economics is the whole game. --- *Paper: [Developing large language model for BIM-based design with domain-specific benchmark and dataset](https://arxiv.org/abs/2602.20812) — Lin, Cai, Ni, Zhou, Pan (2026), arXiv:2602.20812.* --- # This site is managed by Claude > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/blog/hello-world > date: 2026-06-03 > tags: meta, ai, nextjs GitHub READMEs are dead. After Claude and the wave of coding agents, your homepage isn't a static profile — it's a living, agent-readable artifact you can hand to an LLM. This site is **dual-native**: every page is both a clean human document and a machine-readable surface. Try fetching [`/blog/hello-world.md`](/blog/hello-world.md) or [`/llms.txt`](/llms.txt) — an agent gets structured text, you get the rendered page. The whole site is maintained by a crew of Claude agents. New posts, logs, and data updates are authored by skills that validate themselves before shipping. ## What's under the hood The content layer is a single source of truth: MDX files validated with Zod, surfaced identically to humans (this page), to agents (the `.md` variant), and to tools (the MCP server). More on that soon. --- # ABot-Recon: streaming 3D over thousands of frames with a 12-frame memory > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/articles/abot-recon > date: 2026-11-09 > tags: 3d, slam, streaming, feed-forward-reconstruction, long-video, visual-odometry A [post on X](https://x.com/cutetoxicguy/status/2119151400470880396) pointed at ABot-Recon, a streaming 3D reconstruction model from Alibaba's AMAP CV Lab that "doesn't need to keep the entire video history inside a growing attention window". The paper is [arXiv 2608.27529](https://arxiv.org/abs/2608.27529), "Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction". The code is at [amap-cvlab/ABot-Recon](https://github.com/amap-cvlab/ABot-Recon) and the checkpoint at [acvlab/ABot-Recon](https://huggingface.co/acvlab/ABot-Recon) on Hugging Face. There is a [project page](https://amap-cvlab.github.io/ABot-Recon-html/) too. It is the same lab as [ABot-World-0](/articles/abot-world), which this site took apart in August. The claim is easy to state. The model reads each new frame once, attends to the 11 frames before it and nothing older, and still tracks the camera over 18,846-frame drives. This site has covered the other way to get there. [R³ and AMB3R-SLAM](/articles/da3-streaming-reconstruction) keep keyframe banks and pose graphs, and [HiP3R](/articles/hip3r) stitches chunks with two graphs after the fact. ABot-Recon's answer is that most of that state is unnecessary if you change what the network is asked to predict. What I did, and how each claim below is labelled: - **Reported**: a number from the paper or the README that I did not re-run. - **Read**: something I saw in the released source at commit `6e3b2e1` (15 October 2026) or in the checkpoint's safetensors header, which I fetched with an HTTP range request. I did not execute the authors' code: the sandbox refused to import the repository's modules, even its torch-only cache and mask files. - **Measured**: output of my own scripts, run on this box's CPU with 2 threads. [`toy.py`](https://ai.satyajit.si/articles/abot-recon/toy.py) (torch 2.14.1) covers the cache, the rotary positions and the pose chain; [`composition.py`](https://ai.satyajit.si/articles/abot-recon/composition.py) the composition-aware loss; and [`loop_closure_toy.py`](https://ai.satyajit.si/articles/abot-recon/loop_closure_toy.py) the loop-closure back end. The last two are numpy 2.5.4 only and share [`lie.py`](https://ai.satyajit.si/articles/abot-recon/lie.py). Each re-implements a piece of the mechanism from what I read. None of them is the model, and none ever sees an image. - **Reasoned**: arithmetic on the other three. ## The problem: long video breaks streaming models in two ways Feed-forward geometry models (DUSt3R, VGGT, π³) predict cameras and point maps for a set of images in one pass. To run one on a video you make it causal: frame $i$ attends to frames before it, and you keep their keys and values in a cache so they are not recomputed. That is StreamVGGT's design. It has two costs that grow with the stream. The first is memory. Every frame's keys and values stay in the cache, so a video twice as long needs a cache twice as big and attention twice as wide. The second is the prediction target. If every pose is expressed in the first frame's coordinates, then the network at frame 4,000 is asked to say where it is relative to a view it saw minutes ago. The paper's framing is that the target's "estimation range grows with the sequence" (**reported**). A model trained on 32-frame clips has never seen that range. The field's recent answers add memory back in a managed form. LingBot-Map keeps anchor frames, a pose-reference window and a trajectory memory; HorizonStream couples local attention with persistent linear attention; CUT3R and TTT3R carry a recurrent state (**reported**, the paper's related work). ABot-Recon removes the long-range state instead. ## The idea: predict only local things, then chain them For each incoming frame $I_i$ the model outputs three things (**reported**, Section 3.1): 1. a point map $P_i$ in the **current** camera's coordinates, 2. a confidence map $S_i$, 3. one relative pose $T_{i-1\leftarrow i}$ from the current camera to the previous one. None of those depends on how long the stream has been running. A point map in the current camera is the same task at frame 10 and at frame 10,000. So is the motion between two adjacent frames. The global trajectory is never predicted; it is computed, by multiplying the relative poses together: $$ T_{0\leftarrow j} = T_{0\leftarrow 1}\,T_{1\leftarrow 2}\cdots T_{j-1\leftarrow j} $$ and each local point map is moved into the world with its frame's composed pose. That is ordinary visual odometry. The new part is doing it with a learned feed-forward model whose own context is fixed, and training it so the chain does not fall apart.
## The network, with shapes The backbone is π³'s. The released network is built on its code (`abot_recon/modeling/pi3/`), and training starts from π³'s public weights (**read** and **reported**). Here is one frame's path through it at the paper's 504 × 280 input, with every shape taken from the source (**read**, `pi3.py`, `network.py`, `adjacent_pose_head.py`, commit `6e3b2e1`). 1. **Encoder.** DINOv2 ViT-L/14 with registers (`dinov2_vitl14_reg`). 14-pixel patches make 36 × 20 = 720 patch tokens of width 1,024. The encoder sees one frame at a time, so it has no memory at all. 2. **Decoder.** Five learned register tokens are prepended, making 725 tokens a frame. Then 36 blocks of width 1,024 with 16 heads of 64. Even blocks attend within the frame. Odd blocks attend across frames, through the 12-frame window below. Only those 18 odd blocks keep a cache. 3. **Fork.** The outputs of the last two decoder blocks are concatenated, 2,048 wide, and handed to three small decoders: points, confidence and camera. 4. **Geometry head.** A linear head per 14-pixel patch unfolds each token into a 14 × 14 patch of three numbers, $(x, y, \log z)$. The point is $(x z,\; y z,\; z)$ in the current camera, so depth is predicted in log space and the ray direction is scaled by it. That is $P_i \in \mathbb{R}^{280 \times 504 \times 3}$, and the confidence head gives $S_i \in \mathbb{R}^{280 \times 504}$ the same way. 5. **Camera head.** The five register tokens double as camera tokens; the camera decoder leaves them 512 wide. Each goes through a two-layer MLP and the five are averaged: that is $z_i$ in the paper's Equation 2. The pair descriptor of Equation 3 is $$ q_i = \mathcal{R}(z_{i-1}, z_i) = [\,z_{i-1},\; z_i,\; z_i - z_{i-1},\; z_i \odot z_{i-1}\,] \in \mathbb{R}^{2048} $$ The four blocks let a two-layer MLP see both frames, their difference (what moved) and their product (what agrees) without having to learn subtraction. That MLP returns 512 numbers, and two linear layers turn them into a translation $t \in \mathbb{R}^3$ and a scalar-last quaternion, normalised into a rotation $\tilde R$. The quaternion format is enforced: the head raises an error for any other (**read**). The relative pose for frame $i$ is then $$ T_{i-1\leftarrow i} = \begin{bmatrix} \tilde R_{i-1\leftarrow i}\,\mathrm{Exp}(\delta\omega_i) & t_{i-1\leftarrow i} \\ 0 & 1 \end{bmatrix} $$ where $\delta\omega_i \in \mathbb{R}^3$ is the rotation refiner's axis-angle correction (Equation 11), applied on the right, inside the step, before anything is chained. In the code the running pose is one matrix product per frame: `current_pose = previous_pose @ corrected_delta` (**read**). The only state the camera head carries from frame to frame is the previous descriptor, the previous frame's tokens, that running pose and the refiner's buffer of 10 pair features. All of it is fixed-size. ## How memory stays bounded: a 12-frame band, enforced in code The post's heading is "Only 12 frames of local context". That is literal. `InferenceConfig` raises `ValueError("The released model requires a 12-frame local window")` for any other value, and the attention mask in `core/window_mask.py` blocks key frame $T_k$ for query frame $T_q$ when $T_k > T_q$ or when $T_q - T_k$ is more than 11 (**read**). The cache that feeds it, `StreamingKVState.append_committed_frame`, keeps at most 11 past frames: once full, it shifts the buffer left and writes the new frame into the last slot. The comment above that shift records a bug they hit: an overlapping in-place `copy_` "silently corrupts older window entries on CUDA" (**read**). Two things in the code are more interesting than the band itself. **The hooks for long-range memory are there, and switched off.** The cache classes take `num_reference_frames` (frames pinned forever, like LingBot-Map's anchors) and `num_summary_tokens` (a few tokens of every evicted frame, kept as a compressed history). The released constructor passes neither, so both default to 0 (**read**, `model.py` and `network.py`). The paper's claim of "no persistent learned long-range state" is a configuration of a codebase that could carry one. My guess is that the authors tried the anchors and summaries and found them unnecessary; the paper does not report that experiment (**reasoned**). **Positions are absolute, attention is not.** The rotary embedding on the frame axis uses each frame's absolute index, "kept for 3D rotary positions even after older frames leave the window" (**read**, `kv_state.py`). Rotary embeddings make a query-key score depend only on the difference of positions, so frame 20,000 attending to frame 19,995 should see exactly what frame 11 sees attending to frame 6. The angle table is built in float64 and has `max_seq_len` rows, set to `max_frames`, 22,000 by default (**read**). I rebuilt the frame-axis rotation (20 of the 64 head dimensions, θ = 10,000) and compared scores at frame 11 and frame 20,000 for the same offsets (**measured**): in fp32 the largest difference over 725 × 725 scores was 7.6e-6, against a mean score magnitude of 3.5; in bf16 it was 0.15, the same size as bf16's own error at frame 11 (0.11 to 0.15 against a float64 reference). The window's attention loses nothing to a late position. The one practical consequence is a cap: a stream longer than `--max-frames` would index past the table, so a 30,000-frame drive needs the flag raised (**reasoned**; I did not run past it). ## How big the cache is, and what 6.71 GiB buys The paper reports 24.45 FPS and 6.71 of peak GPU memory on KITTI-02 at 504 × 280 on an H100, "input storage excluded" (**reported**, Table 6). The table does not give the unit and the text says "6.71 GB". PyTorch's peak-memory counters are usually divided by $2^{30}$ and printed as "GB", so I read it as GiB below; if it is decimal gigabytes it is 6.25 GiB, and the unexplained remainder further down shrinks from 1.6 to 1.1 GiB (**reasoned**). I wanted to know how much of it is the window. At 504 × 280 with 14-pixel patches a frame is 36 × 20 = 720 patch tokens plus 5 register tokens, 725 in all. Each of the 18 cross-frame layers stores a key and a value for each, 16 heads of 64, in bf16 (**read**: the paged path casts to bf16 because FlashInfer will not take fp32). That is 18 × 2 × 725 × 1,024 × 2 bytes = 53,452,800 bytes, 50.98 MiB per frame (**reasoned**). So the 11-frame window holds 0.548 GiB. The paged allocator reserves 28 frame pages (the window of 12 plus 16 spare), 1.394 GiB, once, at the start (**read** and **reasoned**). The checkpoint is 1,000,934,176 parameters, all float32 in the file, 4.0 GB, which I read from its header without downloading it; nothing in the loader casts them, and inference runs under bf16 autocast (**read**). Weights at 3.73 GiB plus the 1.394 GiB pool is 5.12 GiB; the remaining 1.6 GiB of the reported peak is activations and FlashInfer's 128 MiB workspace (**reasoned**, not profiled). The window is about a fifth of the memory. Had the cache kept everything, KITTI-02's 4,661 frames would need 232 GiB of keys and values, and VBR's 18,846-frame Ciampino-1 drive 938 GiB (**reasoned**). A plain full-history cache is not an option at these lengths on any single GPU. My toy shows the shape on a CPU (**measured**, one attention head, 64 tokens of width 64 per frame). With a 12-frame window, a frame costs 0.31 ms at frame 12 and 0.18 ms at frame 3,000. With a cache that keeps everything it costs 1.3 ms at frame 100, 21 ms at frame 1,000 and 113 ms at frame 3,000. The toy says nothing about H100 throughput; it says the window's per-frame cost is flat and the full cache's is not. ### What is not bounded The post is right that "storing the resulting point cloud still requires space proportional to the amount of reconstructed geometry". The code makes that concrete. `inference_stream_iter` appends each frame's outputs to a Python list and concatenates them at the end, with no move to the CPU inside the loop (**read**). A local point map at 504 × 280 is 1,693,440 bytes in fp32 and its confidence map 564,480 bytes, so dense outputs for all of KITTI-02 come to about 10.5 GB (**reasoned**). The README's `--dense-stride N` and the API's `dense_output_indices` are the escape hatch: poses for every frame, dense geometry for a subset. Use them on long runs. The optional loop closure is not bounded either, by design. It retrieves revisited frames with DINOv2-SALAD descriptors and FAISS, then re-runs ABot-Recon on windows centred on both frames to get a relative pose, so it needs the old images and a descriptor per keyframe (**read**, `loop_closure.py`, and **reported**, Section 4.1). ## The price of a short memory: drift, and why rotation gets the fix A chain of relative poses drifts. Every small error stays in the product forever. The paper's two additions are aimed at exactly this, and the toy below shows why they aim at rotation. I composed 4,661 noisy relative poses along a synthetic drive the size of KITTI-02 (1.094 m a step, 12 random 90-degree turns), aligned the result to ground truth with a Sim(3) fit as the paper does, and averaged eight seeds (**measured**). Translation noise of 5% of every step gives an ATE of 2.03 m. A heading error of 0.05° per step, with no translation noise at all, gives 13.3 m; at 0.1° it is 26.57 m (**measured**). A wrong step length is one wrong step. A wrong heading turns every step after it, so the error grows with distance travelled. Over a gap of 11 frames the 0.05° chain is off by 0.534 m; over 1,000 frames by 26.6 m (**measured**). The toy's noise levels are mine, chosen to show the shape, not estimates of the model's errors. The paper's two fixes are the same diagnosis in network form.
### The rotation refiner, read line by line The paper describes a module that takes the pairwise camera descriptor, adds visual evidence by cross-attending from a pooled frame-pair query into both frames' dense tokens, then runs a gated temporal convolution over recent pair features to predict a small axis-angle correction to the relative rotation. Translation is left as predicted (**reported**, Section 3.3). In symbols, with $f_i$ the fused pair feature: $$ \delta\omega_i = \phi_o\big(T_h(W_i) \odot \sigma(T_g(W_i))\big), \qquad W_i = [f_{i-K+1}, \dots, f_i] $$ $T_h$ proposes a hidden vector from the window, $T_g$ proposes a gate in $(0, 1)$ for each channel, and $\phi_o$ maps the gated result to three numbers. The source is more specific than the notation, in three ways (**read**, `TemporalRotationRefiner` in `adjacent_pose_head.py` and `RELATIVE_CAMERA_HEAD` in `model.py`). - **The window is 10, not 12.** The feature buffer keeps the last 10 pair features, padded with zeros at the start of a stream, and a learned `age_embed` is added to each slot so the network knows how old a feature is. - **The "TCN" is one depthwise convolution.** `conv` and `gate` are `Conv1d` layers with `groups=512` and kernel 10, run over a 10-long buffer with no padding. Each produces exactly one output step. So $T_h(W_i)$ is, per channel, a learned weighted sum of that channel's last 10 values; $T_g$ is another, through a sigmoid. It is a gated linear filter over time, one per channel, not a stack of dilated layers. - **The correction is clamped.** $\phi_o$ is a single linear layer to three numbers, then `max_rad * tanh(...)` with `max_rad` = 2.0°. Each axis-angle component is at most 2.0 degrees, so the refiner can trim a step but never rewrite one. I wanted to know where the 4.75 million parameters sit, so I rebuilt the module's layer list from the source with hidden width 512, a 512-wide camera token and 8 attention heads, and counted (**reasoned** from the code, then checked against the checkpoint): | Piece | What it does | Parameters | |---|---|---| | `desc_proj` | LayerNorm(2,048) and a two-layer MLP on the motion descriptor $q_i$ | 1,315,840 | | `frame_query_proj` | the same shape, on the pooled frame pair $\mathcal{R}(\bar G_{i-1}, \bar G_i)$ | 1,315,840 | | `frame_proj` + role embedding | projects both frames' tokens into the attention's key space, tags which frame | 263,680 | | `frame_attn` + `frame_norm` | one 8-head cross-attention from the pair query into both frames' 1,440 patch tokens | 1,051,648 | | `fuse_proj` | LayerNorm(1,024) and a two-layer MLP: motion plus visual evidence into $f_i$ | 789,504 | | `age_embed` | one 512-vector for each of the 10 slots | 5,120 | | `conv` + `gate` | the two depthwise temporal filters | 11,264 | | `out` | linear to the three-number residual | 1,539 | | **total** | | **4,754,435** | The total matches the checkpoint's 4,754,435 to the parameter (**read**, safetensors header), so this is the released module and not a near relative. The part the paper names, the temporal convolution, is 11,264 parameters, 0.24% of the refiner. Almost all of the refiner is evidence-building: two pair MLPs and one cross-attention, so that each step gets a compact feature that already says how much the camera turned and what the pixels say about it. The temporal part is small because its job is small: weigh the last 10 of those features. Its state is a 10 × 512 buffer, so it does not break the memory bound. One thing the refiner never sees is its own past output. The buffer holds fused features, not past corrections or past composed rotations. It can learn that a run of features looks like a turn, but not that the last five corrections all leaned left (**read** and **reasoned**). ### The composition-aware loss, term by term Training does not only supervise each adjacent pair. For a clip of $N$ frames the supervised set is every pair at most $K-1 = 11$ frames apart (**reported**, Equations 12 to 14): $$ \mathcal{P} = \{(i, j) \mid 0 \le i < j < N,\; j - i \le K-1\} $$ For each pair the predicted adjacent poses between $i$ and $j$ are multiplied out (Equation 5) and compared with the true $T_{i\leftarrow j}$. Rotation is weighted by the gap: $$ \alpha_{ij} = \frac{(j-i)^{\gamma}}{\frac{1}{|\mathcal{P}|}\sum_{(m,n)\in\mathcal{P}} (n-m)^{\gamma}}, \quad 0 < \gamma < 1, \qquad \mathcal{L}_{pose} = \frac{1}{|\mathcal{P}|}\sum_{(i,j)\in\mathcal{P}} \big(\lambda_{trans}\,\ell^{(i,j)}_{trans} + \alpha_{ij}\,\lambda_{rot}\,\ell^{(i,j)}_{rot}\big) $$ Reading it term by term. $\ell_{trans}$ and $\ell_{rot}$ are π³'s translation and rotation losses, applied to the composed transform. The denominator of $\alpha_{ij}$ makes the weights average to 1, so $\gamma$ moves weight between gaps without changing the loss's scale. $\gamma < 1$ makes the weight grow slower than the gap. Translation is not weighted at all. A separate term, $\mathcal{L}_{smooth}$, penalises the size and the frame-to-frame change of the refiner's residuals (**reported**, Equation 15). The paper does not give $\gamma$, the $\lambda$s, or the exact forms of $\ell$; the `train` branch has the training code and I did not read it. Two consequences fall out of the definition (**reasoned**, checked in the second tab of the widget below). First, most of the rotation weight is on long gaps: at $\gamma = 0.5$ in a 128-frame clip, adjacent pairs are 9.5% of the pairs but carry 4.1% of the rotation weight. Second, and this corrects my first version of this article, **a longer clip does not contain longer chains to grade.** $\mathcal{P}$ caps the gap at 11 whatever $N$ is. A 128-frame clip has 1,342 supervised pairs and a 32-frame clip 286, with nearly the same mix of gaps. What 128 frames add is more pairs per clip and fewer clip boundaries, which is the paper's own explanation of its ablation. ### Experiment: what the composition loss can see The point of grading chains is that some errors only show up in a chain. An adjacent-pair loss scores each step on its own. If two predictors make the same size of error per step, it cannot tell them apart, even when one predictor's errors all lean the same way and the other's cancel. Their trajectories are very different. I tested exactly that (**measured**, `composition.py`). Each step's rotation error is an AR(1) process, $e_i = \rho\, e_{i-1} + \sqrt{1-\rho^2}\,\eta_i$, scaled so the per-step spread is 0.05° per axis for every $\rho$. Only the step-to-step correlation changes. For each $\rho$ I computed the adjacent-pair rotation loss (geodesic angle), the rotation term of $\mathcal{L}_{pose}$ above on random 128-frame clips with $K = 12$ and $\gamma = 0.5$, and the Sim(3)-aligned ATE of the whole 4,661-frame drive, over six seeds. | ρ | adjacent loss | composed loss | ATE | |---|---|---|---| | −0.5 | 0.080° | 0.130° | 7.06 m | | 0 | 0.080° | 0.203° | 12.25 m | | 0.5 | 0.080° | 0.313° | 21.32 m | | 0.9 | 0.080° | 0.479° | 54.37 m | | 0.98 | 0.078° | 0.523° | 120.65 m | The adjacent loss is flat at 0.08° across a 17-fold range of ATE. A model trained only on it has no gradient toward the better trajectory. The composed loss tracks the ATE from $\rho = -0.5$ to $\rho = 0.9$, so a model trained on it is pushed toward errors that cancel. That is the paper's ablation in miniature: the composition loss alone took KITTI's ATE from 56.60 m to 27.66 m. The last row is the catch. From $\rho = 0.9$ to $\rho = 0.98$ the ATE more than doubles while the composed loss moves from 0.479° to 0.523°. An AR(1) error with $\rho = 0.98$ stays correlated for about 50 frames, and the loss looks at most 11 frames back. Error that leans the same way for longer than the window, such as a slow calibration bias or a long gentle curve the model consistently under-turns, is invisible to the loss that is supposed to catch drift. Nothing in the 12-frame model sees it either; that is what loop closure is for (**reasoned**). Negative correlation is the other lesson. At $\rho = -0.5$ the ATE is 7.06 m against 12.25 m for independent errors at the same per-step size. A refiner that looks at the last 10 pair features can learn to lean against a recent run of errors, and the composed loss rewards that. Whether the trained refiner does so I cannot say without running it. ### What the ablation says The ablation in the third tab of the drift widget says the loss did the most work: on KITTI, adding it alone took average ATE from 56.60 m to 27.66 m, training on 128-frame clips instead of 32 took it to 22.31 m, and the refiner to 18.25 m (**reported**, Table 8). Every row keeps the 12-frame window. Clip length during training mattered even though neither the window nor the longest supervised chain changed. The paper's reading is more data per clip and fewer artificial clip boundaries; I agree, with one caveat: the paper does not say whether the ablation rows were trained for the same number of steps, and its main recipe's 128-frame stage is a second stage started from the first, so the row may mix clip length with extra training. The paper's own diagnostic shows the result: adjacent-frame error that stays flat across a 4,661-frame sequence, and multi-gap error that grows more slowly than the other streaming models'.
## The results, read against their own tables All of this section is **reported** (Tables 3 to 7) unless marked. **KITTI** (11 sequences, 271 to 4,661 frames): average ATE 18.25 m without loop closure, the lowest in the table; the next streaming model is HorizonStream at 22.51 m, and the best optimisation-based entry is SLAMFormer-∞ at 23.01 m, quoted from its own paper. With loop closure, 13.49 m against HorizonStream's 17.98 m. Per sequence it is not a sweep: on 02, the longest, 69.87 m against HorizonStream's 97.24 m, but on 06 it is 12.04 m against HorizonStream's 5.46 m. **Oxford Spires** (10 sequences of about 3,840 frames): ATE 4.35 m and RPE-R 0.12°, the abstract's "approximately 40%" better than the best prior results. That is against streaming methods: LingBot-Map's 7.32 m and HorizonStream's 0.20° (**reasoned**: 4.35 / 7.32 is 40.6% lower, 0.12 / 0.20 is 40% lower). It is not the best number in Table 4. DPVO reaches 1.56 m, Droid-W 2.27 m and DPV-SLAM 2.95 m, and Droid-W's RPE-R is 0.08°. The paper says so and points out that those three need camera intrinsics while ABot-Recon reads only RGB. That is a fair distinction, and it is also the honest headline: on this benchmark a classical-style optimiser that knows its camera still wins. **VBR** (7 sequences, 8,815 to 18,846 frames): without loop closure, 30.14 m, behind HorizonStream (29.02 m) and LingBot-Map (29.45 m). The paper calls that "competitive", which is accurate. With loop closure it drops to 9.99 m, the lowest in the table, against HorizonStream's 17.64 m with its own loop closure. **Dense reconstruction**: on Oxford Spires, Chamfer distance 1.37 m and F1 91.81% at a 4 m threshold, the best of the nine streaming methods (LingBot-Map 1.68 m and 90.58%). On 7Scenes, F1 94.88% against HorizonStream's 98.22% and LingBot-Map's 96.01%; on TUM-Dynamic, 92.19% against LingBot-Map's 95.97%. The paper's discussion concedes the indoor gap and suggests compact scenes, with lots of revisiting, are where persistent memory still pays. **Speed**: 24.45 FPS and 6.71 GiB, frame by frame on one H100. CUT3R (29.62 FPS, 3.16 GiB) and TTT3R (25.53 FPS, 4.65 GiB) are faster and smaller, and drift far more: CUT3R's KITTI ATE is 209.04 m. LingBot-Map takes 18.87 GiB and HorizonStream 13.04 GiB in the same mode. Two protocol notes. Every trajectory is aligned to ground truth with a Sim(3) Umeyama fit before ATE, so scale is free; a monocular model's metric-scale error is not in these numbers. And the training mixture includes KITTI-360 and Waymo (**reported**, Table 1), which are driving data close to KITTI's domain but not its test sequences. ## Loop closure, read and re-implemented Everything above is the streaming model alone. The best numbers in the paper are not. On VBR, loop closure takes ATE from 30.14 m to 9.99 m, and on KITTI from 18.25 m to 13.49 m (**reported**, Tables 3 and 5). It is a separate, training-free back end, and its code is worth reading, because the bounded-memory story stops at its door.
Where it helps on KITTI is a list of the sequences that revisit places (**reported**, Table 3): 00 goes from 21.86 m to 11.70 m, 02 from 69.87 m to 39.00 m, 05 from 13.54 m to 6.00 m and 06 from 12.04 m to 8.70 m. Sequences 01, 03, 04, 08 and 10 do not change at all. 08 is the interesting zero: it does revisit streets, but in the opposite direction, and a forward-facing camera sees a different picture going the other way, so appearance-based retrieval is unlikely to match them (**reasoned**; the paper does not discuss it). ### What the code does The pipeline in `loop_closure.py` and `sparse_loop/` runs **after** the stream, on the whole trajectory (**read**, defaults in `LoopClosureConfig`): 1. **Describe every frame.** DINOv2-SALAD descriptors at 336 × 336, one per frame (`keyframe_stride` 1). In the demo they are computed by a worker thread while the stream runs. 2. **Retrieve.** FAISS inner-product search, top 5 neighbours per frame. A pair survives if its score exceeds 0.85 and the two frames are more than 30 frames apart. 3. **Suppress.** Candidates are ranked by score and kept greedily; each kept pair blocks every other candidate within 25 frames of either end. At most 50 survive to the next step. 4. **Measure.** For each pair, the 10 frames centred on each end are put in one list, sorted by index, and run through ABot-Recon as a single 20-frame stream. The relative pose between the two centre frames is read off that stream's chained poses. 5. **Optimise.** A sparse SE(3) pose graph: a node every 50 frames plus every loop endpoint, an odometry edge between consecutive nodes from the streamed trajectory, one edge per loop. Levenberg-Marquardt for 30 iterations, first node fixed, on CUDA only (the solver raises without a GPU). 6. **Spread.** Each node's correction is interpolated to every frame between nodes: rotations by slerp, translations linearly. Four details matter more than they look. **The loop measurement asks the model to do something it was not trained for.** In step 4, with $i$ the old frame and $j$ the new one, frame $i+4$ is followed directly by frame $j-5$. To the model that is one adjacent step, and the relative pose between the two visits rests on that one prediction across a jump of thousands of frames. It works when the two visits look alike, which is what retrieval selected for, and the 30-frame and 25-frame rules keep it from being asked about the same place twice in a row (**read** and **reasoned**). **Every loop edge gets the same, small weight.** The weight is `pose_graph_loop_weight * sqrt(inliers / rigid_min_inliers)`, 0.01 × √(128 / 24) = 0.0231. But `inliers` is never counted: every edge is created with the config's `default_inliers`, 128. There is no per-edge verification, and the graph has no robust kernel. Odometry edges get weight 1, and since the residual is multiplied by the weight before it is squared, one loop edge counts for about 1/1,875 of an odometry edge (**read** and **reasoned**). **SE(3), not Sim(3).** The graph's `pose_graph_model` defaults to `se3` and the loop config does not change it. A monocular chain can drift in scale as well as in pose; this back end can bend the trajectory but cannot rescale part of it (**read**). **Rotation and translation share a scale.** Both residual weights are 1.0, so one radian of rotation error costs the same as one metre of translation error. ### Experiment: the released back end on a drive that comes home I re-implemented steps 3, 5 and 6 with the released defaults: the NMS rule, the keyframe stride, the edge weights, the residual $\mathrm{Log}(Z\,\mathrm{Exp}(x_s)\,\mathrm{Exp}(x_d)^{-1})$ on world-to-camera log coordinates, the Levenberg-Marquardt schedule (λ from 1e-6, halved on success, doubled on failure) and the slerp spreading (**measured**, `loop_closure_toy.py`). Steps 1, 2 and 4 need the model and SALAD, so I stood in for them: a candidate is any frame within 4 m of an earlier frame at least 31 frames back, scored by distance, and each loop measurement is the true relative pose with 0.05° and 0.05 m of noise. That makes my loop edges better than the real ones can be. The drive is 4,661 frames, the length of KITTI-02: a 1,300 m × 900 m loop, then 640 frames back over its own start, like KITTI-00. The streamed trajectory has 0.05° of rotation noise and 2% of translation noise per step. The retrieval rule finds 642 candidate frames on the revisit; NMS keeps 20, below the cap of 50. Four seeds. | Setting | Loop edges | ATE | |---|---|---| | no loop closure | 0 | 15.68 m | | released weight | 20 | 5.10 m | | weight × 10 | 20 | 4.00 m | | weight × 100 | 20 | 3.74 m | | released weight | 1 | 7.52 m | | released weight | 5 | 5.57 m | | weight × 100 | 1 | 4.19 m | | released weight, plus one false loop | 20 + 1 | 58.65 m | | weight × 100, plus one false loop | 20 + 1 | 313.87 m | Three readings (**measured**, then **reasoned**). **It works, and it is cautious.** The released setting cuts this drive's ATE by two thirds, 15.68 m to 5.10 m. Raising the loop weight a hundredfold gets 3.74 m. With good edges the low weight leaves about 1.4 m on the table. **One edge does most of it.** A single loop edge at the released weight gets 7.52 m; five get 5.57 m; twenty get 5.10 m. A revisit's edges all say much the same thing, which is why NMS can afford to keep few of them. **One false loop costs more than all the true ones save.** I added a single wrong edge, frames 2,300 and 900, which are 920 m apart, measured as "the same place": two intersections that look alike. At the released weight the ATE goes from 5.10 m to 58.65 m, nearly four times worse than not closing loops at all. At a hundred times the weight it is 313.87 m. The small weight is the only defence the graph has, and it limits the damage rather than preventing it. In the released pipeline, everything rides on the 0.85 SALAD threshold and the model's re-inferred pose being right. The paper's KITTI and VBR rows show no such failure, so retrieval held on those sequences. For a fleet of drives through repetitive streets, I would add a check before trusting an edge: the re-inferred relative pose against the odometry's own estimate, or the point maps' overlap, which the code already has (**reasoned**). **And the memory bound does not cover it.** Step 4 reads the old images back from disk, and steps 1 and 2 keep one descriptor per frame. That is optional and offline, and it is linear in the length of the drive (**read**). ## Training cost, licence and running it The recipe, all **reported** (Section 4.1 and Tables 1 and 2): | Stage | Clips | Iterations | Hardware | What trains | Peak learning rate | |---|---|---|---|---|---| | I | 32 frames | 32K, batch 48 | 48 NVIDIA H20 | everything but the confidence branch; the absolute camera output is replaced by the adjacent-pose head | 1e-4, annealed to 1e-6 | | II | 128 frames | 38K, batch 32 | 32 AMD MI308 | everything, refiner switched on | 5e-5 for the refiner, 2e-5 for the rest | | III | as II | 4K | not stated | confidence branch only | 2e-5 | AdamW throughout, an EMA of the weights with decay 0.999, bf16 mixed precision, gradient clipping at 1.0, one sequence per GPU. Images are resized to width 504 and centre-cropped or padded to 504 × 280. The data mixture is 30 datasets, 62.05% synthetic by sampling weight (TartanAir and TartanAir-v2, OmniWorld-Game, VirtualKITTI2, HyperSim and others) and 37.95% real (DL3DV, Waymo, ScanNet++, KITTI-360 and others, plus unnamed internal data). Unordered multi-view sets are turned into pseudo-videos by walking a camera pose graph. The appendix also shows rejected samples: sideways and upside-down BlendedMVS images and broken TartanAir and OmniWorld scenes, which is a useful reminder that a large mixture needs cleaning. Fine-tuning code is on the repository's `train` branch (**read**, README). The split learning rate in Stage II is the detail I would copy. The refiner starts from scratch, with its output layer initialised to zero, so its first correction is exactly nothing (**read**, `reset_parameters`), on top of a backbone that already predicts good steps. A lower rate for the backbone stops the new module's early, noisy gradients from undoing Stage I (**reasoned**). The licences need care. The source code is Apache-2.0. The weights are not: `MODEL_LICENSE.md` puts them under CC BY-NC 4.0, non-commercial research and education only, because they are derived from π³'s weights, and says commercial use "requires separate written authorization from all relevant rights holders" (**read**). The Hugging Face model card's metadata says `license: apache-2.0`, which contradicts the repository's own file; I would go by the more specific statement and treat the weights as non-commercial. The released setup targets Linux, Python 3.10 or later, PyTorch 2.5.1 and CUDA 12.1, with FlashInfer for the paged cache and an SDPA fallback without it (**read**). The network has a CPU branch, but it would load 4.0 GB of fp32 weights and run a 1B-parameter model per frame, so I did not try. There is a hosted demo on Hugging Face Spaces and ModelScope, and a community port to the Axera AX650N NPU (**read**, README). ```bash pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121 pip install -e . # inside a clone of amap-cvlab/ABot-Recon pip install flashinfer-python # optional: paged KV cache python demo.py --image-dir frames/ --output-dir out/ --no-loop-closure --dense-stride 10 ``` ## Where it sits The streaming 3D reconstruction literature has spent two years adding memory back to feed-forward models. CUT3R and TTT3R carry a recurrent state; StreamVGGT keeps the full causal cache; LingBot-Map keeps anchors, a pose-reference window and a trajectory memory; HorizonStream pairs local attention with persistent linear attention (**reported**, the paper's related work). On this site, [R³ and AMB3R-SLAM](/articles/da3-streaming-reconstruction) keep keyframe banks and optimise pose graphs online, [HiP3R](/articles/hip3r) reconstructs chunks and stitches them with two graphs, and [StructureGS-SLAM](/articles/structuregs-slam) closes drift against walls rather than loops. ABot-Recon is the opposite bet, and it is the classical one. Visual odometry has always predicted adjacent motion and chained it; DPVO and DROID-SLAM do it with a learned matcher and a bundle adjustment over a sliding window, and add a pose graph for loops. ABot-Recon replaces the matcher and the window optimiser with one feed-forward network that never optimises anything at inference, and keeps the pose graph as an optional extra. That is why its comparisons split the way they do: against the memory-heavy streaming models it wins on long drives, and against the classical optimisers with known intrinsics (DPVO, DPV-SLAM, Droid-W) it loses on Oxford Spires (**reasoned** from Tables 3 to 5). The broader overviews are in the site's [3D reconstruction roundup](/articles/3d-reconstruction-roundup). ## What I did not check - I did not run the model. The sandbox refused to execute the repository's code, and the model is 4.0 GB of fp32 weights with a CUDA-first runtime. Every benchmark number above is the paper's. - The 50.98 MiB per frame and the 1.6 GiB of activations are arithmetic from tensor shapes and the reported peak, not a profile. - My first drift toy uses Gaussian noise that is independent from step to step. `composition.py` adds correlation and shows how much it matters (12.25 m to 54.37 m at $\rho = 0.9$), but the correlation structure of the real model's errors is unknown to me. I did not measure it. - `composition.py` evaluates the loss; it does not train anything. Whether a network trained on the composed loss actually learns anti-correlated errors is the paper's Table 8 claim, not something I reproduced. I also had to pick $\gamma$ (0.5); the paper does not publish it. - `loop_closure_toy.py` uses true positions in place of SALAD retrieval and near-true loop measurements in place of the model's re-inference, and its drift has no scale component, so the SE(3) graph's inability to fix scale never shows. Its numbers are an upper bound on what the released back end can do with good edges, and a demonstration of what one bad edge costs. My solver uses numerical Jacobians and a dense solve where the released one uses PyPose Jacobians and a preconditioned conjugate gradient on CUDA; on a graph of this size both should reach the same minimum, but I did not run theirs. - I did not read the `train` branch, so the training-loss details (γ, the λs, the exact π³ losses) are the paper's description only. - I did not verify that the released checkpoint is the one behind Tables 3 to 8. The checkpoint's parameter counts match the paper's (1B in total, 4.75M in the refiner). ## Verdict ABot-Recon is a clean result: a feed-forward model with a fixed 12-frame view, no anchors, no compressed history and no recurrent state, which beats the long-memory streaming models on KITTI and Oxford Spires and ties them on VBR. The memory bound is real and enforced in code, at about half a gibibyte of window on top of the weights. The insight that makes it work is about targets, not caches: ask only for things that look the same at frame 10 and frame 10,000, then spend the training budget on making their product hold together, mostly the rotation. My CPU re-implementations agree with that reading and add two limits to it: the composition loss sees correlated error only within its 11-frame horizon, and the loop-closure back end that produces the best headline numbers trusts every retrieved loop equally, with no verification, so one wrong match undoes all the right ones. What it does not do is beat geometry that knows its camera, or keep the indoor reconstruction edge that persistent memory buys. For a long drive with a GPU and no intrinsics, and a non-commercial use, it is the one I would try first. For anything you ship, the weights' licence decides before the benchmarks do. --- # Astryx: what Meta's design system tells an agent, and how its search ranks > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/articles/astryx > date: 2026-11-09 > tags: agents, mcp, developer-tools, generative-ui, agentic-coding Two posts on 8 November said Meta had open-sourced Astryx, "a design system built for people and the AI agents working with them". One listed React 19 and StyleX, docs, Storybook and a sandbox, an MIT licence and a beta label. The other, in Chinese, added the bigger numbers: eight years of internal use, more than 13,000 apps, 150-plus components, and a CLI and an MCP server through which an agent can read component docs and templates. The news part is stale. Astryx's own blog dates the launch to 18 June 2026, and a "one month" post from July reports 10.3K GitHub stars by then. What is still worth an article is the part the posts gesture at and nobody shows: what does a design system "for agents" actually hand an agent, and is the answer any good? So I cloned the repository at its newest commit, counted what it ships, read the agent surfaces, and queried the hosted MCP server directly. On a first pass the sandbox's permission layer refused to execute the CLI's npm package, and this article said so. With that permission granted, on 10 November I installed the published CLI, Core and the neutral theme at 0.7.11 into a scratch project and ran it: the same 24 searches, every component read at every detail level, `init` both ways, and Core pinned to an older release. Those results are in [the CLI section](#the-cli-run-against-the-same-questions) and are marked measured. ## What is in the repository The repo is a pnpm monorepo with three top-level directories: `packages/` (what is published), `apps/` (the docsite, Storybook, a sandbox, a template viewer and six example apps for Next.js and Vite, with and without Tailwind), and `internal/` (test utilities, an ESLint plugin, and the "vibe tests" I come back to). `LICENSE` is MIT, copyright 2026 Meta Platforms. The root `package.json` declares the same. The published surface is four kinds of package: - `@astryxdesign/core`: the components, the theme system and utilities. Its `package.json` declares 172 export paths, one per component module plus CSS and token entry points. - `@astryxdesign/cli`: docs, search, templates, scaffolding, themes and codemods. - `@astryxdesign/build`: plugins for consumers who compile StyleX from source instead of importing the pre-built CSS. - Seven themes: neutral, butter, chocolate, matcha, stone, gothic and y2k. An eighth directory, `probe`, is a test theme. Three more packages exist but are not stable: `lab` (experimental components, used by Storybook and not published), and `charts` and `vega`, which the README says ship only under the `@canary` dist-tag. ### Counting the 150+ Every component documents itself in a `.doc.mjs` file next to its source, typed as a `ComponentDoc` or a `HookDoc` from the CLI's authoring contract. That makes the count checkable without running anything. In `packages/core/src` (measured): | What | Count | |---|---| | `ComponentDoc` files | 198 | | `HookDoc` files | 63 | | directories holding them | 104 | | `ComponentDoc` files in `lab`, `charts`, `richtext` | 44, 6, 4 | | test files under `core/src` | 812 | The 198 include sub-components with their own docs (`DialogHeader`, `TableCell`, `TypeaheadItem`), and the 104 directories are closer to what a person would call a component. Either way "150+ components" holds for Core alone if you count documented parts, and it would hold for directories if you added `lab`. The README's own phrase is "150+ accessible components". I would read it as documented parts. One small tell that the number moves: the CLI's agent-docs generator counts components live from the installed Core, and when it cannot, it falls back to a hard-coded string, `'90+'` (`packages/cli/foundation/agent-docs/agent-docs.mjs:620`). An agent in a project where discovery fails is told about 90. ### The 13,000 apps and eight years These are reported, and only by Meta. The README says Astryx "grew inside Meta over the last eight years" and powers "13,000+ apps". The docsite blog says "13,000+ products" in one post and "over 13,000 apps" in another. Nothing in the repository can check either number, and nothing tries. The git history starts fresh: my clone is one commit deep, and the copyright line says 2026. I treat both as Meta's description of the internal system this was rebuilt from, which is what the blog post on how it works says it is: "that internal system rebuilt on the open-source stack". ### Theming in one paragraph A theme is a flat map of CSS custom properties, each either one value or a `[light, dark]` pair that becomes `light-dark()`. `defineTheme` in `core/src/theme/defineTheme.ts` takes that map plus optional icon overrides; the theme either injects a `