← Projects

Shoebox — Bill Photo to GST Ledger, Offline

Active
Next.js 16TypeScriptOllama (qwen3.5:9b vision)VitestMIT
View Repository →
▶Product demo

Problem

Small shops in India keep supplier bills in a pile until the accountant asks. Typing them in is slow, and the bills carry mistakes nobody checks: a GSTIN copied wrong, CGST that doesn't match SGST, a handwritten total with two digits swapped, IGST charged on a same-state purchase. Each one can cost input tax credit.

Business Impact

Input tax credit is real money for a trading shop, and it is lost to boring errors: a misread GSTIN, a wrongly charged tax type, a bill entered twice. Shoebox catches those before the accountant sees the register, runs on the shop's own laptop with no subscription, and keeps every bill photo on that machine.

System Approach

  • Local vision model (qwen3.5:9b via Ollama) reads the photo into a fixed JSON schema; fields stream into the UI as it writes
  • Plain TypeScript checks: GSTIN pattern, state code and Luhn mod-36 checksum; quantity × rate − discount = amount; tax against the printed rates; CGST = SGST; tax type by supplier state vs place of supply; grand total; date, 16-character invoice number, GST rates, HSN length
  • Duplicate guard against the ledger: same supplier GSTIN and invoice number can't be claimed twice
  • Review sheet in a modern bahi-khata style: red-pencil underline on the exact wrong field, stamped verdicts (Checked / Look again / Not ready), one-tap GSTIN fixes
  • Ledger with input-tax-credit ready to claim vs on hold, and a purchase-register CSV with one row per GST rate
  • Eval script scores the model against generated sample bills with known truth, including a handwritten bill book

Key Decisions & Trade-offs

  • The model never judges a bill — only arithmetic and published GST rules produce verdicts, so the same bill always gets the same answer
  • Copy, don't correct — the prompt forbids fixing numbers, so the checks see the bill as printed
  • Offline by default — a shop's purchase data stays on its own laptop; no API key, no account
  • One-tap fixes only when exactly one look-alike swap makes a GSTIN valid; ambiguous cases list every candidate instead of guessing
  • Framework-free check library with 35 unit tests, so the rules are testable without a model

Current Status

Public and open source (MIT) at github.com/rohitguta2432/shoebox. Reads, checks, fixes, saves and exports end to end on a laptop; eval on four sample bills: 188/192 fields exact, 4/4 verdicts right, ~17 s per bill on an M5 Pro.

Roadmap

  • Tally XML voucher export alongside the CSV
  • Photo-from-phone on the same Wi-Fi, straight into the pile
  • A WhatsApp intake so a supplier bill photo becomes a ledger entry

What I'd Improve Next

  • Per-HSN rate hints so an item billed at the wrong slab gets flagged
  • Tax-inclusive retail receipts handled as their own bill type

Explore More