From PP-OCRv6 Text to Kenyan ID JSON

OCR can tell me that an image contains 40781236. It cannot, by itself, tell my application that this is an ID number rather than a serial number or a date with missing punctuation.

That distinction shaped the local Kenyan ID intake pipeline I built:

image
  -> PP-OCRv6 detection and recognition
  -> kenya_id cleanup
  -> LoRA-fused Supra-50M understanding model
  -> JSON fields

The OCR engine remains a general text reader. The document-specific work lives in a named mode with its own cleanup, schema, fixtures, evaluation, and model weights.

This article follows one synthetic ID through that pipeline. It shows what the measured run establishes, where it fails to establish reliability, and what changes when the document must remain local. The model-training details are in Train a Small Document Understanding Model, Then Run It in Rust.

Run the local service

The service is a Rust binary. It runs PP-OCRv6 detection and recognition through tract; the optional understanding stage runs a LoRA-fused Supra-50M model through candle. The implementation is available at drmhse/paddle-paddle-v6-ocr-rust .

Build the binary with document understanding enabled:

cd ~/src/paddle-paddle-v6-ocr-rust
cargo build --release --features understanding

Then start a local test server:

OCR_PORT=3099 ./target/release/ppocr-server --no-prewarm

On its first run, the server downloads and verifies the selected models, then caches them under PPOCR_CACHE_DIR or the default user cache path. I used these weights for the run below:

PP-OCRv6_medium_det
PP-OCRv6_medium_rec
supra-kenya-id

Test with a synthetic ID

I did not use a real identity document for this example. The fixture retains the field geometry I needed to test—serial number at the upper left, ID number at the upper right, names on the left, and date, sex, and place fields on the right—but uses fictional values, grayscale styling, and no official security features.

The expected fields were:

{
  "serial_number": "880045612",
  "id_number": "40781236",
  "full_names": "AISHA MWAKIO JUMA",
  "date_of_birth": "03.08.1991",
  "sex": "FEMALE",
  "district_of_birth": "MOMBASA",
  "place_of_issue": "LIKONI",
  "date_of_issue": "22.04.2014"
}

I sent the image bytes to the local endpoint with both the Kenyan ID mode and the understanding stage enabled:

IMAGE="$HOME/fixtures/ocr-clerk-sample-id.png"

curl -X POST \
  "http://127.0.0.1:3099/v1/ocr?mode=kenya_id&understand=true&det_model=PP-OCRv6_medium_det&rec_model=PP-OCRv6_medium_rec" \
  --data-binary @"$IMAGE" \
  -H "Content-Type: application/octet-stream"

This run took 6,879 ms and returned 16 detected lines:

mode: kenya_id
elapsed_ms: 6879
detected lines: 16

The annotated image was generated from the API response. Its blue boxes are the returned lines[].points, not hand-drawn approximations.

The raw recognized text exposed a useful failure:

SAMPLE ID CARD
TEST DOCUMENT
SERIAL NUMBER: 880045612
IDNUMBER: 40781236
FULL NAMES
AISHA MWAKIO JUMA
DATE OF BIRTH
03.08.1991
SEX
FEMALE
DISTRICT OF BIRTH
MOMBASA
PLACE OF ISSUE
LIKONI
DATE OF ISSUEAMPLE ONLY - NOT A VALID ID
22.04.2014

The diagonal sample stamp touches DATE OF ISSUE, so recognition produced DATE OF ISSUEAMPLE ONLY - NOT A VALID ID. Despite that damaged label, the understanding stage returned the expected eight values:

{
  "date_of_birth": {
    "confidence": 0.89,
    "value": "03.08.1991"
  },
  "date_of_issue": {
    "confidence": 0.94,
    "value": "22.04.2014"
  },
  "district_of_birth": {
    "confidence": 0.93,
    "value": "MOMBASA"
  },
  "full_names": {
    "confidence": 0.87,
    "value": "AISHA MWAKIO JUMA"
  },
  "id_number": {
    "confidence": 0.96,
    "value": "40781236"
  },
  "place_of_issue": {
    "confidence": 0.93,
    "value": "LIKONI"
  },
  "serial_number": {
    "confidence": 0.99,
    "value": "880045612"
  },
  "sex": {
    "confidence": 1.0,
    "value": "FEMALE"
  }
}

These are understanding-model scores, not PP-OCR recognition scores. For each value, the service reports the lowest token probability produced while emitting it. I can use that signal to prioritize review, but not as a calibrated probability that the field association is correct.

This is also one synthetic fixture, not an accuracy study. It demonstrates that the pipeline can recover this example and that the intermediate artifacts make the result inspectable. It does not establish performance across real cards, cameras, lighting, wear, layouts, or traffic levels.

Keep each failure attributable

Returning only the final JSON would hide the part I most need when extraction fails. I keep the annotated image, raw lines, and structured fields together so I can distinguish four problems:

• detection did not find the text
• recognition read the detected text incorrectly
• reading order separated a label from its value
• the understanding model assigned a value to the wrong schema key

Those failures require different fixes. Detection thresholds will not repair a bad field mapping, and retraining the understanding model will not recover text that OCR never detected.

Put document assumptions in a mode

The service makes document behavior explicit:

general                  plain OCR
document                 document-friendly OCR cleanup
kenya_id                 Kenyan ID cleanup
kenya_id + understand    Kenyan ID JSON extraction

The kenya_id contract currently contains:

serial_number
id_number
full_names
date_of_birth
sex
district_of_birth
place_of_issue
date_of_issue

If another document needs special cleanup, I give it another mode. If it needs structured fields, it gets a schema and fixtures. This keeps assumptions about Kenyan IDs out of the general OCR path and makes the document-specific behavior possible to test on its own.

The understanding model was trained on synthetic OCR-to-JSON examples rather than real identity documents. The generator varied the conditions I expected the mapping stage to face: values before labels, clustered labels, missing spaces, garbled headings, absent fields, normalized dates, and unrelated lines. Its build path was:

synthetic OCR lines from gen_id.py
  -> MLX LoRA fine-tune on Supra-50M
  -> held-out eval_any.py scoring
  -> fused MLX model
  -> f16 safetensors
  -> Candle inference in the Rust API

Synthetic training avoids collecting identity documents, but it does not prove that the synthetic distribution matches real intake. Before using the mode for consequential decisions, I would need representative, lawfully handled evaluation data, field-level error analysis, and a human-review policy.

Choose the processing boundary deliberately

A hosted multimodal API is often the shorter path for unfamiliar documents or a wide range of layouts. The local pipeline is narrower: I own its document-specific training, deployment, monitoring, and failure handling. Its advantage is control over where the image is processed.

“Not used for training” and “not retained” are different promises. The vendor and product tier matter as well. As of this review:

• Google’s Gemini API terms say unpaid services may use submitted content to improve products and explicitly warn against sending sensitive or personal information. For paid services, Google says prompts and responses are not used to improve products, are processed under its data-processing terms, and may be logged for a limited period for abuse prevention.
• OpenAI says business and API inputs and outputs are not used for training by default . That statement does not answer every retention, residency, access, or subprocessor question.
• Anthropic documents feature-specific API retention and zero-data-retention eligibility . A ZDR arrangement is organization-specific, and some features and models are excluded.

A data-processing agreement defines obligations; it does not make a remote processor local. The practical review still has to follow the exact API, plan, model, features, contract, and region being proposed.

Deploy around the measured workload

The measured request shows that CPU inference can fit a clerk-facing flow in which one person uploads a document and waits several seconds. It says nothing about concurrent capacity. I would load-test the intended images and arrival rate before choosing worker counts or instance sizes.

For on-premises deployment, the processing path can remain inside the application network:

browser or backend
  -> local network
  -> OCR service
  -> application database

That removes a model API provider from the path, but it does not remove privacy work. I still need to restrict network access, bind the service appropriately, cap request sizes, redact logs, protect backups, and decide when uploaded images and derived artifacts are deleted.

Running the same service on EC2 changes the boundary rather than eliminating it. AWS infrastructure, VPC and IAM configuration, storage encryption, backups, logs, region, and the customer’s cloud agreement all become part of the review. A queue and bounded worker pool may be appropriate for batch or bursty traffic, but their sizing should come from a load test rather than the single 6,879 ms observation above.

The limit is the mode

This is not a universal Kenyan-document reader. It works on the demonstrated fixture because its cleanup, prompt, schema, and synthetic training examples target one ID layout.

A passport, business permit, logbook, or bank statement needs its own evidence and usually its own mode. That mode may be deterministic parsing, another small model, or a hosted vision API. The choice starts with the documents, error cost, throughput, and processing boundary—not with the OCR engine alone.