Checkbox Detection Prompt Template

This prompt is designed for document automation engineers who need to programmatically determine the state of every checkbox on a form after the checkbox regions have already been identified. It is a state classifier, not a detector. The ideal user has an upstream process—such as a PDF parser extracting form field widgets, a heuristic algorithm identifying square regions, or a separate object-detection model—that provides precise bounding box coordinates or pre-cropped images of individual checkboxes. The job-to-be-done is reliably classifying each provided region as checked, unchecked, or indeterminate to automate data entry, form completeness audits, or compliance reviews.

Use this prompt when your pipeline already produces clean, isolated checkbox crops or exact coordinates and you need a high-accuracy classification step. It is particularly effective for processing scanned forms where visual noise, fax artifacts, or handwritten marks make rule-based pixel-counting approaches unreliable. The prompt expects you to supply either a list of cropped checkbox images or a full page image paired with an array of bounding box coordinates. It returns a structured JSON array where each entry includes the original region identifier, the classified state, and a confidence score. This output can be directly ingested by downstream validation logic or a human review queue.

Do not use this prompt for free-form checkbox discovery on a full page without pre-cropping or coordinate inputs. It is not designed to locate checkboxes; that task requires an object-detection model, a region-proposal prompt, or a deterministic PDF form parser. Using this prompt for discovery will produce unreliable results and hallucinated bounding boxes. Additionally, avoid this prompt for real-time video streams, multi-page documents where checkbox regions span page breaks without coordinate normalization, or forms where the visual distinction between a printed box border and a checkmark requires sub-pixel analysis beyond typical vision model capabilities. For those scenarios, combine this prompt with a dedicated detection step and coordinate normalization logic.

This prompt template is the core instruction set for a checkbox detection workflow. It is designed to be sent to a multimodal large language model alongside a page image or a structured representation of the document. The prompt instructs the model to identify every checkbox on a page, classify its state, and return the results in a strict JSON schema. Before integrating this prompt into your pipeline, you must replace each square-bracket placeholder with the specific data, rules, and format requirements for your use case.

textCopy
You are a document layout analyst specialized in form field detection. Your task is to analyze the provided document page and identify every checkbox present. A checkbox is a small square or rectangular region, typically empty or containing a mark (check, cross, fill, dot), intended for binary or multi-option selection.

For each checkbox you find, you must determine its state and provide precise spatial coordinates. Use the following definitions:
- **checked**: The box contains a clear intentional mark (checkmark, cross, solid fill, or 'X') that fills or crosses the interior.
- **unchecked**: The box is visually empty or contains only pre-printed guide marks (e.g., faint dots, background patterns).
- **indeterminate**: The box contains a partial, ambiguous, or erased mark where a definitive state cannot be assigned with high confidence.

[INPUT]

[CONSTRAINTS]

You must return your findings as a single JSON object conforming to the following schema. Do not include any text outside the JSON object.

[OUTPUT_SCHEMA]

Before finalizing your output, double-check your work against these rules:
- Do not classify printed box borders or decorative elements as checkboxes.
- A checkmark that clearly extends into an adjacent field should be flagged with a low confidence score for that adjacent field.
- If the scan quality is low, increase the confidence threshold for the 'checked' state to avoid false positives.

To adapt this template, replace [INPUT] with a clear description of the data being passed, such as a base64-encoded image or a reference to a page in an S3 bucket. Replace [CONSTRAINTS] with operational rules, like ignoring specific page areas or handling known form artifacts. The [OUTPUT_SCHEMA] placeholder must be replaced with your exact JSON schema, including fields for checkbox_id, state, bounding_box (with x, y, width, height normalized to 0-1000), and confidence (a float from 0.0 to 1.0). For high-stakes automation, add a final instruction to output a review_required boolean if any state is indeterminate or confidence is below a defined threshold, ensuring a human-in-the-loop checkpoint.

The checkbox detection prompt is designed to be called once per page image or per document region containing candidate checkbox fields. In a production pipeline, you should first run a form field detection or region proposal step to identify candidate checkbox locations, then invoke this prompt on each cropped region or on the full page with bounding box references. This two-stage approach reduces token usage and improves accuracy by giving the model focused visual context. For high-throughput pipelines processing thousands of forms per day, batch multiple checkbox regions into a single prompt call where possible, but keep each batch under 8-10 checkboxes to avoid attention dilution across distant regions.

Wire the prompt into your application with a validation layer that checks the output schema before accepting results. Every checkbox classification must include a state field (one of checked, unchecked, indeterminate), a confidence score between 0.0 and 1.0, and bounding_box coordinates normalized to the input image dimensions. Reject outputs that contain states outside the allowed enum, confidence values outside the valid range, or bounding boxes that fall outside the image boundaries. For indeterminate classifications, require the model to include an indeterminate_reason field (e.g., obscured, partial_mark, low_contrast, ambiguous) to support downstream routing decisions. Implement a retry strategy with exponential backoff: if validation fails, resubmit the same input with the validation error message appended as a correction instruction. After two failed retries, route the region to a human review queue rather than looping indefinitely.

For model selection, use a vision-capable model that supports structured output with bounding box spatial reasoning. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are suitable for this task. Avoid text-only models that require pre-extracted OCR coordinates, as checkbox state detection depends on visual features like mark density, fill pattern, and spatial relationship to the box boundary that OCR pipelines often lose. If you must use a text-only model, preprocess images with a dedicated object detection model to extract checkbox regions and visual features, then pass those features as structured context rather than relying on the LLM to interpret raw coordinate strings.

Logging and observability are critical because checkbox detection failures cascade into form completeness decisions. Log every invocation with: the input image hash or document ID, the prompt version, the model and temperature used, the raw output, validation pass/fail status, retry count, and the final accepted or rejected classification. For checkboxes routed to human review, log the reviewer's correction to build a feedback dataset for future prompt improvements or fine-tuning. Implement a confidence threshold in your application layer: classifications with confidence below 0.85 should trigger human review automatically, while those above 0.95 can proceed through automated pipelines. The 0.85-0.95 band should include a sampled review (e.g., 10% of cases) to monitor for drift.

Failure modes to instrument for include: false positives on printed box borders that resemble checkmarks in low-resolution scans, false negatives on light pencil marks or faded ink, and confusion when a checkmark from an adjacent field extends into the target checkbox boundary. Add eval assertions that specifically test these cases: include test images with printed decorative borders near checkboxes, faint marks at varying opacity levels, and densely packed checkbox grids where marks from neighboring fields bleed into target regions. Run these evals on every prompt change before deployment. For regulated use cases such as insurance claims, government forms, or medical intake, always require human confirmation on any checkbox classified as checked or indeterminate before the form proceeds to downstream processing—never treat the model's classification as ground truth in high-stakes workflows.