MCP server · computer vision

Compare your UI. Inspect the differences. Refine the details.

A reference goes in, your build goes in. Get visual scores and a suggested fix list. Check the diagnostics to decide what to change.

Install it Watch the case study View on GitHub ↗
reference
Reference design: a purple-to-orange brand gradient
candidate
Candidate build: the same gradient with its hues drifted

Illustrative scorecard. These values were not measured from the images shown.

In practice · iOS recreation

Drylendar, rebuilt as DryLender.

See how regional image diagnostics and sampled transitions helped guide an independent iOS recreation. These edited walkthroughs provide visual context, not a synchronized benchmark. Play and scrub the reference and clone separately.

Original reference

Drylendar

Silent excerpts of the settings, calendar settings and drink editor. Watch the settings controls and the logging card's changing drink count and appearance.

Edited and redacted reference footage. Original app: Drylendar by App Craft Consulting.

Read the visual walkthrough

The portrait recording opens in Settings with purchases, notifications, calendar, social-account and privacy controls. A cut shows Intermediate shades, appearance choices and a timezone row whose value is covered. Another cut opens the green zero-drink editor. It briefly closes to the calendar and reopens. The count rises through ten as the card changes through yellow, orange and red. A Blackout toggle changes it between red and dark gray. Down/up arrows, Submit and Cancel remain visible. There is no audio.

Open original video

Independent clone

DryLender

Silent demo of the recreated calendar, logging card and supporting screens. Watch the calendar layout, card opening and number changes in the recreation.

Edited DryLender development demo.

Read the visual walkthrough

A title introduces DryLender. Captioned segments show a monthly calendar, a SobriScore sheet and yearly grids, then range selection and a multi-day drink card. Balloons cross the calendar. A past-day editor changes a drink count and returns to the calendar. Party mode opens a drink card and then a Lock Screen Live Activity with the iOS permission prompt still open. Reward cards open and animate. The final section opens Settings and a CSV export share sheet. The edit includes cuts and held frames. There is no audio.

Open clone video

Independent recreation. No affiliation with or endorsement by Drylendar. Third-party UI and footage are not covered by this repository's MIT license.

Compare a region. Refine the detail.

We extracted reference frames and paired them with clone screenshots. Regional compare_designs diagnostics guided calendar geometry and log-card styling, separating local differences from changes in scroll position.

For the log-card opening and a number change, selected frame sequences helped us inspect changes in motion energy and timing with compare_motion. Similar energy does not establish matching animation.

The useful lesson

Select the same state first, then compare it. Extracting matching frames was a manual step here; keeping frame selection separate from scoring made the diagnostics easier to interpret.

Visual scores do not establish gesture correctness. Taps, swipes and navigation need separate behavioral checks.

That preparation step now has a tool. extract_video_frames takes a short window from a local recording and returns absolute frame paths, source timestamps and a saved manifest. Extraction timestamps support manual preparation; comparison does not itself establish matched actions or cadence.

Recording context and comparison limits

These edits follow different routes and timings, so playback is independent. The original includes only settings, calendar settings and the drink editor. The clone demo predates the build-4 tap fix and does not show the latest build.

The development comparisons used matching regions and selected interactions, not these complete edits. For a repeatable comparison, retain each source timestamp and crop, and sample the same action at a comparable cadence and duration. Read motion_status first; static or unreliable captures are inconclusive. No whole-app fidelity score is claimed.

Use Design Compare MCP in your workflow

Measured synthetic examples

See what changed in the comparison.

The same generated inputs, run with the previous and updated pipelines. Before and after refer to pipeline versions, not a better-looking candidate. These cases test specific failure modes, not human ratings or whole-app fidelity.

Baseline b7139a2d29bc5fd2cae567688e5660b501f509ec
Updated dirty working-tree snapshot fa35542962f731dd8dd9317d5448567023ae4ca66c4fefe9e90bc8de1be4d2e9. Base commit ca87a0923e45211e43222d37ded3fd5084017b74 is not a claim that the measured code equals HEAD.

All results JSON · Report and checks · Exact updated source snapshot

Example 1 / 5

A tiny crop should not invent new colors.

Each 768 × 480 crop contains ten complete 48px periods and the same exact RGB histogram. Width normalization does not resample these inputs. This tests color stability only.

Reference
Reference fixture color-periodic-crop-0px.png, crop origin y=0
Candidate · unchanged between pipelines
Candidate fixture color-periodic-crop-1px.png, crop origin y=1
Actual color scores by crop origin
CaseBeforeAfter
Origin +0px100100
Origin +1px91.4100
Origin +2px92.2100
Origin +3px98.04100
Origin +4px94.12100

Before · b7139a2

Layout score, +1px
85.59
Mode
widget

After · dirty snapshot fa35542962f7

Layout score, +1px
85.59
Mode
widget

Read the color row separately from layout. Shifting the crop can change structure even when its color proportions stay the same. Widget mode intentionally skips registration.

Inputs, settings and raw evidence

color_periodic_crop_0px

Crop ten complete 48px periods from origin y=0. Exact RGB histogram is unchanged; canonical width avoids normalization interpolation. Only color invariance is expected, not identical layout/overall.

{
  "reference": "fixtures/color-periodic-crop-0px.png",
  "candidate": "fixtures/color-periodic-crop-0px.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-periodic-crop-0px.png": {
      "mode": "RGB",
      "pixels_sha256": "3ebc0f90308e6cb579bce2ffc1dfa66e7e327e8a88c064ba439bcfa0cebdf3b5",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "d728f90dbc867b1a04559685a854f5ace1d8679b8fe6a19405fd43127b8ae6d9",
      "shape": [
        480,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        },
        "reference": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        }
      },
      "issues": [],
      "layout_identity_score": 100,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "76b5b88d001843df08c48a9e8c041ee998a3e9ea1953cbe30cc9ee047072c857",
      "inputs": {
        "candidate": {
          "height": 480,
          "rgb_sha256": "7e135dc2abc50deb2055fd01e19bd1d39667fa608128b265017e733aff8c681f",
          "width": 768
        },
        "reference": {
          "height": 480,
          "rgb_sha256": "7e135dc2abc50deb2055fd01e19bd1d39667fa608128b265017e733aff8c681f",
          "width": 768
        }
      },
      "run_key": "8a4468d6d53f758bcfb4d9bde958cf226c9d2786c92aa7803064368159ada87d",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

color_periodic_crop_1px

Crop ten complete 48px periods from origin y=1. Exact RGB histogram is unchanged; canonical width avoids normalization interpolation. Only color invariance is expected, not identical layout/overall.

{
  "reference": "fixtures/color-periodic-crop-0px.png",
  "candidate": "fixtures/color-periodic-crop-1px.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-periodic-crop-0px.png": {
      "mode": "RGB",
      "pixels_sha256": "3ebc0f90308e6cb579bce2ffc1dfa66e7e327e8a88c064ba439bcfa0cebdf3b5",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "d728f90dbc867b1a04559685a854f5ace1d8679b8fe6a19405fd43127b8ae6d9",
      "shape": [
        480,
        768,
        3
      ]
    },
    "fixtures/color-periodic-crop-1px.png": {
      "mode": "RGB",
      "pixels_sha256": "015ac45c9cfb2a78b1a210eb0b9dc0f3b000aa4872db688150c302f6f2a967e5",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "8ec8a1b13d5235c5be9ea788527199fb1568e09c3aebd1d5ce99cb8cf804c4f1",
      "shape": [
        480,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        },
        "reference": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        }
      },
      "issues": [],
      "layout_identity_score": 85.58682074911846,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "76b5b88d001843df08c48a9e8c041ee998a3e9ea1953cbe30cc9ee047072c857",
      "inputs": {
        "candidate": {
          "height": 480,
          "rgb_sha256": "33ca9437e3afe01f25df2a5f0d4cbc33aab8e13d60af151f6ae822c3fb1c0116",
          "width": 768
        },
        "reference": {
          "height": 480,
          "rgb_sha256": "7e135dc2abc50deb2055fd01e19bd1d39667fa608128b265017e733aff8c681f",
          "width": 768
        }
      },
      "run_key": "ee098fbb1f715970bcd3559168a0d2ec4f8c6fc3865b0212359230bced9233fe",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

color_periodic_crop_2px

Crop ten complete 48px periods from origin y=2. Exact RGB histogram is unchanged; canonical width avoids normalization interpolation. Only color invariance is expected, not identical layout/overall.

{
  "reference": "fixtures/color-periodic-crop-0px.png",
  "candidate": "fixtures/color-periodic-crop-2px.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-periodic-crop-0px.png": {
      "mode": "RGB",
      "pixels_sha256": "3ebc0f90308e6cb579bce2ffc1dfa66e7e327e8a88c064ba439bcfa0cebdf3b5",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "d728f90dbc867b1a04559685a854f5ace1d8679b8fe6a19405fd43127b8ae6d9",
      "shape": [
        480,
        768,
        3
      ]
    },
    "fixtures/color-periodic-crop-2px.png": {
      "mode": "RGB",
      "pixels_sha256": "439de9b2896c8e0a188cd56d67abb772dd35c6ac1fef56ecb291af923d176f6d",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "508f3683361a547a090eecef4376e8410ca0b847546ad53470c39c6e64ba27ff",
      "shape": [
        480,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        },
        "reference": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        }
      },
      "issues": [],
      "layout_identity_score": 76.07982467965742,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "76b5b88d001843df08c48a9e8c041ee998a3e9ea1953cbe30cc9ee047072c857",
      "inputs": {
        "candidate": {
          "height": 480,
          "rgb_sha256": "f2c0c21003dd1461f641e582264a434131364e52339e592285f2ed68f0598be9",
          "width": 768
        },
        "reference": {
          "height": 480,
          "rgb_sha256": "7e135dc2abc50deb2055fd01e19bd1d39667fa608128b265017e733aff8c681f",
          "width": 768
        }
      },
      "run_key": "5897f6a7d03c748d8199c3f3b98c2437d9d0824f09612ac864118e197905eea7",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

color_periodic_crop_3px

Crop ten complete 48px periods from origin y=3. Exact RGB histogram is unchanged; canonical width avoids normalization interpolation. Only color invariance is expected, not identical layout/overall.

{
  "reference": "fixtures/color-periodic-crop-0px.png",
  "candidate": "fixtures/color-periodic-crop-3px.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-periodic-crop-0px.png": {
      "mode": "RGB",
      "pixels_sha256": "3ebc0f90308e6cb579bce2ffc1dfa66e7e327e8a88c064ba439bcfa0cebdf3b5",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "d728f90dbc867b1a04559685a854f5ace1d8679b8fe6a19405fd43127b8ae6d9",
      "shape": [
        480,
        768,
        3
      ]
    },
    "fixtures/color-periodic-crop-3px.png": {
      "mode": "RGB",
      "pixels_sha256": "731b65b8d18f5bfe43eb2d2ae55bba46900adf4abd6ec77a7f17fc5685227e49",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "f1b58c02c72ff0b018937efde803c9a154f99a04c12e006e284e7d9401a9b957",
      "shape": [
        480,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        },
        "reference": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        }
      },
      "issues": [],
      "layout_identity_score": 69.74407627838826,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "76b5b88d001843df08c48a9e8c041ee998a3e9ea1953cbe30cc9ee047072c857",
      "inputs": {
        "candidate": {
          "height": 480,
          "rgb_sha256": "2d4d034d2586619f5961aaedd3545948317d53a4bd2ddafb7a2a2f963ac811cd",
          "width": 768
        },
        "reference": {
          "height": 480,
          "rgb_sha256": "7e135dc2abc50deb2055fd01e19bd1d39667fa608128b265017e733aff8c681f",
          "width": 768
        }
      },
      "run_key": "7e7b2f147e1c15a472ffa8578a41cd67a57934b972efa74504526970d2bea894",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

color_periodic_crop_4px

Crop ten complete 48px periods from origin y=4. Exact RGB histogram is unchanged; canonical width avoids normalization interpolation. Only color invariance is expected, not identical layout/overall.

{
  "reference": "fixtures/color-periodic-crop-0px.png",
  "candidate": "fixtures/color-periodic-crop-4px.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-periodic-crop-0px.png": {
      "mode": "RGB",
      "pixels_sha256": "3ebc0f90308e6cb579bce2ffc1dfa66e7e327e8a88c064ba439bcfa0cebdf3b5",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "d728f90dbc867b1a04559685a854f5ace1d8679b8fe6a19405fd43127b8ae6d9",
      "shape": [
        480,
        768,
        3
      ]
    },
    "fixtures/color-periodic-crop-4px.png": {
      "mode": "RGB",
      "pixels_sha256": "c825ea1b85cc672b30754042455171b7e6fcbe19ba45d8efbc5ee81b4027a01e",
      "rgb_histogram_sha256": "3197a16134e9b764faae450c5803381b6773a184f1f519a89860d0f71a84c461",
      "sha256": "ed42e41c59f499b944c623050ca00793ec0b524d0931d3145df5adaeb858e9c3",
      "shape": [
        480,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 480,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        },
        "reference": {
          "gray_stddev": 70.47827178790368,
          "near_uniform": false,
          "rgb_stddev_max": 104.36968347637487
        }
      },
      "issues": [],
      "layout_identity_score": 64.30254643036633,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "76b5b88d001843df08c48a9e8c041ee998a3e9ea1953cbe30cc9ee047072c857",
      "inputs": {
        "candidate": {
          "height": 480,
          "rgb_sha256": "e1a1416dc8a814abb08833c5dc0639dfb29ed78ba1d2216f3720eea7e0360e2e",
          "width": 768
        },
        "reference": {
          "height": 480,
          "rgb_sha256": "7e135dc2abc50deb2055fd01e19bd1d39667fa608128b265017e733aff8c681f",
          "width": 768
        }
      },
      "run_key": "9f3ca281f4a9cdd56c79942453638ff103acfb655067ba15f5687ecd49c70723",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

Baseline raw JSON · Current raw JSON · Run report

Example 2 / 5

A real color-mass swap must still register.

The colors stay the same. Light coverage changes from 81.25% to 18.75%. Candidate bytes are identical in both pipeline runs.

Reference
Reference fixture color-mass-light-width-624.png
Candidate · unchanged between pipelines
Candidate fixture color-mass-light-width-144.png

Before · b7139a2

Color score
100
Color status
scored
Finding types
["missing_region","layout_mismatch","spacing_mismatch","extra_region"]

After · dirty snapshot fa35542962f7

Color score
5.81
Color status
scored
Finding types
["missing_region","layout_mismatch","color_distribution","spacing_mismatch","extra_region"]

A lower color score on this changed input can be a useful correction. It does not establish semantic correctness or general perceptual accuracy.

True-change and identical-input controls
Controls use the same before/after pipelines
CaseBeforeAfter
Identical 50% coverage100100
Accent recolored87.295.79
Accent removed92.6194.87
Reference
Reference fixture calendar-reference.png
Candidate · unchanged between pipelines
Candidate fixture calendar-recolored.png

Robustness to tiny crops must not erase visible changes. These are separate generated control inputs, not improvements to the mass-swap candidate.

Inputs, settings and raw evidence

color_mass_81_25_to_18_75

Same two RGB colors, light coverage 81.25% to 18.75%. A major visible mass swap, not recoloring.

{
  "reference": "fixtures/color-mass-light-width-624.png",
  "candidate": "fixtures/color-mass-light-width-144.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-mass-light-width-624.png": {
      "mode": "RGB",
      "pixels_sha256": "524b444f9c833013d9c549d7627ed7c0945efe1038619e0544a6fba24b577398",
      "rgb_histogram_sha256": "dcf80ec50663a6324309e3e96945e794081ee930e201b277d3af2078f9795ded",
      "sha256": "389c774c9b3a4003723c47a1cfe319e740a06db546db1c8785beb1f143723ffc",
      "shape": [
        160,
        768,
        3
      ]
    },
    "fixtures/color-mass-light-width-144.png": {
      "mode": "RGB",
      "pixels_sha256": "43603e9f422c2a7a2ebed5ed345bfd473e078a031dec2d01a1b5cbad24297aaa",
      "rgb_histogram_sha256": "514cb2f566f0a3e017341a230420c0046689ac513b1a5c4438208a1f6e0bbd32",
      "sha256": "973b1dc6ecd8cbf4ae0505e954ed54676c67d88679d56a43463bd8cca505202e",
      "shape": [
        160,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "height": 160,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "height": 160,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 85.08809772817817,
          "near_uniform": false,
          "rgb_stddev_max": 85.08809772817817
        },
        "reference": {
          "gray_stddev": 85.08809772817817,
          "near_uniform": false,
          "rgb_stddev_max": 85.08809772817817
        }
      },
      "issues": [],
      "layout_identity_score": 50.211578408327796,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "90d400cac49fadfc87d30525c132e501fdd308286a73b9c122be4860adfc8e8f",
      "inputs": {
        "candidate": {
          "height": 160,
          "rgb_sha256": "c8049cd1a5f9c09b9c989f7d5297afaed4932654b9e7f839e1abffd14c782a38",
          "width": 768
        },
        "reference": {
          "height": 160,
          "rgb_sha256": "90a26fb763d6cf0adb0dad8d8c058c8af248931fe8c0a118c026c870bd731ef2",
          "width": 768
        }
      },
      "run_key": "43a9c8072b2d05797508adf11f86e795ce7e0a9ce71c92e9b752446299ffa01d",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

color_mass_50_to_50

Same two RGB colors, light coverage 50% to 50%. Canonical-width input and exact old sampling boundaries isolate mass changes; widget mode skips registration.

{
  "reference": "fixtures/color-mass-light-width-384.png",
  "candidate": "fixtures/color-mass-light-width-384.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/color-mass-light-width-384.png": {
      "mode": "RGB",
      "pixels_sha256": "cbe6f088ed13544a02c80e9436344bb2fe9ed67df65fec68f830f4655a40d22f",
      "rgb_histogram_sha256": "43869d9e1cbd4003867c97efa95a768da95aae11d26c5bec5558169bbc88ade1",
      "sha256": "f16230936aaa2e1a0f5f93c8fdb3d1e4f27ca760938b9e61d7ec6f40c67d137d",
      "shape": [
        160,
        768,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "height": 160,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "height": 160,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 160,
        "orig_w": 768,
        "scale": 1
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 109,
          "near_uniform": false,
          "rgb_stddev_max": 109
        },
        "reference": {
          "gray_stddev": 109,
          "near_uniform": false,
          "rgb_stddev_max": 109
        }
      },
      "issues": [],
      "layout_identity_score": 100,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "7370d9bdbdc816c72c61e1843d7582fd7ed4af92801e92f9b3f39ca60791f2f5",
      "inputs": {
        "candidate": {
          "height": 160,
          "rgb_sha256": "8994daeda2007dee176f132701efcba3b718da117bb312a95ba5ce3768abce98",
          "width": 768
        },
        "reference": {
          "height": 160,
          "rgb_sha256": "8994daeda2007dee176f132701efcba3b718da117bb312a95ba5ce3768abce98",
          "width": 768
        }
      },
      "run_key": "734772a6a1f9b10dc9fb074fbf2368ea2cd2fe77c6d602d0f13676e67620bd20",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

accent_recolored

Replace all blue calendar accents with red; preserve geometry and other colors.

{
  "reference": "fixtures/calendar-reference.png",
  "candidate": "fixtures/calendar-recolored.png",
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/calendar-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "458ef06706ae5589837ca1bac20f5db60abc879827bb62375efef1e86c92f905",
      "sha256": "630f2b254b64e52791583af453335528ef7fe9237cdee77bcc5fc9369e02aa03",
      "shape": [
        320,
        384,
        3
      ]
    },
    "fixtures/calendar-recolored.png": {
      "mode": "RGB",
      "pixels_sha256": "19f548378391a1668a66a45005f8d796655a6a1676ce8288c54c8927e44505b3",
      "sha256": "f620e434a6eb81f98853590ad60843a04f1e859cc3f06e0cb152981c9080753f",
      "shape": [
        320,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "cc": 0.9999,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    },
    "alignment": {
      "cc": 0.9999,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen",
      "outcome": "applied",
      "reason": "ecc_accepted"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 81.53886612028334,
          "near_uniform": false,
          "rgb_stddev_max": 85.98131462712692
        },
        "reference": {
          "gray_stddev": 81.40526778157218,
          "near_uniform": false,
          "rgb_stddev_max": 87.45570871710888
        }
      },
      "issues": [],
      "layout_identity_score": 99.99492684108111,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "26f34f4a3c2cccfa1bbb947a20ac795bac6f532d9fefb0e4e5c4de4525154574",
      "inputs": {
        "candidate": {
          "height": 320,
          "rgb_sha256": "994dbe6f5660545a3dfe2db37b568449bea3d3f7b14673e9b1be7f89973774ca",
          "width": 384
        },
        "reference": {
          "height": 320,
          "rgb_sha256": "86682b46243f49037c4dbbaa876d8a3154e4a0dcd7a105e78e248ee6a4cc526a",
          "width": 384
        }
      },
      "run_key": "2fd7319a72b81875d2353a1ec37d78c0ad40e6f6e0725f13976895c8b4d7e1ba",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

accent_missing

Remove all blue calendar accents, leaving their white card backgrounds.

{
  "reference": "fixtures/calendar-reference.png",
  "candidate": "fixtures/calendar-missing-accent.png",
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/calendar-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "458ef06706ae5589837ca1bac20f5db60abc879827bb62375efef1e86c92f905",
      "sha256": "630f2b254b64e52791583af453335528ef7fe9237cdee77bcc5fc9369e02aa03",
      "shape": [
        320,
        384,
        3
      ]
    },
    "fixtures/calendar-missing-accent.png": {
      "mode": "RGB",
      "pixels_sha256": "5208b7a4bc0b653cfa65eddcfa68b68715aa30afe80ee41206a221eea1fdcbc5",
      "sha256": "8dc54fb85df0830de34bdf074a0032ce423e4af3df05b711f4d591d5bd7dfe89",
      "shape": [
        320,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "cc": 0.9743,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    },
    "alignment": {
      "cc": 0.9743,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen",
      "outcome": "applied",
      "reason": "ecc_accepted"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 80.82157666374721,
          "near_uniform": false,
          "rgb_stddev_max": 86.20106489565393
        },
        "reference": {
          "gray_stddev": 81.40526778157218,
          "near_uniform": false,
          "rgb_stddev_max": 87.45570871710888
        }
      },
      "issues": [],
      "layout_identity_score": 97.9270246455535,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "26f34f4a3c2cccfa1bbb947a20ac795bac6f532d9fefb0e4e5c4de4525154574",
      "inputs": {
        "candidate": {
          "height": 320,
          "rgb_sha256": "8b2b70e825c548c0a15c72829678994d19894f1f32d31c5d71fb9f4a7798b376",
          "width": 384
        },
        "reference": {
          "height": 320,
          "rgb_sha256": "86682b46243f49037c4dbbaa876d8a3154e4a0dcd7a105e78e248ee6a4cc526a",
          "width": 384
        }
      },
      "run_key": "2267731361592db563563dec3a9df3012f2f0683584a6a0a3ac6eb1193bc2563",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

Baseline raw JSON · Current raw JSON · Run report

Example 3 / 5

Equal energy can hide an opposite sampled rhythm.

The candidate reverses all nine frames. The amount of frame-to-frame change stays equal, but its sampled profile differs. No source timestamps or capture cadence are supplied.

Reference
  1. Reference fixture motion-reference-00.png, frame 0Frame 0
  2. Reference fixture motion-reference-01.png, frame 1Frame 1
  3. Reference fixture motion-reference-02.png, frame 2Frame 2
  4. Reference fixture motion-reference-03.png, frame 3Frame 3
  5. Reference fixture motion-reference-04.png, frame 4Frame 4
  6. Reference fixture motion-reference-05.png, frame 5Frame 5
  7. Reference fixture motion-reference-06.png, frame 6Frame 6
  8. Reference fixture motion-reference-07.png, frame 7Frame 7
  9. Reference fixture motion-reference-08.png, frame 8Frame 8
Candidate · unchanged between pipelines
  1. Candidate fixture motion-reversed-00.png, frame 0Frame 0
  2. Candidate fixture motion-reversed-01.png, frame 1Frame 1
  3. Candidate fixture motion-reversed-02.png, frame 2Frame 2
  4. Candidate fixture motion-reversed-03.png, frame 3Frame 3
  5. Candidate fixture motion-reversed-04.png, frame 4Frame 4
  6. Candidate fixture motion-reversed-05.png, frame 5Frame 5
  7. Candidate fixture motion-reversed-06.png, frame 6Frame 6
  8. Candidate fixture motion-reversed-07.png, frame 7Frame 7
  9. Candidate fixture motion-reversed-08.png, frame 8Frame 8

Before · b7139a2

motion_score
100
motion_energy_score
not returned
Meaning
not returned
Energy ratio
1
Frame-index correlation
-0.5153364271492592
Timing status
not returned
Timing reason
not returned
Timing score
not returned

After · dirty snapshot fa35542962f7

motion_score
100
motion_energy_score
100
Meaning
motion_energy_similarity_only
Energy ratio
1
Frame-index correlation
-0.5153364271492592
Timing status
inconclusive
Timing reason
cadence_unknown
Timing score
null

The energy score is not animation fidelity. Negative frame-index correlation describes these samples; unknown cadence leaves elapsed-time alignment inconclusive.

Static control and exact sampled signatures

Before · b7139a2

motion_score
null
motion_status
static
Timing status
not returned

After · dirty snapshot fa35542962f7

motion_score
null
motion_status
static
Timing status
inconclusive
Reference signature
[0.0041,0,0.0146,0,0.0076,0,0.0286,0]
Candidate signature
[0,0.0286,0,0.0076,0,0.0146,0,0.0041]
Frame counts
{"candidate":9,"reference":9}

A true null is shown as null. A field absent from the old worker is shown as not returned.

Inputs, settings and raw evidence

motion_reversed

Reverse frame order. Same mean energy, different sampled rhythm; no capture timestamps are claimed.

{
  "reference": [
    "fixtures/motion-reference-00.png",
    "fixtures/motion-reference-01.png",
    "fixtures/motion-reference-02.png",
    "fixtures/motion-reference-03.png",
    "fixtures/motion-reference-04.png",
    "fixtures/motion-reference-05.png",
    "fixtures/motion-reference-06.png",
    "fixtures/motion-reference-07.png",
    "fixtures/motion-reference-08.png"
  ],
  "candidate": [
    "fixtures/motion-reversed-00.png",
    "fixtures/motion-reversed-01.png",
    "fixtures/motion-reversed-02.png",
    "fixtures/motion-reversed-03.png",
    "fixtures/motion-reversed-04.png",
    "fixtures/motion-reversed-05.png",
    "fixtures/motion-reversed-06.png",
    "fixtures/motion-reversed-07.png",
    "fixtures/motion-reversed-08.png"
  ],
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/motion-reference-00.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-01.png": {
      "mode": "L",
      "pixels_sha256": "1a0c98614c8c370071b47f609c7d98752a03daa0962c880cb99e750ffca7cfce",
      "sha256": "0f6f71b6630a8fbeadcd5852026084bb99ba1edea7da2b5881994ba3b8c53323",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-02.png": {
      "mode": "L",
      "pixels_sha256": "1a0c98614c8c370071b47f609c7d98752a03daa0962c880cb99e750ffca7cfce",
      "sha256": "0f6f71b6630a8fbeadcd5852026084bb99ba1edea7da2b5881994ba3b8c53323",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-03.png": {
      "mode": "L",
      "pixels_sha256": "c05dea69b215a4364684fe683333f199393c2d368b615e272c3efa780bdd4ad1",
      "sha256": "2f9f2fbe0652f89b428e9063e07429251c7604c2b563505cb03bf555d5bdf4ba",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-04.png": {
      "mode": "L",
      "pixels_sha256": "c05dea69b215a4364684fe683333f199393c2d368b615e272c3efa780bdd4ad1",
      "sha256": "2f9f2fbe0652f89b428e9063e07429251c7604c2b563505cb03bf555d5bdf4ba",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-05.png": {
      "mode": "L",
      "pixels_sha256": "5c5752c56196c12a0b55fb4b11216589b9e2a5a890b29e623f689a2200f422d1",
      "sha256": "f0db19fa5b06cb338bd587457b7ce858628195b9be767d2bee3a50bb49a51603",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-06.png": {
      "mode": "L",
      "pixels_sha256": "5c5752c56196c12a0b55fb4b11216589b9e2a5a890b29e623f689a2200f422d1",
      "sha256": "f0db19fa5b06cb338bd587457b7ce858628195b9be767d2bee3a50bb49a51603",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-07.png": {
      "mode": "L",
      "pixels_sha256": "546c16fc13f972c4f86cda49d539439e15e4ee6f7dbfdaeda14b09a8b70493fc",
      "sha256": "bac6f6bca8e0f1130e423ea34db71643eb478cb9a12a493fa1c97ab9cec97878",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reference-08.png": {
      "mode": "L",
      "pixels_sha256": "546c16fc13f972c4f86cda49d539439e15e4ee6f7dbfdaeda14b09a8b70493fc",
      "sha256": "bac6f6bca8e0f1130e423ea34db71643eb478cb9a12a493fa1c97ab9cec97878",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-00.png": {
      "mode": "L",
      "pixels_sha256": "546c16fc13f972c4f86cda49d539439e15e4ee6f7dbfdaeda14b09a8b70493fc",
      "sha256": "bac6f6bca8e0f1130e423ea34db71643eb478cb9a12a493fa1c97ab9cec97878",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-01.png": {
      "mode": "L",
      "pixels_sha256": "546c16fc13f972c4f86cda49d539439e15e4ee6f7dbfdaeda14b09a8b70493fc",
      "sha256": "bac6f6bca8e0f1130e423ea34db71643eb478cb9a12a493fa1c97ab9cec97878",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-02.png": {
      "mode": "L",
      "pixels_sha256": "5c5752c56196c12a0b55fb4b11216589b9e2a5a890b29e623f689a2200f422d1",
      "sha256": "f0db19fa5b06cb338bd587457b7ce858628195b9be767d2bee3a50bb49a51603",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-03.png": {
      "mode": "L",
      "pixels_sha256": "5c5752c56196c12a0b55fb4b11216589b9e2a5a890b29e623f689a2200f422d1",
      "sha256": "f0db19fa5b06cb338bd587457b7ce858628195b9be767d2bee3a50bb49a51603",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-04.png": {
      "mode": "L",
      "pixels_sha256": "c05dea69b215a4364684fe683333f199393c2d368b615e272c3efa780bdd4ad1",
      "sha256": "2f9f2fbe0652f89b428e9063e07429251c7604c2b563505cb03bf555d5bdf4ba",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-05.png": {
      "mode": "L",
      "pixels_sha256": "c05dea69b215a4364684fe683333f199393c2d368b615e272c3efa780bdd4ad1",
      "sha256": "2f9f2fbe0652f89b428e9063e07429251c7604c2b563505cb03bf555d5bdf4ba",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-06.png": {
      "mode": "L",
      "pixels_sha256": "1a0c98614c8c370071b47f609c7d98752a03daa0962c880cb99e750ffca7cfce",
      "sha256": "0f6f71b6630a8fbeadcd5852026084bb99ba1edea7da2b5881994ba3b8c53323",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-07.png": {
      "mode": "L",
      "pixels_sha256": "1a0c98614c8c370071b47f609c7d98752a03daa0962c880cb99e750ffca7cfce",
      "sha256": "0f6f71b6630a8fbeadcd5852026084bb99ba1edea7da2b5881994ba3b8c53323",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-reversed-08.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": []
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    }
  }
}

motion_both_static

Neither side moves. A null motion score is inconclusive, not perfect fidelity.

{
  "reference": [
    "fixtures/motion-static-00.png",
    "fixtures/motion-static-01.png",
    "fixtures/motion-static-02.png",
    "fixtures/motion-static-03.png",
    "fixtures/motion-static-04.png",
    "fixtures/motion-static-05.png",
    "fixtures/motion-static-06.png",
    "fixtures/motion-static-07.png",
    "fixtures/motion-static-08.png"
  ],
  "candidate": [
    "fixtures/motion-static-00.png",
    "fixtures/motion-static-01.png",
    "fixtures/motion-static-02.png",
    "fixtures/motion-static-03.png",
    "fixtures/motion-static-04.png",
    "fixtures/motion-static-05.png",
    "fixtures/motion-static-06.png",
    "fixtures/motion-static-07.png",
    "fixtures/motion-static-08.png"
  ],
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/motion-static-00.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-01.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-02.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-03.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-04.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-05.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-06.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-07.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    },
    "fixtures/motion-static-08.png": {
      "mode": "L",
      "pixels_sha256": "6955058556630e4d905359f6f6d811f9187d2a5b47b0c6862c953ed5cc7b93ba",
      "sha256": "2199c0f6da9106037124b340433375de42f9b400b87eb7b151cc8c36e2e4e334",
      "shape": [
        50,
        100
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": []
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    }
  }
}

Baseline raw JSON · Current raw JSON · Run report

Example 4 / 5

Alignment fallback needs a review signal.

Different flat inputs make ECC undefined. Identity fallback is not successful registration, and a near-uniform observation is not proof of a failed capture.

Reference
Reference fixture ecc-flat-reference.png
Candidate · unchanged between pipelines
Candidate fixture ecc-flat-candidate.png

Before · b7139a2

Alignment method
identity
Fallback
true
Outcome
not returned
Reason
not returned
Diagnostic status
not returned
Issues
not returned

After · dirty snapshot fa35542962f7

Alignment method
identity
Fallback
true
Outcome
identity_fallback
Reason
ecc_nonconvergence
Diagnostic status
review
Issues
[{"code":"near_uniform_input","side":"reference"},{"code":"near_uniform_input","side":"candidate"},{"code":"alignment_unresolved"}]

The old response already had a free-text fallback note. The updated result adds a stable outcome, reason and explicit review issue.

Identical flat-input control

Before · b7139a2

Outcome
not returned
Diagnostic status
not returned
Issues
not returned

After · dirty snapshot fa35542962f7

Outcome
identity_fallback
Diagnostic status
review
Issues
[{"code":"near_uniform_input","side":"reference"},{"code":"near_uniform_input","side":"candidate"}]

Inspect the control separately. Uniform content should not manufacture an unresolved-alignment warning.

Inputs, settings and raw evidence

ecc_failure

Constant images make ECC undefined. Identity fallback must remain visible; not a quality success.

{
  "reference": "fixtures/ecc-flat-reference.png",
  "candidate": "fixtures/ecc-flat-candidate.png",
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/ecc-flat-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "14008f8b2d6f8db1f81cf8b8bde0ce14577f0a366a5149e657a7e404c0d979ba",
      "sha256": "3e324bbcacca2c4faf5a3604c294a1ef2b5d30b26fb1183cd63c282ea1106664",
      "shape": [
        128,
        192,
        3
      ]
    },
    "fixtures/ecc-flat-candidate.png": {
      "mode": "RGB",
      "pixels_sha256": "13560babf5cbf41eccf49d508d574cfe424ba27055635b1cf8151313de64ac2a",
      "sha256": "6aed298d5e18c0ddb034c495c64208ae47dabdafa751093e7a37be0e711e5028",
      "shape": [
        128,
        192,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "screen",
      "note": "ECC did not converge; fell back to identity (error)"
    },
    "canvas": {
      "candidate": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "height": 512,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "screen",
      "note": "ECC did not converge; fell back to identity (error)",
      "outcome": "identity_fallback",
      "reason": "ecc_nonconvergence"
    },
    "canvas": {
      "candidate": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "height": 512,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 0,
          "near_uniform": true,
          "rgb_stddev_max": 0
        },
        "reference": {
          "gray_stddev": 0,
          "near_uniform": true,
          "rgb_stddev_max": 0
        }
      },
      "issues": [
        {
          "code": "near_uniform_input",
          "side": "reference"
        },
        {
          "code": "near_uniform_input",
          "side": "candidate"
        },
        {
          "code": "alignment_unresolved"
        }
      ],
      "layout_identity_score": 38.470692681115636,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "review",
      "version": 1
    },
    "provenance": {
      "comparison_key": "de94bdb8665e5c83ec94f079a4045018a66ecd43fc8432d067e3bba88dbea74a",
      "inputs": {
        "candidate": {
          "height": 128,
          "rgb_sha256": "17626ea6817fce3e79e6f6588abb273f9e65411ebdf76dd8cf7efb2aa9d0f32e",
          "width": 192
        },
        "reference": {
          "height": 128,
          "rgb_sha256": "abf485a99ab309e743825130bd9bc12ae4eb7bee8f8a907ba828c200a251591b",
          "width": 192
        }
      },
      "run_key": "82c8421dc01a81ed79bec61da518d17cb0046af943a5490b7e7d330ac23bd878",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

flat_identical_screen

Identical solid inputs still need a near-uniform observation, not an unresolved-alignment or failed-capture claim.

{
  "reference": "fixtures/ecc-flat-reference.png",
  "candidate": "fixtures/ecc-flat-reference.png",
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/ecc-flat-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "14008f8b2d6f8db1f81cf8b8bde0ce14577f0a366a5149e657a7e404c0d979ba",
      "sha256": "3e324bbcacca2c4faf5a3604c294a1ef2b5d30b26fb1183cd63c282ea1106664",
      "shape": [
        128,
        192,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "screen",
      "note": "ECC did not converge; fell back to identity (error)"
    },
    "canvas": {
      "candidate": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "height": 512,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "screen",
      "note": "ECC did not converge; fell back to identity (error)",
      "outcome": "identity_fallback",
      "reason": "ecc_nonconvergence"
    },
    "canvas": {
      "candidate": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "height": 512,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 0,
          "near_uniform": true,
          "rgb_stddev_max": 0
        },
        "reference": {
          "gray_stddev": 0,
          "near_uniform": true,
          "rgb_stddev_max": 0
        }
      },
      "issues": [
        {
          "code": "near_uniform_input",
          "side": "reference"
        },
        {
          "code": "near_uniform_input",
          "side": "candidate"
        }
      ],
      "layout_identity_score": 100,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "review",
      "version": 1
    },
    "provenance": {
      "comparison_key": "de94bdb8665e5c83ec94f079a4045018a66ecd43fc8432d067e3bba88dbea74a",
      "inputs": {
        "candidate": {
          "height": 128,
          "rgb_sha256": "abf485a99ab309e743825130bd9bc12ae4eb7bee8f8a907ba828c200a251591b",
          "width": 192
        },
        "reference": {
          "height": 128,
          "rgb_sha256": "abf485a99ab309e743825130bd9bc12ae4eb7bee8f8a907ba828c200a251591b",
          "width": 192
        }
      },
      "run_key": "4e66d52be228d094afaa8e1c8268cbff95e1d17bb504a7b49898e661440b31d9",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

Baseline raw JSON · Current raw JSON · Run report

Example 5 / 5

No compared pixels means inconclusive evidence.

These are the original textured inputs, before the declared full-image ignore rectangle. A legacy number can remain high or low even when no pixels remain in scope.

Reference
Reference fixture calendar-reference.png
Candidate · unchanged between pipelines
Candidate fixture calendar-omitted-header-avatar.png

Before · b7139a2

Legacy overall
100
Diagnostic status
not returned
Compared fraction
not returned
Issues
not returned

After · dirty snapshot fa35542962f7

Legacy overall
100
Diagnostic status
inconclusive
Compared fraction
0
Issues
[{"code":"mask_fill_semantics"},{"code":"empty_compare_scope"}]
Legacy overall scores, not evidence of retained content
CaseBeforeAfter
Equal modal fills100100
Different modal fills11

Read the scope status before the number. Fill-based masks can affect metrics; they do not establish correctness of the ignored area.

Full image versus explicit crop, and identity keys
Different scopes, not recovered content
CaseBeforeAfter
Full omitted header38.2937.05
Explicit common crop100100
Reference
Reference fixture calendar-reference.png
Candidate · unchanged between pipelines
Candidate fixture calendar-omitted-header-avatar.png
Reference
Reference fixture calendar-scoped-reference.png
Candidate · unchanged between pipelines
Candidate fixture calendar-scoped-candidate.png

The caller cropped both inputs to [0,80,384,320]. The matching crop excludes the missing header and avatar. It does not fix them. Crop and full-image scores answer different questions.

Worker identity keys for the all-ignored case are below. The baseline does not return these fields. A comparison key identifies the reference and settings; the run key also identifies the candidate. Caller context changes protocol identity, not pixel scores.

Before · b7139a2

comparison_key
not returned
run_key
not returned

After · dirty snapshot fa35542962f7

comparison_key
33197977a5fb134b444fac18ad4542933a038ac5ce0a2997511f02c3371750f2
run_key
b404a91103d16277947928828436d35b339df7b9fc7272a7d514374a4fcd76fa
Actual comparison keys; the baseline lacks this API
CaseBeforeAfter
No declared contextnot returned26f34f4a3c2cccfa1bbb947a20ac795bac6f532d9fefb0e4e5c4de4525154574
Declared context v1not returnedff96497350e900945866fde6e3fdecef9739f46b90db5fc5fb0de2c42eca211a
Declared context v2not returneda48ed9cb261a746d4dcd2efc4ac281d3207f5dd3e6315e3d16a9590b219d3b8f
Inputs, settings and raw evidence

scope_all_ignored

Entire textured image ignored with equal modal edge fills. Legacy high score is not retained evidence.

{
  "reference": "fixtures/calendar-reference.png",
  "candidate": "fixtures/calendar-omitted-header-avatar.png",
  "params": {
    "ignore_regions": [
      [
        0,
        0,
        384,
        320
      ]
    ],
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/calendar-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "458ef06706ae5589837ca1bac20f5db60abc879827bb62375efef1e86c92f905",
      "sha256": "630f2b254b64e52791583af453335528ef7fe9237cdee77bcc5fc9369e02aa03",
      "shape": [
        320,
        384,
        3
      ]
    },
    "fixtures/calendar-omitted-header-avatar.png": {
      "mode": "RGB",
      "pixels_sha256": "49db31b383f02b08474d2c678909f9d9abcd4e5701db99679f4266e29b4fcc47",
      "sha256": "eb986f08ede457d12a2a0b20bb384cfac0abe94f96a77d6974f69e0d61423b34",
      "shape": [
        320,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "ignore_regions": [
        [
          0,
          0,
          384,
          320
        ]
      ],
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "ignore_regions": [
        [
          0,
          0,
          384,
          320
        ]
      ],
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 40.313476801693085,
          "near_uniform": false,
          "rgb_stddev_max": 45.83730413808101
        },
        "reference": {
          "gray_stddev": 81.40526778157218,
          "near_uniform": false,
          "rgb_stddev_max": 87.45570871710888
        }
      },
      "issues": [
        {
          "code": "mask_fill_semantics"
        },
        {
          "code": "empty_compare_scope"
        }
      ],
      "layout_identity_score": 100,
      "scope": {
        "compared_fraction": 0,
        "ignored_fraction": 1
      },
      "status": "inconclusive",
      "version": 1
    },
    "provenance": {
      "comparison_key": "33197977a5fb134b444fac18ad4542933a038ac5ce0a2997511f02c3371750f2",
      "inputs": {
        "candidate": {
          "height": 320,
          "rgb_sha256": "398e5b19d422ebdfafe906eaf4ee444ab5c364c494ffd5e0a94312a7e5ffde17",
          "width": 384
        },
        "reference": {
          "height": 320,
          "rgb_sha256": "86682b46243f49037c4dbbaa876d8a3154e4a0dcd7a105e78e248ee6a4cc526a",
          "width": 384
        }
      },
      "run_key": "b404a91103d16277947928828436d35b339df7b9fc7272a7d514374a4fcd76fa",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [
          [
            0,
            0,
            384,
            320
          ]
        ],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

scope_all_ignored_different_fill

Entire textured image ignored with different per-image fills. Legacy low score is also no retained evidence.

{
  "reference": "fixtures/scope-white-edge.png",
  "candidate": "fixtures/scope-black-edge.png",
  "params": {
    "ignore_regions": [
      [
        0,
        0,
        192,
        128
      ]
    ],
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/scope-white-edge.png": {
      "mode": "RGB",
      "pixels_sha256": "0257933b4bce67ebae1b584fb5000457de417b3717bee45c7ed07c02cd95153e",
      "sha256": "7045141e4c5b5f99190e475dfa9a55e5d8bbfeb3d968505ef17e160fa9a0d939",
      "shape": [
        128,
        192,
        3
      ]
    },
    "fixtures/scope-black-edge.png": {
      "mode": "RGB",
      "pixels_sha256": "304199e16e322fdeb44030d174273bd881fc207b1bada36e0f687862c9d1efce",
      "sha256": "40649e127cac21b87c4f69484641afaeefeab00e433954a8610afb181f205edd",
      "shape": [
        128,
        192,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "ignore_regions": [
        [
          0,
          0,
          192,
          128
        ]
      ],
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "height": 512,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "ignore_regions": [
        [
          0,
          0,
          192,
          128
        ]
      ],
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "height": 512,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 128,
        "orig_w": 192,
        "scale": 4
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 52.25689927525503,
          "near_uniform": false,
          "rgb_stddev_max": 87.86558285219871
        },
        "reference": {
          "gray_stddev": 65.66796192111696,
          "near_uniform": false,
          "rgb_stddev_max": 85.5533306718777
        }
      },
      "issues": [
        {
          "code": "mask_fill_semantics"
        },
        {
          "code": "mask_fill_colors_differ"
        },
        {
          "code": "empty_compare_scope"
        }
      ],
      "layout_identity_score": 0.00999900009999,
      "scope": {
        "compared_fraction": 0,
        "ignored_fraction": 1
      },
      "status": "inconclusive",
      "version": 1
    },
    "provenance": {
      "comparison_key": "a7105baea03e2f35ca13a432ef4fe3801925c6157f02cfa6c2e40b734c342f70",
      "inputs": {
        "candidate": {
          "height": 128,
          "rgb_sha256": "df6030a4d0db0fe254fb4bc434fdf689feb7a1d00e1d6c76d6fdf5b9b4e2b57f",
          "width": 192
        },
        "reference": {
          "height": 128,
          "rgb_sha256": "cc98fa66bf91d257b5c6b2c6d2044ba99bd9fc006a21a446b745151d0e12f9b8",
          "width": 192
        }
      },
      "run_key": "d734c2095e657f306acde7cb02e949a76c7da894860961cbf0b02c680f8b917b",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [
          [
            0,
            0,
            192,
            128
          ]
        ],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

header_omitted_full

Header and avatar absent; full images remain in scope. Not a palette-only control.

{
  "reference": "fixtures/calendar-reference.png",
  "candidate": "fixtures/calendar-omitted-header-avatar.png",
  "params": {
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/calendar-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "458ef06706ae5589837ca1bac20f5db60abc879827bb62375efef1e86c92f905",
      "sha256": "630f2b254b64e52791583af453335528ef7fe9237cdee77bcc5fc9369e02aa03",
      "shape": [
        320,
        384,
        3
      ]
    },
    "fixtures/calendar-omitted-header-avatar.png": {
      "mode": "RGB",
      "pixels_sha256": "49db31b383f02b08474d2c678909f9d9abcd4e5701db99679f4266e29b4fcc47",
      "sha256": "eb986f08ede457d12a2a0b20bb384cfac0abe94f96a77d6974f69e0d61423b34",
      "shape": [
        320,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "cc": 0.3393,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "return_visuals": false
    },
    "alignment": {
      "cc": 0.3393,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen",
      "outcome": "applied",
      "reason": "ecc_accepted"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 40.313476801693085,
          "near_uniform": false,
          "rgb_stddev_max": 45.83730413808101
        },
        "reference": {
          "gray_stddev": 81.40526778157218,
          "near_uniform": false,
          "rgb_stddev_max": 87.45570871710888
        }
      },
      "issues": [],
      "layout_identity_score": 85.46661804963476,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "26f34f4a3c2cccfa1bbb947a20ac795bac6f532d9fefb0e4e5c4de4525154574",
      "inputs": {
        "candidate": {
          "height": 320,
          "rgb_sha256": "398e5b19d422ebdfafe906eaf4ee444ab5c364c494ffd5e0a94312a7e5ffde17",
          "width": 384
        },
        "reference": {
          "height": 320,
          "rgb_sha256": "86682b46243f49037c4dbbaa876d8a3154e4a0dcd7a105e78e248ee6a4cc526a",
          "width": 384
        }
      },
      "run_key": "a433a639dbcde25abda180e2e099a37b8c317c5a3597c3136b195842e9d5d61e",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

header_omitted_scoped

Caller explicitly crops BOTH images to [0,80,384,320], using widget mode. Header/avatar are out of scope, not recovered.

{
  "reference": "fixtures/calendar-scoped-reference.png",
  "candidate": "fixtures/calendar-scoped-candidate.png",
  "params": {
    "mode": "widget",
    "return_visuals": false
  },
  "fixtures": {
    "fixtures/calendar-scoped-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "26ee6e5b59437d3a0283822e592ae6b34ee57a593706f0848d97d994382d59ea",
      "sha256": "78edd9168c34e37b718e1d70e33a2f07614a0d5a571c75a4938663575e5baeb7",
      "shape": [
        240,
        384,
        3
      ]
    },
    "fixtures/calendar-scoped-candidate.png": {
      "mode": "RGB",
      "pixels_sha256": "26ee6e5b59437d3a0283822e592ae6b34ee57a593706f0848d97d994382d59ea",
      "sha256": "78edd9168c34e37b718e1d70e33a2f07614a0d5a571c75a4938663575e5baeb7",
      "shape": [
        240,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "unsupported_metadata_params": [],
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop"
    },
    "canvas": {
      "candidate": {
        "orig_h": 240,
        "orig_w": 384,
        "scale": 2
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 240,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "mode": "widget",
      "return_visuals": false
    },
    "alignment": {
      "fallback": true,
      "method": "identity",
      "mode": "widget",
      "note": "widget mode assumes a pre-aligned crop",
      "outcome": "not_requested",
      "reason": "widget_identity"
    },
    "canvas": {
      "candidate": {
        "orig_h": 240,
        "orig_w": 384,
        "scale": 2
      },
      "height": 480,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 240,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 45.87302849971023,
          "near_uniform": false,
          "rgb_stddev_max": 52.44753212309309
        },
        "reference": {
          "gray_stddev": 45.87302849971023,
          "near_uniform": false,
          "rgb_stddev_max": 52.44753212309309
        }
      },
      "issues": [],
      "layout_identity_score": 100,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "54e84523ef9a1f932649804a24118af4004a23305ebe8c0586c6f546af84d852",
      "inputs": {
        "candidate": {
          "height": 240,
          "rgb_sha256": "82c88751b2d6bac849f9e3c785a65e790a69139d14a31ae1ab480221d5025ecd",
          "width": 384
        },
        "reference": {
          "height": 240,
          "rgb_sha256": "82c88751b2d6bac849f9e3c785a65e790a69139d14a31ae1ab480221d5025ecd",
          "width": 384
        }
      },
      "run_key": "d3a544937106f7dadf3cd320e4658407a06f31ad6cfa198c5de83ead8c88471e",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": null,
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "widget",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

settings_context_v1

Same pair with caller-declared protocol metadata. Metadata must not change numeric scores; protocol changes split comparison keys.

{
  "reference": "fixtures/calendar-reference.png",
  "candidate": "fixtures/calendar-omitted-header-avatar.png",
  "params": {
    "comparison_context": {
      "protocolKey": "d1e9764c0b362634a650a022318f1af1d66ed32ca9ab37a99956e9234bcf340c",
      "scopeId": "generated-calendar-full"
    },
    "return_visuals": false
  },
  "context_manifest": {
    "capture": "fixed-generated-pair",
    "declared_exclusions": [],
    "revision": 1,
    "scope": "generated-calendar-full"
  },
  "fixtures": {
    "fixtures/calendar-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "458ef06706ae5589837ca1bac20f5db60abc879827bb62375efef1e86c92f905",
      "sha256": "630f2b254b64e52791583af453335528ef7fe9237cdee77bcc5fc9369e02aa03",
      "shape": [
        320,
        384,
        3
      ]
    },
    "fixtures/calendar-omitted-header-avatar.png": {
      "mode": "RGB",
      "pixels_sha256": "49db31b383f02b08474d2c678909f9d9abcd4e5701db99679f4266e29b4fcc47",
      "sha256": "eb986f08ede457d12a2a0b20bb384cfac0abe94f96a77d6974f69e0d61423b34",
      "shape": [
        320,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [
      "comparison_context"
    ],
    "alignment": {
      "cc": 0.3393,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "comparison_context": {
        "protocolKey": "d1e9764c0b362634a650a022318f1af1d66ed32ca9ab37a99956e9234bcf340c",
        "scopeId": "generated-calendar-full"
      },
      "return_visuals": false
    },
    "alignment": {
      "cc": 0.3393,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen",
      "outcome": "applied",
      "reason": "ecc_accepted"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 40.313476801693085,
          "near_uniform": false,
          "rgb_stddev_max": 45.83730413808101
        },
        "reference": {
          "gray_stddev": 81.40526778157218,
          "near_uniform": false,
          "rgb_stddev_max": 87.45570871710888
        }
      },
      "issues": [],
      "layout_identity_score": 85.46661804963476,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "ff96497350e900945866fde6e3fdecef9739f46b90db5fc5fb0de2c42eca211a",
      "inputs": {
        "candidate": {
          "height": 320,
          "rgb_sha256": "398e5b19d422ebdfafe906eaf4ee444ab5c364c494ffd5e0a94312a7e5ffde17",
          "width": 384
        },
        "reference": {
          "height": 320,
          "rgb_sha256": "86682b46243f49037c4dbbaa876d8a3154e4a0dcd7a105e78e248ee6a4cc526a",
          "width": 384
        }
      },
      "run_key": "a71b9d0c32de3940cf87b338aebcb6b0027faf9a02c4202f9c5debed127be8b7",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": {
          "protocolKey": "d1e9764c0b362634a650a022318f1af1d66ed32ca9ab37a99956e9234bcf340c",
          "scopeId": "generated-calendar-full"
        },
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

settings_context_v2

Same pair with caller-declared protocol metadata. Metadata must not change numeric scores; protocol changes split comparison keys.

{
  "reference": "fixtures/calendar-reference.png",
  "candidate": "fixtures/calendar-omitted-header-avatar.png",
  "params": {
    "comparison_context": {
      "protocolKey": "51c5a773abf6bd7f355b39ba01ce2d4e1b3a83e8255c6430bb6941dfb7f025c1",
      "scopeId": "generated-calendar-full"
    },
    "return_visuals": false
  },
  "context_manifest": {
    "capture": "fixed-generated-pair",
    "declared_exclusions": [],
    "revision": 2,
    "scope": "generated-calendar-full"
  },
  "fixtures": {
    "fixtures/calendar-reference.png": {
      "mode": "RGB",
      "pixels_sha256": "458ef06706ae5589837ca1bac20f5db60abc879827bb62375efef1e86c92f905",
      "sha256": "630f2b254b64e52791583af453335528ef7fe9237cdee77bcc5fc9369e02aa03",
      "shape": [
        320,
        384,
        3
      ]
    },
    "fixtures/calendar-omitted-header-avatar.png": {
      "mode": "RGB",
      "pixels_sha256": "49db31b383f02b08474d2c678909f9d9abcd4e5701db99679f4266e29b4fcc47",
      "sha256": "eb986f08ede457d12a2a0b20bb384cfac0abe94f96a77d6974f69e0d61423b34",
      "shape": [
        320,
        384,
        3
      ]
    }
  },
  "baseline": {
    "effective_params": {
      "return_visuals": false
    },
    "unsupported_metadata_params": [
      "comparison_context"
    ],
    "alignment": {
      "cc": 0.3393,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    }
  },
  "current": {
    "effective_params": {
      "comparison_context": {
        "protocolKey": "51c5a773abf6bd7f355b39ba01ce2d4e1b3a83e8255c6430bb6941dfb7f025c1",
        "scopeId": "generated-calendar-full"
      },
      "return_visuals": false
    },
    "alignment": {
      "cc": 0.3393,
      "dx": 0,
      "dy": 0,
      "fallback": false,
      "method": "ecc_translation",
      "mode": "screen",
      "outcome": "applied",
      "reason": "ecc_accepted"
    },
    "canvas": {
      "candidate": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "height": 640,
      "ignore_input_space": "reference_original_pixels",
      "ignore_space": "canonical_768",
      "reference": {
        "orig_h": 320,
        "orig_w": 384,
        "scale": 2
      },
      "width": 768
    },
    "diagnostics": {
      "capture": {
        "candidate": {
          "gray_stddev": 40.313476801693085,
          "near_uniform": false,
          "rgb_stddev_max": 45.83730413808101
        },
        "reference": {
          "gray_stddev": 81.40526778157218,
          "near_uniform": false,
          "rgb_stddev_max": 87.45570871710888
        }
      },
      "issues": [],
      "layout_identity_score": 85.46661804963476,
      "scope": {
        "compared_fraction": 1,
        "ignored_fraction": 0
      },
      "status": "ok",
      "version": 1
    },
    "provenance": {
      "comparison_key": "a48ed9cb261a746d4dcd2efc4ac281d3207f5dd3e6315e3d16a9590b219d3b8f",
      "inputs": {
        "candidate": {
          "height": 320,
          "rgb_sha256": "398e5b19d422ebdfafe906eaf4ee444ab5c364c494ffd5e0a94312a7e5ffde17",
          "width": 384
        },
        "reference": {
          "height": 320,
          "rgb_sha256": "86682b46243f49037c4dbbaa876d8a3154e4a0dcd7a105e78e248ee6a4cc526a",
          "width": 384
        }
      },
      "run_key": "b02ffe0a4df149720bb5e7400107628b8f2ba685474fa26d4c2b9a6df6a16ffe",
      "settings": {
        "alignment_policy": "ecc-translation-guards-v1",
        "canonical_width": 768,
        "comparison_context": {
          "protocolKey": "51c5a773abf6bd7f355b39ba01ce2d4e1b3a83e8255c6430bb6941dfb7f025c1",
          "scopeId": "generated-calendar-full"
        },
        "engine_revision": "sha256:af00e69615847f5fbdbbb23c9edf742ae6d38fb785ab518ba658eb7e63498144",
        "ignore_policy": "reference-boxes-per-image-modal-fill-v1",
        "ignore_regions": [],
        "mode": "screen",
        "normalization_policy": "independent-width-modal-bottom-pad-v1",
        "preset": "default",
        "runtime_versions": {
          "Pillow": "12.3.0",
          "numpy": "2.5.1",
          "opencv": "5.0.0",
          "python": "3.13.7",
          "scikit-image": "0.26.0",
          "scikit-learn": "1.9.0",
          "scipy": "1.18.0"
        },
        "weights": {
          "color": 0.2,
          "content": 0.15,
          "layout": 0.35,
          "spacing": 0.15,
          "typography": 0.15
        }
      },
      "version": 1
    }
  }
}

Baseline raw JSON · Current raw JSON · Run report

Run identity and reproduction

Recorded 2026-10-23T15:36:17.848931+00:00. Phase integrated. Integration rerun requested by caller; inspect source hashes and failed checks before publication.

vision_worker.pipeline.compare / compare_motion; isolated immutable snapshots; no stdio/RPC overhead. Synthetic behavior checks do not establish perceptual accuracy, transport behavior or deployment readiness.

/Users/codenameakshay/Development/codenameakshay/design-compare-mcp/worker/.venv/bin/python scripts/reliability-benchmark.py --baseline b7139a2d29bc5fd2cae567688e5660b501f509ec --output /Users/codenameakshay/Development/codenameakshay/design-compare-mcp/docs/reliability/runs/final-rerun --repeats 3 --warmups 1 --phase integrated

Fixture generation: calendar-motion-and-color-controls-v5; Pillow drawing, integer coordinates, NumPy default_rng(74) for nine-stripe colors; exact new256color boundary histograms; review3 common-pixel transport and shortlist controls

Results SHA-256 f602c03535a66066796d086f76660c6dec132920b69502d766c6c94400579741. Harness SHA-256 a0f860e88c22c88ae0331765e4f4d81cf5c6061e37619bc8b672d45caf3db0a4.

{
  "baseline": {
    "files": {
      "worker/pyproject.toml": "bca746f25d7459ac486d53e86f7f6ad5634bbfdcc54db2e642c936130caf7d41",
      "worker/vision_worker/__init__.py": "a9df0cd401acf7ac777a36433291c98e5276347aa2a3ba85d5e6444409028686",
      "worker/vision_worker/__main__.py": "862f6feb3a897b1add4014d524b782be86c092eb27acd165aa3d2c2a4545305b",
      "worker/vision_worker/aggregate.py": "4f694075facbe4a4db024bdd743a78e886ccc7758144f637249bd10e1d68870d",
      "worker/vision_worker/align.py": "10d081789aabaa8fdc026d37593a876693fe129903c64984241770dac2e1436a",
      "worker/vision_worker/io_utils.py": "80ba55b5038cd347bd25c3ca79f649dbc092e0c5517059a89e4a5a484b5af5e9",
      "worker/vision_worker/metrics/__init__.py": "4cd624363c9e2d61624a35eef96bf591e8532521e479fdea93dbb1b62c67fcba",
      "worker/vision_worker/metrics/color.py": "7dc7fa90d348000a633ce885d1d2bac45e2cdf20556a2572ff2fc80d04f319ae",
      "worker/vision_worker/metrics/content.py": "16c93f0e05d08909cfb2c5fceb8855301eae7f92c938ed46fcdd05addf50aea5",
      "worker/vision_worker/metrics/spacing.py": "a63833e3166c3877ba3e5a381b2249f2dda34af83f072c1a9412f827bcdd8fe0",
      "worker/vision_worker/metrics/structure.py": "e9747971a1d717c59de4d74e4e92c2082524a3fa2d4fa4a51d171f0654c78ce3",
      "worker/vision_worker/metrics/typography.py": "ec79568f13e90725ac0dae79d8fb42da2db7210014fe4f8a2722f966882dc0d7",
      "worker/vision_worker/motion.py": "919c94c738821e403a35f18e30ff3b91ac575d5f61b820aee85ec540b5964cb9",
      "worker/vision_worker/normalize.py": "ffb6edc6a18564717e38847f8eb5cf81bef26ad6ed8415d6ef6ff8934c1042fd",
      "worker/vision_worker/pipeline.py": "5e4a01baa04990553d1d2005f3dbfd6a88b6157e7620836e61d067f28a4ed780",
      "worker/vision_worker/visuals.py": "c7b38a035832d91b0bfb5947e5740b4c9aacba15232968a9bcdd8ad6bd3f27d9"
    },
    "sha256": "c6a3181a64d5a832c25a5ac914b7d132166fabb9723eabfa0b349afe94179d7f"
  },
  "current": {
    "files": {
      "worker/pyproject.toml": "de97b5cc117df1cb4744d659f6552a5d28eab63e6e4bf79e8bb3f910cc796fa1",
      "worker/vision_worker/__init__.py": "1d4822f6ffcd2c7d44fb83f83f68b020ff79a978847737e9430ad8cd0584592b",
      "worker/vision_worker/__main__.py": "a0c16f1f2fa961fc816fa65ae2f451c4e0ba2da31d73e82a79586e66cefff7c9",
      "worker/vision_worker/aggregate.py": "4f694075facbe4a4db024bdd743a78e886ccc7758144f637249bd10e1d68870d",
      "worker/vision_worker/align.py": "9be365675ec078b83f00fae86223b019268c4cd414432c2028b88bae70a5998b",
      "worker/vision_worker/io_utils.py": "80ba55b5038cd347bd25c3ca79f649dbc092e0c5517059a89e4a5a484b5af5e9",
      "worker/vision_worker/metrics/__init__.py": "4cd624363c9e2d61624a35eef96bf591e8532521e479fdea93dbb1b62c67fcba",
      "worker/vision_worker/metrics/color.py": "296712351518bf2a88b46dcfa9f0811dc8ce68f8c1989d09d624e87d62724e46",
      "worker/vision_worker/metrics/content.py": "16c93f0e05d08909cfb2c5fceb8855301eae7f92c938ed46fcdd05addf50aea5",
      "worker/vision_worker/metrics/spacing.py": "a63833e3166c3877ba3e5a381b2249f2dda34af83f072c1a9412f827bcdd8fe0",
      "worker/vision_worker/metrics/structure.py": "e9747971a1d717c59de4d74e4e92c2082524a3fa2d4fa4a51d171f0654c78ce3",
      "worker/vision_worker/metrics/typography.py": "ec79568f13e90725ac0dae79d8fb42da2db7210014fe4f8a2722f966882dc0d7",
      "worker/vision_worker/motion.py": "aeb2cf71efb93f6f1b08db23bb272f57b3b7affd864e9ff8f4df657fc4a12042",
      "worker/vision_worker/normalize.py": "ffb6edc6a18564717e38847f8eb5cf81bef26ad6ed8415d6ef6ff8934c1042fd",
      "worker/vision_worker/pipeline.py": "e121550fb92801346b0aca9c99e615c503eb59b0eebff27a26a6cd502faed76e",
      "worker/vision_worker/reliability.py": "5a7567cd1104f206aec1d0f8a42f6aa91086064f701820e51e66cd3969eb68c8",
      "worker/vision_worker/visuals.py": "c7b38a035832d91b0bfb5947e5740b4c9aacba15232968a9bcdd8ad6bd3f27d9"
    },
    "sha256": "fa35542962f731dd8dd9317d5448567023ae4ca66c4fefe9e90bc8de1be4d2e9"
  },
  "environment": {
    "cpu_count": 18,
    "dependencies": {
      "ImageIO": "2.37.4",
      "design-compare-vision-worker": "0.12.0",
      "joblib": "1.5.3",
      "lazy-loader": "0.5",
      "narwhals": "2.24.0",
      "networkx": "3.6.1",
      "numpy": "2.5.1",
      "opencv-python-headless": "5.0.0.93",
      "packaging": "26.2",
      "pillow": "12.3.0",
      "pip": "26.1.2",
      "scikit-image": "0.26.0",
      "scikit-learn": "1.9.0",
      "scipy": "1.18.0",
      "threadpoolctl": "3.6.0",
      "tifffile": "2026.7.14"
    },
    "executable": "/Users/codenameakshay/Development/codenameakshay/design-compare-mcp/worker/.venv/bin/python",
    "loaded_worker_files": {
      "vision_worker": "worker/vision_worker/__init__.py",
      "vision_worker.aggregate": "worker/vision_worker/aggregate.py",
      "vision_worker.align": "worker/vision_worker/align.py",
      "vision_worker.io_utils": "worker/vision_worker/io_utils.py",
      "vision_worker.metrics": "worker/vision_worker/metrics/__init__.py",
      "vision_worker.metrics.color": "worker/vision_worker/metrics/color.py",
      "vision_worker.metrics.content": "worker/vision_worker/metrics/content.py",
      "vision_worker.metrics.spacing": "worker/vision_worker/metrics/spacing.py",
      "vision_worker.metrics.structure": "worker/vision_worker/metrics/structure.py",
      "vision_worker.metrics.typography": "worker/vision_worker/metrics/typography.py",
      "vision_worker.motion": "worker/vision_worker/motion.py",
      "vision_worker.normalize": "worker/vision_worker/normalize.py",
      "vision_worker.pipeline": "worker/vision_worker/pipeline.py",
      "vision_worker.reliability": "worker/vision_worker/reliability.py",
      "vision_worker.visuals": "worker/vision_worker/visuals.py"
    },
    "machine": "arm64",
    "numpy": "2.5.1",
    "opencv": "5.0.0",
    "opencv_threads": 18,
    "platform": "macOS-27.0.1-arm64-arm-64bit-Mach-O",
    "processor": "arm",
    "python": "3.13.7 (v3.13.7:bcee1c32211, Aug 14 2025, 19:10:51) [Clang 16.0.0 (clang-1600.0.26.6)]",
    "source_manifest": {
      "files": {
        "worker/pyproject.toml": "de97b5cc117df1cb4744d659f6552a5d28eab63e6e4bf79e8bb3f910cc796fa1",
        "worker/vision_worker/__init__.py": "1d4822f6ffcd2c7d44fb83f83f68b020ff79a978847737e9430ad8cd0584592b",
        "worker/vision_worker/__main__.py": "a0c16f1f2fa961fc816fa65ae2f451c4e0ba2da31d73e82a79586e66cefff7c9",
        "worker/vision_worker/aggregate.py": "4f694075facbe4a4db024bdd743a78e886ccc7758144f637249bd10e1d68870d",
        "worker/vision_worker/align.py": "9be365675ec078b83f00fae86223b019268c4cd414432c2028b88bae70a5998b",
        "worker/vision_worker/io_utils.py": "80ba55b5038cd347bd25c3ca79f649dbc092e0c5517059a89e4a5a484b5af5e9",
        "worker/vision_worker/metrics/__init__.py": "4cd624363c9e2d61624a35eef96bf591e8532521e479fdea93dbb1b62c67fcba",
        "worker/vision_worker/metrics/color.py": "296712351518bf2a88b46dcfa9f0811dc8ce68f8c1989d09d624e87d62724e46",
        "worker/vision_worker/metrics/content.py": "16c93f0e05d08909cfb2c5fceb8855301eae7f92c938ed46fcdd05addf50aea5",
        "worker/vision_worker/metrics/spacing.py": "a63833e3166c3877ba3e5a381b2249f2dda34af83f072c1a9412f827bcdd8fe0",
        "worker/vision_worker/metrics/structure.py": "e9747971a1d717c59de4d74e4e92c2082524a3fa2d4fa4a51d171f0654c78ce3",
        "worker/vision_worker/metrics/typography.py": "ec79568f13e90725ac0dae79d8fb42da2db7210014fe4f8a2722f966882dc0d7",
        "worker/vision_worker/motion.py": "aeb2cf71efb93f6f1b08db23bb272f57b3b7affd864e9ff8f4df657fc4a12042",
        "worker/vision_worker/normalize.py": "ffb6edc6a18564717e38847f8eb5cf81bef26ad6ed8415d6ef6ff8934c1042fd",
        "worker/vision_worker/pipeline.py": "e121550fb92801346b0aca9c99e615c503eb59b0eebff27a26a6cd502faed76e",
        "worker/vision_worker/reliability.py": "5a7567cd1104f206aec1d0f8a42f6aa91086064f701820e51e66cd3969eb68c8",
        "worker/vision_worker/visuals.py": "c7b38a035832d91b0bfb5947e5740b4c9aacba15232968a9bcdd8ad6bd3f27d9"
      },
      "sha256": "fa35542962f731dd8dd9317d5448567023ae4ca66c4fefe9e90bc8de1be4d2e9"
    },
    "thread_environment": {
      "MKL_NUM_THREADS": "1",
      "NUMEXPR_NUM_THREADS": "1",
      "OMP_NUM_THREADS": "1",
      "OPENBLAS_NUM_THREADS": "1",
      "PYTHONHASHSEED": "0",
      "VECLIB_MAXIMUM_THREADS": "1"
    },
    "worker_module": "worker/vision_worker/__init__.py",
    "worker_version": "0.12.0"
  }
}

Source manifests · Exact fixture generator · All input declarations and hashes

The idea

A reference, a candidate, and evidence to inspect.

A static comparison returns dimension scores, findings and optional diagnostic images. Use the same state, crop, masks and settings when comparing revisions. Scores are heuristic signals, not a guarantee that a higher number looks better.

Built for the loop

Give your agent evidence to review.

Render the same state, compare it, inspect the findings and diagnostic images, then verify the change. Keep the comparison setup fixed. A higher score alone is not an acceptance test.

bounded review · pseudocode 
for attempt in bounded_review_budget:
    render the candidate in the reference state
    compare with fixed crop, masks, mode and weights
    inspect dimension status, findings and diagnostics
    if inconclusive or no verified next fix:
        stop and report
    apply one reviewed fix
    review the next render and run behavioral checks
  • Keep inputs comparable. Preserve reference, crop, masks, mode and weights.
  • Review each finding. Confirm it against the diagnostic images before changing code.
  • Verify the result. Review screenshots and test behavior separately. Use project-specific thresholds only after validation.

See measured synthetic examples

reviewable

Scores to inspect, not promises to optimize.

Compare revisions under the same settings. Confirm changes in the images and dimension results.

five dimensions

Five dimensions to inspect.

Layout, color, content, typography and spacing have separate results to help locate differences.

punch-list

Suggested fixes beside the scores.

Findings suggest color shifts and missing or extra regions. Confirm them in the images before applying a fix.

motion

Inspect change across frames.

Motion diagnostics summarize sampled pixel changes, not animation fidelity. They do not establish matching direction, easing or interaction. Read motion_status first.

deterministic

Repeat the comparison under the same setup.

Keep image bytes, settings, tool version and environment fixed. Repeatability does not guarantee perceptual accuracy.

honest

Published checks, with their limits.

Inspect synthetic regression checks and historical human-label calibration. Each measures a different kind of evidence.

What it measures

What the metrics inspect.

Layout

weight 0.35

SSIM compares grayscale structure. Screen mode may align a global translation; inspect alignment diagnostics.

weakness grayscale; plateaus under heavy blur.

Color

weight 0.20

Projects every pixel onto a fixed color lattice. Combines maximum sliced mass distance across 64 directions with source-density-scaled soft rare-color coverage in chroma-compressed Lab proxy units. Each direction uses the same source-query density correction for own and target occupancy.

weakness not CIEDE2000 or 3D optimal transport. Equal-histogram spatial swaps are invisible. Soft coverage does not prove exact absence. Color-shift findings summarize certified unmatched mass on both sides, not any recoloring operation. Failure to certify absence does not prove presence. The independent color review records a close-hue reversal as recolored mass grows. Local qualification and independent PASS cover the frozen v2 contract; full-project checks and hosted CI remain pending. No universal accuracy or monotonicity claim.

Content

weight 0.15

A contrast-adaptive detector matches the reference's blocks to yours by IoU. Missing or extra blocks become findings.

weakness coarse; region boxes wobble.

Typography

weight 0.15

Text lines by morphology, compared on amount and scale. Not OCR, not font identity.

weakness misses a wrong font; null when no text.

Spacing

weight 0.15

Outer margins and vertical rhythm from major blocks, each within a tolerance.

weakness coarsest; null on sparse views.

Aggregate

geometric

Numeric dimensions combine as a weighted geometric mean. Non-applicable dimensions are omitted and weights renormalized. Inspect failures before interpreting the result.

exp(Σ (wᵢ / Σw) · ln(clamp(scoreᵢ, 1, 100)))

Try it

Explore the formulas.

Browser-only illustrations with adjustable inputs. These controls do not run the image-comparison worker.

Weighted geometric mean

For the same scores and positive weights, the geometric mean is no greater than the arithmetic mean. This does not prevent metric gaming. Slider scores are illustrative inputs.

—
weighted geometric mean
arithmetic mean—
—

Illustrative single-color ΔE

Compare two chosen sRGB colors with CIEDE2000. This illustration does not extract palettes, compare their proportions, or run the full image metric.

→
ΔE2000—

See measured synthetic examples

Install & register

Install the server and vision worker.

Needs Node ≥ 20 and Python ≥ 3.10. capture_frames needs local Chrome. Local recording extraction also needs FFmpeg and ffprobe on PATH.

1 · clone & build 
git clone https://github.com/codenameakshay/design-compare-mcp
cd design-compare-mcp
npm install
npm run build
2 · set up the Python vision worker 
python3 -m venv worker/.venv
worker/.venv/bin/pip install -e './worker[cv]'
npm run smoke   # see output for coverage and skips
3 · register with Claude Code 
claude mcp add design-compare --scope user -- \
  node /abs/path/to/design-compare-mcp/dist/index.js
4 · register with Cursor 
// ~/.cursor/mcp.json
{
  "mcpServers": {
    "design-compare": {
      "command": "node",
      "args": ["/abs/path/to/design-compare-mcp/dist/index.js"]
    }
  }
}

Registers five tools: ping, compare_designs, compare_motion, capture_frames and extract_video_frames. Rebuild dist/ and restart the client after you pull.

Historical evidence

Historical human-label calibration.

An earlier scoring pipeline was evaluated on 26 dark-library components and their Flutter ports, each rated 0–100 by eye. These Spearman correlations have not been revalidated for the reliability changes.

Source: pre-reliability documentation at b7139a2. The underlying run's exact version was not recorded.

−0.03historical default-preset result
+0.44historical dark-ui result
+0.54best weighting found in that search, fitted to the same 26 pairs

Historical per-dimension correlations

  • layout
    +0.33
  • typography
    +0.31
  • content
    +0.23
  • color
    +0.03
  • spacing
    −0.13

Layout and typography had the strongest positive correlations on this dataset. The weight search used the same 26 pairs, not an independent held-out evaluation. Use your own labeled pairs in calibration/ to evaluate your UI.

Where it can mislead you

Motion energy. Similar motion energy can hide different paths or timing. Static and unreliable captures are inconclusive. Read motion_status.
Color ranking. The old dark-library dataset showed little color-ranking signal. Palette errors can still matter in a single-theme UI.
Typography ≠ font. Amount and scale, not font family or weight.
One-shot animations. An animation that ends before sampling can be missed. Inspect the frames and status.
Ignore chrome. Status bars and OS chrome: pass ignoreRegions as [x, y, w, h] in original reference pixels.
Capture URLs. capture_frames accepts http and https only. Set DESIGN_COMPARE_ALLOWED_ROOTS to restrict local file reads and capture output paths, not network access.

FAQ

Frequently asked questions

What problem does this solve?

It adds repeatable measurements and diagnostic images to visual review. The output helps identify differences; it does not replace visual or behavioral checks.

What do I feed it?

Two local images, a reference and a candidate. widget mode for a tight crop, screen mode for a full view. Optional ignoreRegions in original reference pixels. For motion, two frame sequences from capture_frames using http or https URLs. For recorded video, extract a bounded local window with extract_video_frames, then review timestamps, sampling completeness and matching action before comparison.

How is the score computed?

Applicable numeric dimensions contribute to the weighted geometric mean, with scores clamped to 1–100 and weights renormalized. Inspect failed and non-applicable dimensions before interpreting the aggregate. If no dimensions have numeric scores, the aggregate is 0, not a usable comparison.

Can I ignore a status bar?

Yes. Pass ignoreRegions as [x, y, w, h] boxes in original reference pixels. Finding boxes come back in the canonical 768 canvas. Use the returned canvas mapping to translate.

Is the number accurate?

The published human-label correlations describe an earlier pipeline on 26 pairs. They are not a current accuracy estimate for the reliability changes. Synthetic regression checks test specific cases, not broad perceptual accuracy.

Can I calibrate it for my own UI?

Yes. Drop (reference, candidate, your 0–100 label) pairs into calibration/ and run scripts/calibrate.py.

Why TypeScript and Python?

TypeScript speaks MCP and drives headless-Chrome capture; a resident Python worker does the vision, so the heavy imports load once, not per call.

Is it safe to run?

It reads local paths you pass. Set DESIGN_COMPARE_ALLOWED_ROOTS to restrict local file reads and capture output paths. These are path restrictions, not process or network isolation. capture_frames accepts http and https; use trusted URLs. Recording extraction runs local FFmpeg, so use trusted recordings and keep the decoder updated.

What's the license?

MIT. Source, design notes, and the calibration workflow are on GitHub.

Compare the details. Review the evidence.

Give your agent measurements and diagnostics to inspect.

Get it on GitHub ↗ Install steps