Repository navigation
Conversation
Adds otsuThresholdFromHistogram, which evaluates the same split criterion with cumulative sums, so it can be used on histograms of small regions. determineOtsuThreshold uses it; results are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The locator binarizes the image with one Otsu threshold. Large uniform areas, such as a white background or the empty corners of a rotated photo, pull that threshold above the barcode's light bars, and the barcode is not located. locator.thresholding: 'patch' computes a threshold per patch from the patch and half a patch around it; patches without contrast stay empty. The default remains 'global'. Refs ericblade#650 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
There is no change log for this pull request yet. |
There was a problem hiding this comment.
🟢 Approval recommended
The review surfaced no unresolved defects, with no outstanding objective issues left to address on the change.
No open issues 🎉
What changed in this PR
This PR adds an opt-in per-patch binarization mode to the locator to address adaptive thresholding under uneven lighting (refs #650). The locator previously used a single global Otsu threshold, which fails when large uniform areas (e.g., white borders or the empty corners of a rotated photo) skew the threshold past the barcode's light bars. A new locator.thresholding: 'global' | 'patch' option computes an Otsu threshold per patch (over the patch plus a half-patch margin) and leaves low-contrast patches empty. The default remains 'global', so existing behavior is unchanged.
Changes:
- Extracted Otsu logic into reusable
otsuThresholdFromHistogramincv_utils.js(via a running-sum formulation) and reimplementeddetermineOtsuThresholdon top of it. - Added
binarizePerPatch(pluspatchHistogram/hasContrasthelpers and aMIN_PATCH_CONTRASTheuristic) to the locator, selected by the newthresholding: 'patch'config. - Wired the new
thresholdingoption through all config defaults, the type definitions, README/docs, and added unit + integration tests (with newwhite_borderfixtures).
| File | Description |
|---|---|
src/common/cv_utils.js |
Extracts otsuThresholdFromHistogram and rebuilds determineOtsuThreshold on it using cumulative sums. |
src/locator/barcode_locator.js |
Adds per-patch binarization path, histogram/contrast helpers, and branches binarizeImage on thresholding. |
src/config/config.dev.ts, config.node.ts, config.prod.ts |
Add thresholding: 'global' default to the locator config. |
type-definitions/quagga.d.ts |
Adds the thresholding?: 'global' | 'patch' locator option with JSDoc default. |
src/common/test/cv_utils.spec.js |
Adds a unit test verifying otsuThresholdFromHistogram matches the previous per-split computation. |
test/integration/decoders/white_border.spec.ts |
New integration test covering 'patch' thresholding on white-border fixtures. |
cypress/e2e/integration.cy.ts |
Registers the new white-border spec for browser runs. |
README.md, docs/reference/configuration.md |
Document the new thresholding option and when to use 'patch'. |
Review effort: Balanced
|
So.. my understanding of this, in real basic terms, then -- with 'global' set, everything remains exactly as it was (other than the algorithm with the histogram, which looks like it's probably a "this is faster by theory" sort of thing, but as fast as they are and as much as they are called, they don't come out to any different really... ) ... and with patch set, basically, it treats everything that isn't the patch it's examining and half a patch size around it as if it's effectively border? If that's a basic layman's understanding of how it works, I wonder if something like the pre-processor that I proposed is something that should be plugged into that feature as well -- #626 ... It's something I haven't merged because I feel like it needs more thought put into it than what a LLM was capable of giving us a year ago (and probably now, too) without significant direction. And that would lead into questions like this -- if we had a pre-processing pipeline for the image as a whole, would we also want a pre-processing pipeline for patches as well? of course that leads into the ideas i wanted to build for a 2.0 version where the image acquisition, the location, the decoding, are all completely separate pieces of a pipeline, and you just stick them together in whichever fashion you need them, like Unix pipes. But I'm not sure if/when I'll ever get to finishing that idea, so maybe adding a pipeline for that into the existing architecture is something we should do. As well, my understanding of the description, is that with this enabled -- location becomes better, but decoding becomes worse? If you could provide samples where location and decoding improve, or specifically document which ones in the existing suite improve, that would be helpful. My gut feeling is that something like that might be a reason to include a tunable parameter for how far out this affects. I think I honestly need to see the insides of what's going on (ie what the locator is actually seeing) before I have any concrete ideas, though. |
|
That is probably the most ADHD comment I have written in a long time, sorry for all the divergences there. DId I make sense? |
|
Yes, you made sense! I’ve clarified the documentation and added a comparison page so you can see the locator’s binary images, candidate boxes, and decoded values side by side. Your understanding of global is right. For patch, pixels outside the patch’s neighborhood simply don’t contribute to its threshold; they aren’t treated as white border. My description of the decoding results was confusing. The change produces more correct reads and more incorrect reads overall. It changes the locator’s binary image and resulting boxes; the decoder still samples the grayscale input and binarizes its scanlines separately. There are concrete end-to-end improvements: with halfSample: true, the eight white-border fixtures go from 2/8 correct reads to 8/8, with no incorrect values in this check. All five UPC fixtures and ean-001.jpg gain the correct read. The tradeoff is real, though. Without half sampling, patch mode gets only 4/8, produces a wrong value on ean-001.jpg, and loses the correct read on ean-006.jpg. I’ve documented those cases too. The comparison page is at docs/examples/locator_thresholding.html; build with npm run build:dev and serve the repository root. A configurable neighborhood could be worth investigating, but I don’t yet have evidence that it fixes the regressions. |
Refs #650
The locator binarizes the whole image with one Otsu threshold. A large uniform area becomes one of Otsu's two classes on its own and moves the threshold away from the barcode. On
test/fixtures/upc/image-001.jpg, a white border the size of a 20° rotation canvas (no rotation, no resampling) moves the threshold from 99 to 199. At 199, every pixel of the photo, including the barcode's light bars, is dark, and nothing is located. With a mid-gray border instead, the upc fixtures rotated by 20° and 45° decode 10 of 10.This adds
locator.thresholding: 'global' | 'patch'.'patch'computes an Otsu threshold for each locator patch from the patch and half a patch around it, and leaves patches whose gray values span fewer than 40 levels (ignoring the darkest and lightest 2%) empty. 20 and 60 instead of 40 gave nearly the same results on the fixtures. The default stays'global': with'patch'as the default, 17 existing fixture tests fail across 11 formats (among them code_39, code_93, ean and upc_e).The Otsu computation moves into
cv_utils.otsuThresholdFromHistogram, whichdetermineOtsuThresholdnow uses. A test compares it with the previous per-split computation on random histograms, and all existing test outcomes are unchanged. The new fixtures intest/fixtures/white_border/(8 files, 0.7 MB) are the upc and ean fixtures pasted onto a white 766×670 canvas.npm run test:node: 480 passing, 58 pending on master; 489 passing, 58 pending with this change. With halfSample: true, the white-border fixtures decode 2 of 8 with'global'and 8 of 8 with'patch'. With halfSample: false the counts are 2 and 4. The new spec is also registered incypress/e2e/integration.cy.ts, where it passes. The one failure in that file, eanimage-005.jpgwith halfSample: false, also fails on master.'patch'locates more barcodes, but the readers also misread more of them: photos with a wrong read almost double.Photo numbers come from Chrome 155 (
decodeSingle,inputStream.size: 800,halfSample: true,patchSize: 'medium', readers ean/ean_8/upc/upc_e,multiple: true) on photos from the public BarBeR dataset, mostly EAN-13. The 90° and 30° sets are the same 120 photos rotated in a canvas around their center, onto a white canvas large enough for the whole photo. The 40° set is 100 other BarBeR photos rotated 40° within their original size, with white corners. A case counts as decoded when the expected value is among the results.Photos
120 photos: 05102009104.jpg 05102009121.jpg 05102009144.jpg 05102009155.jpg 06102009253.jpg 1040000062669_1.jpg 2007002006212-01_N95.jpg 2007002124862-01_N95.jpg 20140607_201249.jpg 20140607_201315.jpg 20140607_201347.jpg 20140607_201548.jpg 20140607_201705.jpg 20140607_201745.jpg 20140607_201913.jpg 20140607_202016.jpg 20140607_202442.jpg 20140607_202504.jpg 20140607_202538.jpg 20140607_202559.jpg 20140607_202623.jpg 20140607_202704.jpg 20140607_202817.jpg 20140607_202855.jpg 20140607_203032.jpg 20140607_203052.jpg 20140607_203123.jpg 20140607_203407.jpg 20140607_203428.jpg 20140607_203438.jpg 20140607_203449.jpg 20140607_203547.jpg 20140607_203551.jpg 20140607_203621.jpg 20140607_203758.jpg 20140607_204009.jpg 20140607_204014.jpg 20140607_204127.jpg 20140607_204147.jpg 20140607_204155.jpg 20140607_204218.jpg 20140607_204342.jpg 20140607_204353.jpg 20140607_204549.jpg 20140607_204600.jpg 20140607_204604.jpg 20140607_204608.jpg 20140607_204618.jpg 20140607_204659.jpg 20200916_212940.jpg 20200916_213006.jpg 2689120051895.jpg 2952211005443_1.jpg 2952347007267.jpg 3073781000139.jpg 3083680012294-01_N95.jpg 3121040014394-01_N95.jpg 3165950077018-01_N95.jpg 3171920011272_2.jpg 3181731138013-01_N95.jpg 3181731138013-02_N95.jpg 3181731140009-02_N95.jpg 3222476758729_2.jpg 3250392605414.jpg 3256223374719.jpg 3256224118282.jpg 3256224223597.jpg 3256227447945.jpg 3263670237719.jpg 3263858602612.jpg 3265980837474.jpg 3270160603718.jpg 3275461001775_1.jpg 3343929291218.jpg 3347430019309.jpg 3347430022439.jpg 3378639920127.jpg 3384480028721.jpg 3389090003342_1.jpg 3389682001244_2.jpg 3428201435396.jpg 3456700083749_1.jpg 3492390018241_2.jpg 3523320200136_2.jpg 3524130000350.jpg 3532820000092_2.jpg 3574660026948-01_N95.jpg 3580281813034_2.jpg 3596710353224.jpg 3598580630047-01_N95.jpg 3600520825700-01_N95.jpg 3608580921639.jpg 3760046253835_2.jpg 4000007037234-01_N95.jpg 4000196015112-01_N95.jpg 4000269101247-01_N95.jpg 4000400001955-01_N95.jpg 4000400002150-01_N95.jpg 4000400002945-01_N95.jpg 4000400002945-02_N95.jpg 4000400037947-01_N95.jpg 4000400070340-01_N95.jpg 4000400181121-01_N95.jpg 4000508045752-02_N95.jpg 4000576484736-01_N95.jpg 4000582500093-01_N95.jpg 4000869042056-01_N95.jpg 4000869060098-01_N95.jpg 4000953398915-01_N95.jpg 4001133962810-01_N95.jpg 4001419009611-01_N95.jpg 4001432078533-01_N95.jpg 4001438012135-01_N95.jpg 4001686365816.jpg 4001686405925.jpg 4001686407066.jpg 4001702025304-01_N95.jpg 4001724819400.jpg 4001845521107-01_N95.jpg 4001845525204-01_N95.jpg
40° set, source photos: 06102009240.jpg IMAG0548.jpg 8002270766633.jpg ProductBarcode102_0.jpg 8000700000005-01_N95.jpg 3533631905576_1.jpg 20140607_203905.jpg [P]ISBN_18_0041.jpg PICT0009.JPG IMAG0404.jpg 5029053009834-01.jpg ProductBarcode079.jpg 4008695730185-01_N95.jpg 3264057554320_1.jpg 20140607_203847.jpg EAN13_09_0100.jpg PICT00fghgh02.JPG IMAG0400.jpg 3272770014205.jpg ProductBarcode037.jpg 4001513000613-02_N95.jpg 3435020008618_2.jpg 20140607_202511.jpg [S]ISBN_02_0081.jpg PICT001gfghfg7.JPG IMAG0504.jpg 5410683018059-01.jpg ProductBarcode111.jpg 4003694852128-01_N95.jpg 4311501625217_8.jpg 20140607_202234.jpg [S]ISBN_02_0015.jpg Foto(791).jpg IMAG0525.jpg 8009205601045.jpg ProductBarcode389_0.jpg 4000290008195-01_N95.jpg 9400563452017_2.jpg 20140607_202508.jpg Excise_stamp_0054.jpg 05102009207.jpg IMAG0551.jpg 5000354800887.jpg ProductBarcode032_0.jpg 4017026055484-01_N95.jpg 3390122203951_1.jpg 20140607_203012.jpg [P]ISBN_18_0095.jpg PICT00dfs31.JPG IMAG0490.jpg 3254561041607.jpg ProductBarcode042.jpg 7300400113465-01_N95.jpg 5021047102439_2.jpg 20140607_204432.jpg [S]ISBN_02_0017.jpg 05102009114.jpg IMAG0433.jpg 9006900011529-01.jpg ProductBarcode024_0.jpg 4306517027032-01_N95.jpg 3215470518352_2.jpg 20140607_203321.jpg [P]ISBN_18_0091.jpg 06102009246.jpg IMAG0491.jpg 5000171055330.jpg ProductBarcode108.jpg 4002644601175-01_N95.jpg 3700175002591_2.jpg 20140607_203058.jpg [P]ISBN_18_0084.jpg PICTwa0015.JPG IMAG0398.jpg 3175681137776.jpg ProductBarcode054_0.jpg 2007001572312-02_N95.jpg 8000500285572_2.jpg 20140607_201735.jpg [S]ISBN_02_0047.jpg 05102009106.jpg IMAG0469.jpg 7613379110190.jpg ProductBarcode088.jpg 4053800081041-01_N95.jpg 7613033968051_1.jpg 20140607_201330.jpg EAN13_09_0074.jpg 06102009242.jpg IMAG0536.jpg 8595139738815.jpg ProductBarcode106.jpg 4210201219767-01_N95.jpg 3417549311288_1.jpg 20140607_203404.jpg [P]ISBN_18_0010.jpg PICT0017.JPG IMAG0411.jpg 8801353003807.jpg ProductBarcode212.jpg
On a private retail benchmark that I can't share (60 photos per group, same config), decoded photos went from 34.9% to 56.6%. For uneven lighting, the subject of #650, they went from 37% to 60%. Photos with a wrong read went from 36 to 67, and the 9 very large inputs went from 6 to 3 decoded. The median runtime went from 14 to 15 ms.