Slightly off topic: One day in the late 90s or early 2000s, I wanted to transfer the contents of my Atari 8-bit disks to my PC. I did know about SIO2PC, a cable that converts between Atari's SIO port and the PC's serial port and a piece of software to go with it. But I didn't have the electronics skills to build one (even though it is actually very easy).
So I ended up writing a Basic program on the Atari to read data from the disk sector by sector and paint it on the screen (with the large 4-color pixels of graphics mode 3). The Atari was connected to the TV card of my PC and a Delphi program I wrote was running on the PC that kept taking screen shots and trying to decode the data from there. I quickly learned that empty sectors threw off my pixel position calibration so I added a mask pattern and a checksum. The sector address was also included. With that, I was able to transfer all my disk contents to my PC. To this day I consider it my greatest engineering achievement :)
Some ten years later, I went on to build an SIO2PC program called AspeQt. A more up-to-date community fork called RespeQt is still the most popular tool in that category used by the community. It even has its own subforum on AtariAge[1].
Reminds me of how the first generation iPod firmware and keys were reverse engineered. They blinked out bit patterns on the LCD backlight and used a camera to read the data in! https://mastodon.social/@bagder/111538350617290554
Citation needed, but I think I read that hackers dumped the firmware of Canon cameras by blinking the single red LED on the camera (and foregoing the main graphical LCD). See the CHDK and Magic Lantern projects.
> So is the rest all content? Not quite! There is also error correction, which is used to make sure that the QR code can still be read even if some parts are damaged, blurry, or missing. The error correction is generated by some fancy math, and we don't care about it here for the purpose of reading it by hand.
This should be explained better... I mean sure, the math can be hard.. but is the error correction appended at the end? After every byte? In the middle? Can we still read a qr code by hand if it has error correction (assuming it's not damaged, and we can skip the math part?)?
Barcodes use the last digit as a checksum. The checksum is some adding up the numbers and multiplying or something, then % 10 (only 0-9 possible values). As you can see with this math, the error checking has a 1/10 of a chance of coincidentally showing a correct checksum in cases where the data is incorrect. Either way, that last digit on barcodes is insignificant to the actual product's unique barcode ID.
Fun story: The barcode was ripped and the checksum # was frayed off of an item I was returning to Home Depot. I frantically tried to do the math quicker that the checker could manually look it up. But they won and I'll never forget it.
A few common barcodes including UPC work this way but mostly this is the exception. The 2D ones being discussed here like QR and Aztec use actual error correcting codes, so none of them. In the linear space Codabar (libraries) is a widely used example that doesn’t. GS-128 doesn’t, etc.
When I worked at a grocery store in the 2000s, you didn't need the checksum digit when keying in a barcode by hand. In fact, manually entering the checksum digit would result in a "product not found" error.
So I wonder if the checker had to manually look anything up, assuming it was only the checksum digit that was frayed off.
Perhaps some POS systems do require the checksum digit for manually-entered barcodes, though.
the data in the code is encoded as a bitstream using possibly different alphabets, byte is just one of them.
Then, depending on the size and error correction level (not covered in the article where this is decoded from) the bitstream is split up into blocks, ecc data is calculated for both, and then the blocks are interleaved.
So this method will fail for any qr version above 5 for any level of ecc, and 3-5 with high levels of ecc.
An Artisanal QR Code [1] shows how to compute the checksum by hand using arithmetic similar to a long division. There are actually two slightly different codes (Reed-Solomon and BCH) for different pieces of information.
Great explanation of how to read QR codes, thanks for the write up! Here's a nice one about 1D codes and how to decode your average UPC codes you find on products in (American) supermarkets:
I tried reading them manually about ten years ago, but gave up because of the mask. The mask pattern needs to be applied (xor) before you can even tell as much as the data type...
Does anyone know how much it actually helps QR code readers to have this mask pattern applied?
Last month, I tried to find out what the optimal error correction setting is (answer: none, if the matrix isn't damaged, like in my use-case where it's shown on a screen), which wasn't easy to find. I didn't find anyone who actually went out into the real world and tried out different settings and different scanning implementations to see what the effect is of choosing a good mask or choosing a good error correction value. In the end, a theoretical answer was enough for my application because I can assume the QRs to be pristine, but in a perfect world we also wouldn't need a mask setting. I then tried to do some research of my own by reading a small QR off my screen with different error correction levels and an intentionally shaking camera to simulate read issues when someone points a smartphone, but found no significant difference between any of the settings. With there being at least four alignment markers, a huge quiet area, and a "timing pattern", does it really matter if there are white blobs due to not having a mask?
(The secondary reason why I gave up trying to read QRs visually is because I noticed they always have the URL written below as fallback anyway. Since then, I did see a few times where there was no fallback, but it's very rare)
The error correction is funny. If you don't use it, your message takes up less space, and the QR is smaller. If you have a fixed space to fit it in (e.g. printing to a sticker), that means your pixels can be physically bigger. For reliable reading, I found that physically bigger pixels helped more than the error correction itself.
Depending on the length of your message, sometimes you can turn up the error correction level without impacting the final QR dimensions, and in those cases it's a win/win.
Physically bigger pixels help a lot indeed. The encoding mode also helps make pixels bigger.
This is very convenient when you control the QR reader and need to represent long numeric identifiers like UUIDs.
For example:
9728983f-7d7d-4189-b624-f92781e36650 (lowercase UUID):
=> length=36, 15 pixels between markers
JWM9GFVXFN0RKDH4Z4KR3RV6A0 (base32 UUID):
=> length=26, 11 pixels between markers
9728983F-7D7D-4189-B624-F92781E36650 (uppercase UUID):
=> length=36, 11 pixels between markers
200924207194334734815443970355691218512 (decimal UUID):
=> length=39, 7 pixels between markers
The uppercase UUID has bigger pixels because it used a different encoding, and gets the same results as the shorter base32 uuid.
The decimal UUID is a longer string, but results in much bigger pixels because it can use numeric encoding.
I have a QR code base attendance tracker [1], where attendees show the code [2] on their phones (glares, etc.), in bad lighting conditions, etc. Bigger pixels means scanning stays quick. Same with parcel tracking [3] where field agents might need to scan QR codes in barely-lit hallways, etc.
The error correction isn't meant for folks like us who think about things like "how can I make this message smaller to increase the readability of the QR code", it's for the people that take their entire tracking parameter-infected URL and dump it into a version 25 (117x117) code and plaster it onto a huge sign out front where a bird promptly shits on it and a teenager slaps a sticker on it and somehow the data remains.
I also have the impression that a common use of error correction is the "QR Code with a logo in the middle" style. With an appropriate level of error correction, you can carve an empty patch in the middle of the logo and slap a logo/icon there, and the QR code remains readable.
At least this is the way I did it years ago when I thought it would be fancy to have my contact data as a QR Code on my visit card.
Last year I did some experimentation along these lines and I also found that no error correction was best, at least when 3d printing QR codes. I was trying to find how small I could print it and still have it be detectable. Like the sibling comment, I suspect it is because of individually larger pixels.
The problem with that, as I understand it, is that your 3d print (or any print or screen) cannot get damaged at all. The pixels being bigger makes it easier to scan ...when it's fully in view and fully intact. Any pixel fails, and you can never recover the original data (besides some guesses, e.g. if the protocol identifier gets damaged a la htwps://)
Error correction doesn't seem to be for easier scanning, but for errors that might occur
(... and stupid logo overlays)
but this information is annoyingly hard to find. Hence my question about the mask xor overlay: does that actually help? Did anyone try it, or did it just sound like a good idea?
Error correction is a fascinating topic. Shannon discovered in the mid-20th century what the theoretical limit was, yet nobody knew how to achieve it. When we suddenly got close in the 1990s with the patent of turbocodes the community reacted with disbelief!
Of course, that's what's enabled things like fast and reliable mobile networks.
Readit News
So I ended up writing a Basic program on the Atari to read data from the disk sector by sector and paint it on the screen (with the large 4-color pixels of graphics mode 3). The Atari was connected to the TV card of my PC and a Delphi program I wrote was running on the PC that kept taking screen shots and trying to decode the data from there. I quickly learned that empty sectors threw off my pixel position calibration so I added a mask pattern and a checksum. The sector address was also included. With that, I was able to transfer all my disk contents to my PC. To this day I consider it my greatest engineering achievement :)
Some ten years later, I went on to build an SIO2PC program called AspeQt. A more up-to-date community fork called RespeQt is still the most popular tool in that category used by the community. It even has its own subforum on AtariAge[1].
[1] https://forums.atariage.com/forum/184-respeqt-sio2pc-softwar...
This should be explained better... I mean sure, the math can be hard.. but is the error correction appended at the end? After every byte? In the middle? Can we still read a qr code by hand if it has error correction (assuming it's not damaged, and we can skip the math part?)?
Fun story: The barcode was ripped and the checksum # was frayed off of an item I was returning to Home Depot. I frantically tried to do the math quicker that the checker could manually look it up. But they won and I'll never forget it.
https://www.gs1.org/services/how-calculate-check-digit-manua...
https://www.simplybarcodes.com/barcode_check_digit_calculato...
So I wonder if the checker had to manually look anything up, assuming it was only the checksum digit that was frayed off.
Perhaps some POS systems do require the checksum digit for manually-entered barcodes, though.
https://www.ecb.torontomu.ca/~elf/abacus/feynman.html
Then, depending on the size and error correction level (not covered in the article where this is decoded from) the bitstream is split up into blocks, ecc data is calculated for both, and then the blocks are interleaved.
So this method will fail for any qr version above 5 for any level of ecc, and 3-5 with high levels of ecc.
[1] https://www.quaxio.com/an_artisanal_qr_code.html
Decoding small QR codes by hand (2012) - https://news.ycombinator.com/item?id=36173441 - Jun 2023 (69 comments)
How a QR code works - https://news.ycombinator.com/item?id=32837565 - Sep 2022 (114 comments)
Creating a QR Code step by step - https://news.ycombinator.com/item?id=24119124 - Aug 2020 (41 comments)
Creating a QR Code step by step - https://news.ycombinator.com/item?id=18360847 - Nov 2018 (34 comments)
https://scanbot.io/blog/how-do-barcodes-work/
Does anyone know how much it actually helps QR code readers to have this mask pattern applied?
Last month, I tried to find out what the optimal error correction setting is (answer: none, if the matrix isn't damaged, like in my use-case where it's shown on a screen), which wasn't easy to find. I didn't find anyone who actually went out into the real world and tried out different settings and different scanning implementations to see what the effect is of choosing a good mask or choosing a good error correction value. In the end, a theoretical answer was enough for my application because I can assume the QRs to be pristine, but in a perfect world we also wouldn't need a mask setting. I then tried to do some research of my own by reading a small QR off my screen with different error correction levels and an intentionally shaking camera to simulate read issues when someone points a smartphone, but found no significant difference between any of the settings. With there being at least four alignment markers, a huge quiet area, and a "timing pattern", does it really matter if there are white blobs due to not having a mask?
(The secondary reason why I gave up trying to read QRs visually is because I noticed they always have the URL written below as fallback anyway. Since then, I did see a few times where there was no fallback, but it's very rare)
Depending on the length of your message, sometimes you can turn up the error correction level without impacting the final QR dimensions, and in those cases it's a win/win.
By making the pixels of the code bigger, the camera can be a bit blurry to get the code big enough, and then it'll read.
This is very convenient when you control the QR reader and need to represent long numeric identifiers like UUIDs.
For example:
The uppercase UUID has bigger pixels because it used a different encoding, and gets the same results as the shorter base32 uuid.The decimal UUID is a longer string, but results in much bigger pixels because it can use numeric encoding.
I have a QR code base attendance tracker [1], where attendees show the code [2] on their phones (glares, etc.), in bad lighting conditions, etc. Bigger pixels means scanning stays quick. Same with parcel tracking [3] where field agents might need to scan QR codes in barely-lit hallways, etc.
[1] https://workspace.google.com/marketplace/app/qr_code_pass_fo...
[2] https://share.darkaa.com/!qntvtzNPWJ
[3] https://admin.trak.codes/
At least this is the way I did it years ago when I thought it would be fancy to have my contact data as a QR Code on my visit card.
Error correction doesn't seem to be for easier scanning, but for errors that might occur
(... and stupid logo overlays)
but this information is annoyingly hard to find. Hence my question about the mask xor overlay: does that actually help? Did anyone try it, or did it just sound like a good idea?
My interactive web page on creating a QR code step by step (so basically the inverse process): https://www.nayuki.io/page/creating-a-qr-code-step-by-step
[0] https://docs.beaconstac.com/en/articles/6018654-what-is-erro...
Of course, that's what's enabled things like fast and reliable mobile networks.