I see that bounty at the bottom, so tossing away my chances here, but this visualization is just asking to be mapped onto a Hilbert Curve. [0] When you "stripe" the data like this, points that are sorted close together could end up pretty far apart, since a distance in the Y axis skips an entire row of data as you move down, rather than a distance in the X axis which is 1-to-1 with the source data.
If you map it onto a hilbert curve, the X and Y axis mean nothing, but visually points that are close together in the sorted list, will be visually close together in the output image.
Since the first part of an ISBN is the country, then the second part is the publisher, and the third part is the title, with a check sum at the end, I would remove the checksum and sort them each as a big number. (no hyphens)
You should end up with "islands", where you see big areas covered by big publishing countries, with these "islands" having bright spots for the publisher codes.
Bonus points for labeling these areas!
I set up something a while ago [1] for an interview that does this with weather data. It makes the seasons really obvious since they're all grouped together.
What property makes the Hilbert curve desirable compared to, say, a snake pattern, with which neighbouring ISBNs are also neighbours in the visualisation?
The worry I have with Hilbert curves is that they make the result look like there are distinct "squares" of data [0] when really this is just an artifact of how Hilbert curves work. In that sense, the current visualization is more useful, because it's straightforward to identify the location of each country in it.
In a snake pattern, the neighbouring pixels on the left and right are related, but the ones above and below have skipped a whole row.
And yeah that’s true! you end up with squares with Hilbert curves. But those squares are all « related » data. Then those squares are related to the squares near it. Zoom out more and that grouping of squares is related to the neighbouring macro-squares etc etc.
Basically the square shape is a positive. Kind of like how charting the derivative lets you see how random/related information is, grouping into these squares gives you a visualization of pattern-ness, rather than any specific measurement.
> What property makes the Hilbert curve desirable compared to, say, a snake pattern, with which neighbouring ISBNs are also neighbours in the visualisation?
2D neighbourhood is better than 1D one
> The worry I have with Hilbert curves is that they make the result look like there are distinct "squares" of data
that's the point, tho? instead of distinct lines of taken ISBNs in a row, you get distinct squares if taken ISBNs in a row - much more noticeable
The thing is, ISBNs aren't hierarchical --- they are bought in blocks (or even individually at an exorbitant markup, says the guy who bought one to reprint a single book), so this doesn't show anything really interesting/useful.
A visualization using LoC or even Dewey Decimal would be far more useful, esp. if it also linked to public domain and copyright-free repositories/lists, say an interactive and visual version of John Mark Ockerbloom's:
ISBN's are hierarchical, what do you mean? Like Gaul, ISBNs are divided into multiple parts, where one part is for the language, another is for the publisher, and the last is for the title. The last part is a checksum. https://en.wikipedia.org/wiki/ISBN#Overview
Yes, but this internal hierarchy for an issued number doesn't tell anything beyond those facts about a specific edition of a specific text.
One can't use ISBNs alone to create a hierarchical listing of texts which is useful for anything beyond browsing by language/publisher/order in which the ISBN was generated.
A visual and interactive representation of books by LoC or some other cataloging system would actually be useful.
It only sort of shows that. ISBNs are issued by edition, not title, so many books would have more than one. And books published before 1970 or so might not be represented at all if they have no recent edition.
They can't even have a tiny fraction of the world's books. Each edition of the book gets a new ISBN... if a book is released as a paperback, hardback, kindle edition, pdf, and epub then there are supposed to be five ISBNs.
The vast, vast majority have only been released as dead-tree versions. They have none of those. The books they scan may have an ISBN, but the scans do not have them. Like all Project Gutenberg books, their books have no ISBNs at all. From a strict point of view, they've released new editions of these books.
One thing it shows is how ISBNs are allocated much faster than they are used, judging by the amount of black pixels.
The image contains 1000*800 pixels at 2500 ISBNs per pixel, so it's visualizing 2e9 ISBNs. ISBN-13 contains 12 digits plus one check digit, so we might have expected the image to be 500 times bigger/denser than the current image. The fact that it's at its current size suggests that only ISBNs with 978 and 979 prefixes are included, and since the bottom half is more sparse, that probably corresponds to the new 979 range.
I thought it was my color blindness that made me not able to distinguish between the red and green pixels as described (i only see red and black ones), but even with a browser extension that counters color blindness i can't distinguish more colors. Is this just me, or is the graph weird?
Fwiw (not color-blind) I can see red, green and black pixels. The graph doesn't look weird to the naked eye.
Find the interactive visualiser by scrolling down, and switch it to "Files in Anna's Archive [md5]". This will highlight the location of the green pixels in grey.
element {
max-width: 100%;
margin: 0 auto;
filter: hue-rotate(-90deg);
}
Usually I use "filter: saturate(100);", but that didn't really work well for this image. You might have to adjust the rotation degree though, -90 worked best for me.
Anna's archive is one of the wonders of the world. If we almost destroyed our species but Anna's archive endured, there would be hope for a relatively expedient reconstruction.
It appears that the IP of the server is blocked in the EU. I get this from my ISP (Ziggo, in the Netherlands):
Deze website is geblokkeerd
Europese sancties
De Raad van Europa heeft besloten dat de websites van RT (voorheen Russia Today) en Sputnik News niet meer mogen worden doorgegeven. De website die je probeert te bezoeken, valt onder deze Europese sanctie.
VodafoneZiggo is verplicht de sanctie uit te voeren en heeft de website geblokkeerd.
"This server couldn't prove that it's annas-archive.org; its security certificate is from *.hs.llnwd.net. This may be caused by a misconfiguration or an attacker intercepting your connection."
Yes. A DNS request for annas-archive.org to my ISP (EE in the UK) returns an address for www.ukispcourtorders.co.uk, which also gives a security warning. If I click through the warning on either site I get an HTTP 400 error.
According to Wikipedia, www.ukispcourtorders.co.uk used to list the blocked domains and the court orders responsible.
Kind of hard to tell what corresponds to what in these graphs, maybe if someone could point out Bookland (i.e. 978), it would be a bit easier to orient oneself?
annoying non-answer to my question. i already know all about anna's archive. i'm asking if a person can download these isbns and use them to make data visualizations without fear of breaking a law? https://software.annas-archive.li/AnnaArchivist/annas-archiv...
Readit News
If you map it onto a hilbert curve, the X and Y axis mean nothing, but visually points that are close together in the sorted list, will be visually close together in the output image.
Since the first part of an ISBN is the country, then the second part is the publisher, and the third part is the title, with a check sum at the end, I would remove the checksum and sort them each as a big number. (no hyphens)
You should end up with "islands", where you see big areas covered by big publishing countries, with these "islands" having bright spots for the publisher codes.
Bonus points for labeling these areas!
I set up something a while ago [1] for an interview that does this with weather data. It makes the seasons really obvious since they're all grouped together.
[0] https://en.wikipedia.org/wiki/Hilbert_curve
[1] https://graypegg.com/hilbert (https://github.com/graypegg/hilbertcurveplayground code if anyone wants to go for the prize using this! Please at least mention me if you decide to reuse this code, but I can't stop ya lol)
[0] https://github.com/jakubcerveny/gilbert
[1] https://jakubcerveny.github.io/gilbert/demo/
The worry I have with Hilbert curves is that they make the result look like there are distinct "squares" of data [0] when really this is just an artifact of how Hilbert curves work. In that sense, the current visualization is more useful, because it's straightforward to identify the location of each country in it.
[0] https://raw.githubusercontent.com/jakubcerveny/gilbert/maste...
And yeah that’s true! you end up with squares with Hilbert curves. But those squares are all « related » data. Then those squares are related to the squares near it. Zoom out more and that grouping of squares is related to the neighbouring macro-squares etc etc.
Basically the square shape is a positive. Kind of like how charting the derivative lets you see how random/related information is, grouping into these squares gives you a visualization of pattern-ness, rather than any specific measurement.
2D neighbourhood is better than 1D one
> The worry I have with Hilbert curves is that they make the result look like there are distinct "squares" of data
that's the point, tho? instead of distinct lines of taken ISBNs in a row, you get distinct squares if taken ISBNs in a row - much more noticeable
A visualization using LoC or even Dewey Decimal would be far more useful, esp. if it also linked to public domain and copyright-free repositories/lists, say an interactive and visual version of John Mark Ockerbloom's:
https://onlinebooks.library.upenn.edu/
One can't use ISBNs alone to create a hierarchical listing of texts which is useful for anything beyond browsing by language/publisher/order in which the ISBN was generated.
A visual and interactive representation of books by LoC or some other cataloging system would actually be useful.
The vast, vast majority have only been released as dead-tree versions. They have none of those. The books they scan may have an ISBN, but the scans do not have them. Like all Project Gutenberg books, their books have no ISBNs at all. From a strict point of view, they've released new editions of these books.
The image contains 1000*800 pixels at 2500 ISBNs per pixel, so it's visualizing 2e9 ISBNs. ISBN-13 contains 12 digits plus one check digit, so we might have expected the image to be 500 times bigger/denser than the current image. The fact that it's at its current size suggests that only ISBNs with 978 and 979 prefixes are included, and since the bottom half is more sparse, that probably corresponds to the new 979 range.
Find the interactive visualiser by scrolling down, and switch it to "Files in Anna's Archive [md5]". This will highlight the location of the green pixels in grey.
- Right-click the image and select "Inspect".
- Add a new CSS hue-rotate filter to the element:
Usually I use "filter: saturate(100);", but that didn't really work well for this image. You might have to adjust the rotation degree though, -90 worked best for me.Can you change the green channel to blue to better view it?
If self-destruction is a necessary premise here, is that really a good thing?
Deze website is geblokkeerd
Europese sancties
De Raad van Europa heeft besloten dat de websites van RT (voorheen Russia Today) en Sputnik News niet meer mogen worden doorgegeven. De website die je probeert te bezoeken, valt onder deze Europese sanctie.
VodafoneZiggo is verplicht de sanctie uit te voeren en heeft de website geblokkeerd.
https://web.archive.org/web/20250106112552/https://annas-arc...
"This server couldn't prove that it's annas-archive.org; its security certificate is from *.hs.llnwd.net. This may be caused by a misconfiguration or an attacker intercepting your connection."
According to Wikipedia, www.ukispcourtorders.co.uk used to list the blocked domains and the court orders responsible.
https://en.wikipedia.org/wiki/List_of_websites_blocked_in_th...