I wonder if instead of shrinking the text geometrically when zoomed out, you could shrink it semantically (with a LLM or something similar) as in the wikipedia image ZUI example. Of course this would require an enormous amount of pre-processing to get multiple levels of zoom shrinkage.
I like this idea, but it should have a limit - because every time I zoom in to read the name of a road on a map and the text resizes to a smaller size for infinity I wish it didn't.
Just as much as a map. I did something similar using a local LLM to summarize paragraphs. Using a public turbo model or even something local would be very doable.
Wait, how is this possible?? I tried to visualize a few tens of MB of text in the browser [1] on a canvas and it gets really choppy if I don't optimize it by not showing stuff that can't be read.
Impressive that it can handle gigabytes of text!
I'd wager by making the requirements simpler. Wideboard is rendering monospace, so every document is made up of of 8x16px cells, and you just have to decide which glyph to render in each.
From a quick skim of the source code this is mostly precalulated. Basically each document is rendered as one rectange, where the shader converts the current position to the row/column of the character to be rendered. There is one texture that holds the length and starting offset of each line (using one pixel per line), and another texture that for each offset tells you which glyph to render (using one pixel per glyp). The third texture is the glyph map that holds the actual glpyhs (a simple 256x256 texture of all supported characters). The last texture is static, the other two are calculated once as the document is loaded, and there is a neat shader that uses all three textures to render the actual document view. Since the shader is sampled at fewer points if the rectangle takes up fewer pixels on your screen this works out well for zoomed out views.
Brings back bad memories of Miro. It's that sort of ZUI that encourages every middle manager to spend hour-long meetings looking for ten cards and three wireframes across four projects, when it could've just been an email with bullet points and a screenshot.
Ha, I just mentioned Oberon in another HN thread. I was actually reading the Oberon book last week. Wirth goes from OS to programming language to hardware to compiler in one manual. I didn't actually realize it was a zooming interface though.
Hint: it's a dead end people! Been around forever as an idea, lots of academic studies and some industry apps. Doesn't really work and has never taken off despite many attempts. Many reasons for this, including the real world doesn't include zoom at multiple orders of magnitude, going in all planes is harder than a linear list for people to scan, semantics in 2D isn't something that's easy to grasp cognitively unless you specifically arranged things that way (which isn't the case for this kind of software, and even then you can lose what's happening / adjusting everything to account for something new isn't well supported), and not many use cases where this is broadly useful (especially compared to other tried-and-true approaches).
Hint: it's a dead end people! Been around forever as an idea, lots of academic studies and some industry apps.
Is it, though? For one thing, many feel that projects like Dataland led directly to the modern Windowed User Interface. One can think of opening windows as a kind of limited zooming with very specific affordances. The iOS UI is cited in the Wikipedia article on ZUI as being another example.
Basically, any UI in which one can "drill down" into greater detail or smaller/more specific contexts is effectively a kind of zooming. The problem, is losing the connection to the larger context.
Another thing to consider: At the time many of these things were tried out, there wasn't enough capability in the CPU and UI to render complex data fast enough. In fact, I was involved with a round-trip UML DB to UI product dating from the 1990's, which looked great in demos. However, it became dog slow when actually given someone's company database. This was a problem which dogged many projects with innovative UI ideas. (A problem which many CAD software apps deal with, and which seem to have been solved to some degree.)
semantics in 2D isn't something that's easy to grasp cognitively unless you specifically arranged things that way
I was hoping that this was going to be about zooming contexts rather than renderings. It would be much effective to me if I could zoom in/out of contexts like on a distributed system diagram, and zooming in and in, until I got to a code browser for part of subsystem I wanted to change, and have those changes immediately effective, like a REPL but in the style of Smalltalk image.
It works with maps since they are fundamentally spatial, every country has its fixed position on the globe which doesn't change. With most other data there is no intuitive way to map the data into a 2D space and it gets even worse when that data frequently changes, completely changing whatever layout scheme you might have come up with.
A little bit of zooming can still work great when you have a very limited item count, e.g. Mission Control on MacOS to manage windows or thumbnail view of tabs in some browsers. But as core UI for the whole OS it gets quite confusing when you don't really have an intuitive understanding of where things should be.
Figma is popular in spite of this. It's my number one frustration. Actually being able to link someone to a specific view is nigh impossible to do consistently.
I think the only proven use cases for ZUIs have been Maps, Calendars, and Photo apps (Apple Photos and Google Photos). The last two to switch between different time period views: Day->Month->Year for photos and Day>Week>Month for calendars.
There is maybe some room for design too (Figma), but there are many times I wish it DIDN'T do that and encouraged better frame/file organization instead.
Similarly, it's too often used for (mis)organization, like in Miro or Prezi or "mind map" apps. I feel like those try to shoehorn information into Sherlock-like "mind palaces" in a way that only makes sense for the creator but are inscrutable to everyone else and just makes information harder to find later on. They always lead to some sort of pixel-hunting where the presenter zooms out and in, out and in, out and in, wasting time on navigation and placefinding instead of information dissemination.
I think the overall point is some types of hierarchical information naturally lend themselves to "drill down" type UIs more than others. When you have levels of detail you don't need to see at first glance, drilling/zooming is awesome. When you have a bunch of things of equal hierarchy, presenting them in a big flat pile isn't any better in the virtual world than in the real world... it's the digital equivalent of 10,000 sticky notes on a wall.
>Ben Shneiderman was inspired by the 1960's 'Op Art' and the exhibits that he came across at the Museum of Modern Art in New York. Op Art or Optical art is a form of kinetic art related to geometric designs that create movement in the eyes.
>This site features draft designs and full views of the Treemap Art project. View more about the exhibitions.
>By Ben Shneiderman
>Although I conceived treemaps for purely functional purposes (understanding the allocation of space on a hard drive), I was always aware that there were appealing aesthetic aspects to treemaps. Maybe my experiences with OP-ART movements of the 60s & 70s gave me the idea that a treemap might become a work of art. That idea was revived in 2013 by way of my contacts with Manuel Lima who produced a beautiful coffee-table book on the history of trees that has several chapters on treemaps and their variations.
>I believe that there are at least four aesthetic aspects of treemaps:
>1. layout design (slice-and-dice, squarified, ordered, strip, etc.),
>2. color palette (muted, bold, sequential, divergent, rainbow, etc.), and,
>3. aspect ratio of the entire image (square, golden ratio, wide, tall, etc.).
>4. prominence of borders for each region, each hierarchy level, and the surrounding box
Ben Shneiderman: Every AlgoRiThm has ART in it: Treemap Art Project:
>Ben Shneiderman, distinguished university professor, University of Maryland, College Park and National Academy of Engineering member, spoke at the October 16, 2014 DC Art Science Evening Rendezvous (DASER). Ben Shneiderman described the invention of treemaps and showed examples of its usage. He then turned to the aesthetics of treemaps, which led him to create the “Every AlgoRiThm has ART in it: Treemap Art Project” exhibit on view in the Keck Center first floor galleries (www.cpnas.org). He demonstrated how users of the free treemap application can generate their own artworks, without programming.
The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines — Don Hopkins — October 1989 (a paper I wrote when I worked with Ben Shneiderman at his University of Maryland Human Computer Interaction Lab):
>Darkness fell in from every side, a sphere of singing black, pressure on the extended crystal nerves of the universe of data he had nearly become… And when he was nothing, compressed at the heart of all that dark, there came a point where the dark could be no more, and something tore. The Kuang program spurted from tarnished cloud, Case’s consciousness divided like beads of mercury, arcing above an endless beach the color of the dark silver clouds. His vision was spherical, as though a single retina lined the inner surface of a globe that contained all things, if all things could be counted.
>[Gibson, Neuromancer]
>The Pseudo Scientific Visualizer is the object browser for the other half of your brain, a fish-eye lens for the macroscopic examination of data. It can display arbitrarily large, arbitrarily deep structures, in a fixed amount of space. It shows form, texture, density, depth, fan out, and complexity.
"Another thing I know about screens is, you can go into them as far as you want forever. And you can come out of them too, but you might not end up in the same place that you started. You might get lost. Or you might not."
Readit News
[1]: https://tomthe.github.io/hackmap/
From a quick skim of the source code this is mostly precalulated. Basically each document is rendered as one rectange, where the shader converts the current position to the row/column of the character to be rendered. There is one texture that holds the length and starting offset of each line (using one pixel per line), and another texture that for each offset tells you which glyph to render (using one pixel per glyp). The third texture is the glyph map that holds the actual glpyhs (a simple 256x256 texture of all supported characters). The last texture is static, the other two are calculated once as the document is loaded, and there is a neat shader that uses all three textures to render the actual document view. Since the shader is sampled at fewer points if the rectangle takes up fewer pixels on your screen this works out well for zoomed out views.
like google maps.
https://ignorethecode.net/blog/2009/04/22/oberon/
1. https://eaglemode.sourceforge.net/screenshots.html 2. https://aur.archlinux.org/packages/eaglemode
Many modern text editors have a zoomed out view of the code that you can use to scroll around it.
There's photo browsers with thumbnail views. As you zoom in to each photo, more information is exposed.
Then there's video and audio editing software with complex zoomable timelines.
They're kind of everywhere to be honest, just a bit hidden.
Is it, though? For one thing, many feel that projects like Dataland led directly to the modern Windowed User Interface. One can think of opening windows as a kind of limited zooming with very specific affordances. The iOS UI is cited in the Wikipedia article on ZUI as being another example.
Basically, any UI in which one can "drill down" into greater detail or smaller/more specific contexts is effectively a kind of zooming. The problem, is losing the connection to the larger context.
Another thing to consider: At the time many of these things were tried out, there wasn't enough capability in the CPU and UI to render complex data fast enough. In fact, I was involved with a round-trip UML DB to UI product dating from the 1990's, which looked great in demos. However, it became dog slow when actually given someone's company database. This was a problem which dogged many projects with innovative UI ideas. (A problem which many CAD software apps deal with, and which seem to have been solved to some degree.)
semantics in 2D isn't something that's easy to grasp cognitively unless you specifically arranged things that way
Do you have specific examples?
Google Maps and Waze and friends being universally useful these days?
A little bit of zooming can still work great when you have a very limited item count, e.g. Mission Control on MacOS to manage windows or thumbnail view of tabs in some browsers. But as core UI for the whole OS it gets quite confusing when you don't really have an intuitive understanding of where things should be.
Indeed. For a concrete example see Ted Nelson's "Zigzag" UI concept:
https://en.m.wikipedia.org/wiki/ZigZag_(software)
https://www.youtube.com/watch?v=WEj9vqVvHPc
tl;dr: a lot of good and interesting ideas that should never actually be implemented.
Similarly, it's too often used for (mis)organization, like in Miro or Prezi or "mind map" apps. I feel like those try to shoehorn information into Sherlock-like "mind palaces" in a way that only makes sense for the creator but are inscrutable to everyone else and just makes information harder to find later on. They always lead to some sort of pixel-hunting where the presenter zooms out and in, out and in, out and in, wasting time on navigation and placefinding instead of information dissemination.
-------
On the other hand, it CAN be useful for some visualizations, like taxonomy: https://itol.embl.de/itol.cgi or https://www.onezoom.org/life.html
"Drill down for details" like in disk space analysis: https://www.youtube.com/watch?v=BKClylmlv3w&t=1s (or similar one in D3: https://observablehq.com/@d3/zoomable-sunburst)
Treemaps: https://observablehq.com/@d3/zoomable-treemap
I think the overall point is some types of hierarchical information naturally lend themselves to "drill down" type UIs more than others. When you have levels of detail you don't need to see at first glance, drilling/zooming is awesome. When you have a bunch of things of equal hierarchy, presenting them in a big flat pile isn't any better in the virtual world than in the real world... it's the digital equivalent of 10,000 sticky notes on a wall.
https://www.cs.umd.edu/~ben/papers/Johnson1991Tree.pdf
HCIL Archive: Treemap Home Page:
https://www.cs.umd.edu/projects/hcil/treemap/
How Ben Shneiderman’s Treemaps Found Place In The Museum Of Modern Art:
https://analyticsindiamag.com/how-ben-shneidermans-treemaps-...
>Ben Shneiderman was inspired by the 1960's 'Op Art' and the exhibits that he came across at the Museum of Modern Art in New York. Op Art or Optical art is a form of kinetic art related to geometric designs that create movement in the eyes.
Ben Shneiderman's Treemap Art:
https://treemapart.wordpress.com/
>This site features draft designs and full views of the Treemap Art project. View more about the exhibitions.
>By Ben Shneiderman
>Although I conceived treemaps for purely functional purposes (understanding the allocation of space on a hard drive), I was always aware that there were appealing aesthetic aspects to treemaps. Maybe my experiences with OP-ART movements of the 60s & 70s gave me the idea that a treemap might become a work of art. That idea was revived in 2013 by way of my contacts with Manuel Lima who produced a beautiful coffee-table book on the history of trees that has several chapters on treemaps and their variations.
>I believe that there are at least four aesthetic aspects of treemaps:
>1. layout design (slice-and-dice, squarified, ordered, strip, etc.), >2. color palette (muted, bold, sequential, divergent, rainbow, etc.), and, >3. aspect ratio of the entire image (square, golden ratio, wide, tall, etc.). >4. prominence of borders for each region, each hierarchy level, and the surrounding box
Ben Shneiderman: Every AlgoRiThm has ART in it: Treemap Art Project:
https://www.youtube.com/watch?v=4LW4m6BdQXI
>Ben Shneiderman, distinguished university professor, University of Maryland, College Park and National Academy of Engineering member, spoke at the October 16, 2014 DC Art Science Evening Rendezvous (DASER). Ben Shneiderman described the invention of treemaps and showed examples of its usage. He then turned to the aesthetics of treemaps, which led him to create the “Every AlgoRiThm has ART in it: Treemap Art Project” exhibit on view in the Keck Center first floor galleries (www.cpnas.org). He demonstrated how users of the free treemap application can generate their own artworks, without programming.
The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines — Don Hopkins — October 1989 (a paper I wrote when I worked with Ben Shneiderman at his University of Maryland Human Computer Interaction Lab):
https://donhopkins.medium.com/the-shape-of-psiber-space-octo...
>The Pseudo Scientific Visualizer
>Darkness fell in from every side, a sphere of singing black, pressure on the extended crystal nerves of the universe of data he had nearly become… And when he was nothing, compressed at the heart of all that dark, there came a point where the dark could be no more, and something tore. The Kuang program spurted from tarnished cloud, Case’s consciousness divided like beads of mercury, arcing above an endless beach the color of the dark silver clouds. His vision was spherical, as though a single retina lined the inner surface of a globe that contained all things, if all things could be counted.
>[Gibson, Neuromancer]
>The Pseudo Scientific Visualizer is the object browser for the other half of your brain, a fish-eye lens for the macroscopic examination of data. It can display arbitrarily large, arbitrarily deep structures, in a fixed amount of space. It shows form, texture, density, depth, fan out, and complexity.
https://www.youtube.com/watch?v=Q4OIcwt8vcE
"Another thing I know about screens is, you can go into them as far as you want forever. And you can come out of them too, but you might not end up in the same place that you started. You might get lost. Or you might not."