I love this! I tried to apply the same idea to scan the tallest tree in New England with a drone. It didn't come out great, but I might just try again now.
I have been creating animations using a similar process but with a regular camera and manually splicing the frames together. [1,2,3] The effect is quite interesting in how it forces focus on the subject reducing the background into an abstract pattern. Each 'line' is around 15px wide.
I also shot a timelapse of the Tokyo skyline at sunset and applied a similar process [4], then motion tracked it so that time is traveling across the frame from left to right[5]. Each line here is 4 pixels wide and the original animation is in 8k.
Does anyone know what it looks like when you use a line scan camera to take a picture of the landscape from a moving car or train? I suspect the parallax produces some interesting distortions.
Sorry for the purple trees. The camera is sensitive to near infrared, in which trees are highly reflective, and I haven't taken any trains since buying an IR cut filter. Some of these also have dropped frames and other artifacts.
Iirc, at the last Olympics, Omega paired a high-frequency linear display with their finish-line strip cameras. Regular cameras saw a flashing line, but the backdrop to photo-finishes was an Omega logo. Very subtle, but impressive to pull off.
IMO the denoising looks rather unnatural and emphasizes the remaining artifacts, especially color fringe around details. Personally I'd leave that turned off. Also, with respect to the demosaic step, I wonder if it's possible to implement a version of RCD [1] for improved resolution without the artifacts that seem to result from the current process.
Yeah I actually have it disabled by default since it makes the horizontal stripes more obvious and it's also extremely slow. Also, I found that my vertical stripe correction doesn't work in all cases and sometimes introduces more stripes. Lots more work to do.
As for RCD demosaicing, that's my next step. The color fringing is due to the naive linear interpolation for the red and blue channels. But, with the RCD strategy, if we consider that the green channel has full coverage of the image, we could use it as a guide to make interpolation better.
When you do the demosaicing, and perhaps other steps, did you ever consider declaring the x-positions, spline parameters, ... as latent variables to estimate?
Consider a color histogram, then the logo (showing color oscillations) would have a wider spread and lower peaked histogram versus a correctly mapped (just the few colors plus or minus some noise) which would show a very thin but strong peak in colorspace. A a high-variance color occupation has higher entropy compared to a low-variance strongly centered peak (or multipeak) distribution.
So it seems colorspace entropy could be a strong term in a loss function for optimization (using RMAD).
Readit News
Here is how it came out: https://www.daviddegner.com/wp-content/uploads/2023/09/Tree-...
It was part of this story: https://www.daviddegner.com/photography/discovering-old-grow...
[1] https://youtube.com/shorts/VQuI1wW8hAw [2] https://youtube.com/shorts/vE6kLolf57w [3] https://youtube.com/shorts/QxvFyasQYAY
I also shot a timelapse of the Tokyo skyline at sunset and applied a similar process [4], then motion tracked it so that time is traveling across the frame from left to right[5]. Each line here is 4 pixels wide and the original animation is in 8k.
[4] https://youtu.be/wTma28gwSk0 [5] https://youtu.be/v5HLX5wFEGk
https://news.ycombinator.com/item?id=35738987
Anyway, I was looking for line-scan images of people walking down a busy street. Curious what they would look like.
Must be somewhat interesting deciding on the background content, too.
Nankai 6000 series, Osaka:
https://i.dllu.net/nankai_19b8df3e827215a2.jpg
Scenery in France:
https://i.dllu.net/preview_l_b01915cc69f35644.png
Marseille, France:
https://i.dllu.net/preview_raw_7292be4e58de5cd0.png
California:
https://i.dllu.net/preview_raw_d5ec50534991d1a4.png
https://i.dllu.net/preview_raw_e06b551444359536.png
Sorry for the purple trees. The camera is sensitive to near infrared, in which trees are highly reflective, and I haven't taken any trains since buying an IR cut filter. Some of these also have dropped frames and other artifacts.
You can get some cool distortions at very slow speeds, but at car or train speeds you won’t see anything
Dead Comment
[1] https://github.com/LuisSR/RCD-Demosaicing
As for RCD demosaicing, that's my next step. The color fringing is due to the naive linear interpolation for the red and blue channels. But, with the RCD strategy, if we consider that the green channel has full coverage of the image, we could use it as a guide to make interpolation better.
Consider a color histogram, then the logo (showing color oscillations) would have a wider spread and lower peaked histogram versus a correctly mapped (just the few colors plus or minus some noise) which would show a very thin but strong peak in colorspace. A a high-variance color occupation has higher entropy compared to a low-variance strongly centered peak (or multipeak) distribution.
So it seems colorspace entropy could be a strong term in a loss function for optimization (using RMAD).