Look at PPM. Your prediction model would work better with personalised data. PPM is efficient (nb. I see you are using python - look at this https://github.com/willwade/pylm - although be warned - i think my code is not quite right..)
Layout shifting for finger movement - well its great if you didnt have to look. The time for visual processing the letters adds a significant lag (its why typical word prediction isnt used that much and when it is - not over 3 predictions (I have papers on this if you are interested). But its not all bad..
Switch users who need next letter prediction this could dramatically support their rate of input. (view https://youtu.be/Bhj5vs9P5cw?si=VnytfH_vdEUWuLok&t=73 - now note how the keyboard blocks the scan up. But imagine if it just scanned each letter first by next most likely - or heck - like this repo - actually changes button position and kept the scan pattern the same. It would be a ton more efficient)
Exactly, a key should never be in the same place twice in a row- that's just wasted movement that could be better optimized. Letters the computer predicts you probably won't use should automatically disappear, and be replaced with numerous copies of others based on the things it thinks you will type.
We should extend this concept of constant automatic optimization to all aspects of everyday life- for example your workplace location should physically relocate each day to an optimal location based on where each person coming in that day lives. An algorithm should tell you where to put away the dishes in your kitchen based on a constantly changing optimization algorithm, so that your dinner plates are in a different cabinet each day. Language itself should be radically redesigned daily to keep it optimal, with changelogs pushed out to be learned and memorized each morning before communicating with anyone.
A good part is that it will do nothing to improve security and will actively harm peoples' ability to type things correctly, and the best part is when it makes its way into scientific literature and algorithms. Good luck reproducing that which was entered incorrectly!
I think that this is an attempt to stop the “clean key” problem. Security system keypads (especially outdoor ones) tend not to be cleaned, so as time passes, it is easy to spot the dirty keys. Dirty keys are not being pressed and are therefore not in the passcode.
So look for the clean keys and try combinations from there. In a 4 digit (0-9) keypad, knowing the clean numbers drops the possible codes from 9999 to 24 (if my early morning math holds up).
Also helps the issue of someone looking over the shoulder of a valid person. Chances are they are just seeing the position and not the character pressed. So the keyboard changes and you actually need to know the character not just the old position.
This would be an interesting one to integrate into password entry forms... although you'd need to show the randomised keyboard layout on screen.
Or have a keyboard with oled or e-ink keys, like the Optimus Maximus [0] promised to deliver. It's kinda weird that nobody else seems to have picked up on this concept since then. Probably just impractical or too expensive.
I read that its patents expired in 2016; around 2015 there was a concept for an e-ink button keyboard, but that site is now a plain gambling ad. There's also https://www.nemeio.com/ that still works, but its buttons look like sunken screens under plastic domes.
A number of countries use this when giving your pin for a credit card or similar (I've noticed it in both Greece and India).
I can't help but feel like it's less secure than the default layout - I'm quite good at hiding my PIN and typing quickly, but when the positions of the numbers are randomised, I feel like I practically end up saying my PIN out loud as I try and remember it.
Had this at an ATM recently, and it took a couple of tries at my PIN before I looked at the keypad and realized what was going on. One more wrong PIN and I could have lost my card.
One way to avoid this element of surprise, for those who do not enjoy surprises, could be to have only a single key whose action upon clicking would simply be the most probable next action.
This project brilliantly points out that we are limited and shaped by how keys and layouts are presented to us.
What I don’t understand is why not go all the way. Why not also change the concept of what’s needed to enter a character? It would make sense that keys aren’t the ideal affordance to express a key every time.
Use a volume slider for a letter, use radio buttons, on/off toggles. What if I could draw the letter on my trackpad?
I hope this opens the door for a thriving ecosystem of expansions.
I have been proposing to adapt Shorthand* to computer tactile input many times. Touchpads and touchscreens can use it. It seems faster and easier for both the computer and the person than having to recognize slow handwriting.
Secretaries used this for many decades, because it allowed them to record speech in real-time.
There is always a dismissive comment about how this is not a good idea for one reason or another, but I still think it would be a great feature to have.
(Google's gboard offers a handwriting keyboard, but it's on the touch screen and doesn't count. Also, it's not graffiti, so a bit more error prone. It does recognize joined-up handwriting though.)
There's also some keyboards on f-droid for braille text input and morse code input.
(In iMessage, as one easy example app, there's an entry box to send a new message. If you tap into that and get a keyboard click the keyboard icon in the bottom corner and there's a "moving finger" icon you can tap to get a trackpad to draw letters on. Older and smaller Watches won't even have the keyboard and will just have the trackpad.)
I had the idea this week to use the Rocksmith USB adapter to develop a stenography system that allows me to control my computer using my guitars with chords, notes, and riffs.
There's a Rocksmith mod that allows you to control the menu using the guitar. Also, the Rocksmith USB cable is simply an ADC (with lower latency than most)
> What about analog partial key press? There are Hall effect keyboard switches that do that, I want to try some.
That sounds fun, now I really want to try out a keyboard where half press is lowercase and full press is UPPERCASE! No more pinky strain from shift presses.
This is true. We’ve been so formatted by big keyboard that I never considered just the tip of my fingers. I hope you’ll make good use of our opposable thumbs in this new keyboard paradigm
I'd put some random keys on the bottom of my seat or wired into the TV remote or the doorbell button of my neighbours. The ordered shape and regular size, position of keys is too much of a burden to mee, constricts my wondering creative mind too much.
Took me 1 year to rewire my brain to switch from Qwerty to Dvorak (I struggled with from RSI for nearly 6 years) and nearly 2 years to gain full speed. It is hard, very hard for the brain. It is mentally tiring to rewire your brain like this and you will do your job at a much worse efficiency (forget pair programming for a while). You need new keyboard stickers also. Great job at making this idea but it is not practical.
I switched to Dvorak 20 years ago and I love it. It only took me about 2 weeks to learn and to pass my qwerty typing speed. I could actually touch type on either keyboard for the first year or two after I switched. I recommend not using stickers or dvorak keyboards at all, it helps you learn faster and your family won't get pissy because they can't use your computer anymore.
Also, it is not difficult to use other peoples computers. You can switch keyboards pretty easy these days on windows and mac. In fact, my very first computer (Apple IIc) had a mechanical switch to flip between qwerty and dvorak.
And there's also the problem of using computers that aren't yours. I haven't tried (and I will never), but I imagine switching between layouts is at least very inconvenient.
Dvorak is supported by almost every computing device under the sun today. Old Apple models (I want to say the IIc?) even had a hardware button to switch layouts. Switching between Dvorak and Qwerty is usually an easy to find User setting. (If doing it on someone else's user account, you just have to remember to be kind and switch it back when done.)
As a Colemak touch typer I envy that some days. Colemak is available easily everywhere but Windows. macOS, iOS, Android, and Linux all have it similarly out of the box with Dvorak, but Windows it still requires an install and that install still requires Admin approval because keyboard layouts are needed by kernel-level drivers. But I knew that when I switched, took a few years to realize there's no shame in hunt-and-pecking on QWERTY when using someone else's machine.
(I was also privileged when I switched to know that I didn't need to use shared machines. I was far enough along in grad school where I was allowed/encouraged to do everything on a personal laptop, and the few remaining "lab classes" with shared machines all allowed Remote Desktop. That took advantage of things like if you remote desktop back to one of your own machines, your remote machine still responds to your chosen layout once you've logged in. Just have to hunt and peck long enough to type your machine's address, username, and password.)
I switched to Dvorak ±8 years ago, and I can still type qwerty almost as well as I could back then. I could imagine it being a problem if you're using other people's machines on a daily basis though.
thats wonderful. id like that to not change the order of the letters - but change the highlight order. Do a round 1 of frequency order first (just do first say 6 letters) then do a round 2 which is standard order..
i probably am not making much sense. Look at where I'm coming from in the world of Assistive Tech - https://docs.acecentre.org.uk/products/echo (go to around 5 min mark in the vide)
Thank you for the link! That looks fascinating. I have instantly added it to my "keyboard" bookmarks folder!
I will promptly forget it exists and keep typing on the random layout I learned to touch type on 20 years ago, all the while suffering from terrible finger acrobatics and wishing for a better layout every single day.
I've made my own layouts[0][1] and I'd say the many variants of Hands Down are probably the best example of a layout that's been developed in a thoughtful manner.
Very nice website with many solid principles! Though one big flaw in all the designs I've seen is that they rely on bad data - text corpus that is the output of human editing efforts, not the input.
So all those "0.00001%" stats aren't as precise as they seem, and you still see the frequently used backspace/enter in the awful pinky position despite the official philosophy of "Hands Down is easy on the pinkies"
Have you seen any design that is based on actual human input?
As far as I'm aware hands down layouts were primarily designed to be used on a split ergonomic keyboard with thumb cluster, the space and enter lives on the thumb cluster.
For those who type on German keyboards or in German, there are a number of layouts that have evolved around the Neo 2 layout [1]. The main driving force is ergonomics, but since not everyone works the same way, many variants have emerged (and still do) [2]. Interesting is the craftsmanship, including customized programs to optimize the layouts [3].
Yeah when I was a 20 or so and just started working I taught myself Dvorak for a while, but realized how bad it was for coding in C. So I made my own C language one as well.
Of course you still have to type text and at the time you couldn't easily switch between different layouts, so after a time I got bored of it and gave up.
Readit News
Look at PPM. Your prediction model would work better with personalised data. PPM is efficient (nb. I see you are using python - look at this https://github.com/willwade/pylm - although be warned - i think my code is not quite right..)
Layout shifting for finger movement - well its great if you didnt have to look. The time for visual processing the letters adds a significant lag (its why typical word prediction isnt used that much and when it is - not over 3 predictions (I have papers on this if you are interested). But its not all bad..
Switch users who need next letter prediction this could dramatically support their rate of input. (view https://youtu.be/Bhj5vs9P5cw?si=VnytfH_vdEUWuLok&t=73 - now note how the keyboard blocks the scan up. But imagine if it just scanned each letter first by next most likely - or heck - like this repo - actually changes button position and kept the scan pattern the same. It would be a ton more efficient)
(and a bit of a rabbit hole.. What if keys had word predictions on them? This is basically the end result of ACE-LP: https://discovery.dundee.ac.uk/en/publications/ace-lp-augmen...)
We should extend this concept of constant automatic optimization to all aspects of everyday life- for example your workplace location should physically relocate each day to an optimal location based on where each person coming in that day lives. An algorithm should tell you where to put away the dishes in your kitchen based on a constantly changing optimization algorithm, so that your dinner plates are in a different cabinet each day. Language itself should be radically redesigned daily to keep it optimal, with changelogs pushed out to be learned and memorized each morning before communicating with anyone.
So look for the clean keys and try combinations from there. In a 4 digit (0-9) keypad, knowing the clean numbers drops the possible codes from 9999 to 24 (if my early morning math holds up).
Also helps the issue of someone looking over the shoulder of a valid person. Chances are they are just seeing the position and not the character pressed. So the keyboard changes and you actually need to know the character not just the old position.
Or have a keyboard with oled or e-ink keys, like the Optimus Maximus [0] promised to deliver. It's kinda weird that nobody else seems to have picked up on this concept since then. Probably just impractical or too expensive.
I read that its patents expired in 2016; around 2015 there was a concept for an e-ink button keyboard, but that site is now a plain gambling ad. There's also https://www.nemeio.com/ that still works, but its buttons look like sunken screens under plastic domes.
[0] https://en.wikipedia.org/wiki/Optimus_Maximus_keyboard
It also takes ~5s to render the page on my PC, no idea WTF it's doing during that time, probably mining crypto.
Hmm actually, it doesn't rearrange on every digit, it's a static but random layout every time you go to that page.
I can't help but feel like it's less secure than the default layout - I'm quite good at hiding my PIN and typing quickly, but when the positions of the numbers are randomised, I feel like I practically end up saying my PIN out loud as I try and remember it.
What I don’t understand is why not go all the way. Why not also change the concept of what’s needed to enter a character? It would make sense that keys aren’t the ideal affordance to express a key every time.
Use a volume slider for a letter, use radio buttons, on/off toggles. What if I could draw the letter on my trackpad?
I hope this opens the door for a thriving ecosystem of expansions.
I have been proposing to adapt Shorthand* to computer tactile input many times. Touchpads and touchscreens can use it. It seems faster and easier for both the computer and the person than having to recognize slow handwriting.
Secretaries used this for many decades, because it allowed them to record speech in real-time.
There is always a dismissive comment about how this is not a good idea for one reason or another, but I still think it would be a great feature to have.
* https://en.wikipedia.org/wiki/Shorthand
The Apple Newton did that!
https://en.wikipedia.org/wiki/Apple_Newton#Notes
(Google's gboard offers a handwriting keyboard, but it's on the touch screen and doesn't count. Also, it's not graffiti, so a bit more error prone. It does recognize joined-up handwriting though.)
There's also some keyboards on f-droid for braille text input and morse code input.
(In iMessage, as one easy example app, there's an entry box to send a new message. If you tap into that and get a keyboard click the keyboard icon in the bottom corner and there's a "moving finger" icon you can tap to get a trackpad to draw letters on. Older and smaller Watches won't even have the keyboard and will just have the trackpad.)
Why are key switches only press?
What about analog partial key press? There are Hall effect keyboard switches that do that, I want to try some.
What about finger twist for input? I don't think anything exists? Would that be a volume knob on a key switch?
That sounds fun, now I really want to try out a keyboard where half press is lowercase and full press is UPPERCASE! No more pinky strain from shift presses.
[1] : https://www.charachorder.com
You absolutely do not and it even hampers your ability to quickly learn to type efficiently if you rely on key legends or layout references.
Also, it is not difficult to use other peoples computers. You can switch keyboards pretty easy these days on windows and mac. In fact, my very first computer (Apple IIc) had a mechanical switch to flip between qwerty and dvorak.
As a Colemak touch typer I envy that some days. Colemak is available easily everywhere but Windows. macOS, iOS, Android, and Linux all have it similarly out of the box with Dvorak, but Windows it still requires an install and that install still requires Admin approval because keyboard layouts are needed by kernel-level drivers. But I knew that when I switched, took a few years to realize there's no shame in hunt-and-pecking on QWERTY when using someone else's machine.
(I was also privileged when I switched to know that I didn't need to use shared machines. I was far enough along in grad school where I was allowed/encouraged to do everything on a personal laptop, and the few remaining "lab classes" with shared machines all allowed Remote Desktop. That took advantage of things like if you remote desktop back to one of your own machines, your remote machine still responds to your chosen layout once you've logged in. Just have to hunt and peck long enough to type your machine's address, username, and password.)
For hunt and peck typists.
i probably am not making much sense. Look at where I'm coming from in the world of Assistive Tech - https://docs.acecentre.org.uk/products/echo (go to around 5 min mark in the vide)
I'd be quite interested in trying that. Maybe as a learning aid. But if you get your typing speed up, I can see it being visually very noisy.
Not only that, people just change them to suit their needs! They use things like Hands Down [0] as a guide, and make up their own layout.
Wild.
[0]: https://sites.google.com/alanreiser.com/handsdown
I will promptly forget it exists and keep typing on the random layout I learned to touch type on 20 years ago, all the while suffering from terrible finger acrobatics and wishing for a better layout every single day.
[0]: https://www.jonashietala.se/series/t-34/
[1]: https://www.jonashietala.se/blog/2024/11/26/the_current_cybe...
So all those "0.00001%" stats aren't as precise as they seem, and you still see the frequently used backspace/enter in the awful pinky position despite the official philosophy of "Hands Down is easy on the pinkies"
Have you seen any design that is based on actual human input?
[1] https://www.neo-layout.org/Layouts/
[2] https://maximilian-schillinger.de/keyboard-layouts-neo-adnw-...
[3] https://hg.sr.ht/~arnebab/evolve-keyboard-layout
Changed to US qwerty 30 years ago, never looking back.
site is a bit awkward to browse though.
Of course you still have to type text and at the time you couldn't easily switch between different layouts, so after a time I got bored of it and gave up.
Deleted Comment