I hate having a polluted home directory, especially when an application thinks it should get a non hidden directory in my home!
The one that upsets me the most is the default directory for go modules `~/go`. This frustrates me so much. I refused to install any go apps or use it for development for years. I've unfortunately had to give in, and it can at least be overridden by setting `GOPATH`, but it is a terrible, terrible default.
The worst offenders are CLI tools written by people on Macs that do not respect XDG, because it's not a thing over there. So every tool gets to pollute your dotfiles with its own stupid directory. .rustup, .mix, .npm, .yarn, etc.
But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
Weirdly, enough, golang is one of the only programming languages that actually has built-in support for a cross-OS config dir location: [os.UserConfigDir()][1].
I don't really ever program in golang, but whenever I write a Node.JS/Python tool that does need a user-global config file, I just write my own implementation of it:
function userConfigDir() {
switch (process.platform) {
case 'darwin':
return `${os.homedir()}/Library/Application Support`;
case 'win32':
if (process.env['APPDATA']) {
return process.env['APPDATA'];
} else {
throw new Error('%APPDATA% is not set correctly');
}
case 'aix':
case 'freebsd':
case 'openbsd':
case 'sunos':
case 'linux':
return process.env['XDG_CONFIG_HOME'] || `${os.homedir()}/.config`;
default:
throw new Error(`The platform ${process.platform} is currently unsupported.`);
}
}
>But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
Counterpoint:
I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in.
I got used to it and in HOME I put some of my manually installed tools like `AMD_AOCL` for the AMD AOCL libraries, `Android/android-studio` for Android Studio, `intel` for IPP/OpenAPI, etc.
Don't want it anymore? Easy rm -rf, no need to go digging for where it could be hidden.
IMO, the GOPATH itself is a terrible design. It makes you mix together things from unrelated projects in the same directory structure, instead of each project having its own independent directory structure, like on every other programming language. The GOPATH design goes so much against the way I organize my projects, that it's the main reason I have never got interested into learning Go.
Perhaps this design makes more sense for those who prefer to keep several unrelated projects together in a single monorepo, but that's not my preference.
organization structure makes a ton of sense for every project and as long as you organize all of your languages into this one tree, everything just works.
This hasn't been the case since go modules exist, if I understand your issue correctly: https://go.dev/blog/using-go-modules. You can additionally vendor each project's dependencies
In a situation like this, when I have control over /etc/passwd and can create accounts as I please, then I might have a single account, let's say /home/amturing, then (as a subdirectory) another account /home/amturing/amtgo.
Unruly apps that benefit from UID/GID regimentation can be constrained in this way. The whole app assuming ~amtgo is also helpful.
EDIT: I also remembered this useful advice on prefixing private shell scripts in your path with a comma:
$HOME is the one directory which belongs to the user. In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
I have given up on the idea of a clean home dir a long time ago. I have "networked" directories (pCloud,Dropbox) in my home for everything I care for, and those are organized "perfectly".
Additionally I symlink .vimrc and .gitconfig to a git repos. Then everything else in my home dir can be total garbage, it no longer matters. The big plus is that if a machine dies I can be back in business in minutes on another one.
Same thing, wish that from the start unix had some standardized "applications put everything they want here" home directory that's separate from the "here user chooses to put their own files" home directory
At the start, unix commands didn't need or use configuration files. The world really was that simple, for a while. Now we have even simple desktop apps, that I have not changed a single setting in, insisting to write a whole directory tree of whatever data in my home.
Or absolutely moronic things like ~/.config/VSCod*/*Cache* -- really, cache in config?
I wish more developers would seek out and embrace libraries for saving things in sensible locations. Python has one called “platformdirs” that takes the guesswork out of where to save things automatically for the user’s platform. I have no doubt all the other major environments have something similar.
If you ask me, this should all be built-in, but even then (because on many platforms, it is), people can’t help rolling their own.
To alleviate the issue with most applications dumping their files in your home directory, i found xdg-ninja[0] to be helpful.
In short, it scans all your programs and determines if you can configure it to respect the xdg standards. It doesn't work for everything, but most applications do seem to have such an option.
This is so individual; his solution is not useful for me, and my solution won't be useful for anyone else.
My home directory is nearly empty, because all the files I work with are in OwnCloud (so the real question is: what is the directory structure in OwnCloud). Local Git repositories are on a completely separate partition.
Since KeepassXC now handles SSH keys, the keys from .ssh are now in the Keepass-file, which is on OwnCloud. That was a huge simplification, and now there is nothing in my home directory that I really care much about.
The only thing that matters to me is portability when I switch systems, and aware of work vs personal environments
I need to be able to login and run a single command to sync to have the appropriate set of files for whatever /linux/work/personal/desktop/server I am on
For example, I never want certain environment variables to be exported to work systems, like my home vault endpoint or certain tokens.
I recently switched to home-manager from the NixOS project and it looks very promising. The Nix language is complicated but the abstraction to define all these different types of environments is exactly what I need, and git branches manage the work/personal split of file content
I think it's a nice idea, but don't like the media structure into family and so on. Looks like he's going to have a ton of problems with duplicates and then edited duplicates down the line, which can easily be mixed up, and then you lose your edits and so on.
Pictures are better sorted with EXIF keywords. So, metadata stored in the picture itself, perhaps even in the MIME type. Thus, if a picture is family related, just tag it #family, or #personx and so on. This is why I store them in the same folder on a per-date basis. The rest is keywords edited with programs such as Adobe Bridge and so on.
As for document file name structure, text files and the like, I've used both `Date then Descrpition.txt` or `Keyword Title or Description and then Date.txt` with the date obviously being an ISO date such that `YYYY-MM-DD-hhmm` with `-hhmm` being optional, also for sorting reasons. Sometimes I like sorting on "topics" i.e. keywords or titles, but other times, such as in logs, I like the date first because it's more essential to know when you logged something, and not necessarily the topic.
You should think the date is superfluous since it's also stored in the system. However, my experience that when you move around a file, the date eventually changes, and certainly if you make a mistake along the way. Meanwhile a filename date doesn't change. Also it helps with list sorting where applicable.
Media-specific content-addressed stores (ideally as overlays which refer to but don't change your directory structure or file organization) seem to be the way to go with this.
I think photoprism and photostructure don't really care about your directory structure or organization, but paperless(-ngx, or whatever is the most current iteration) is "notorious" for being opinionated about organization / not wanting to respect your organization.
I used camlistore/perkeep for photos for a while, but google photos has an absolutely killer feature of knowing who is in every picture, even accounting for age: although two of my boys who are 8 years apart look really similar at similar ages, it knows who is who. I don't know if it uses facial analysis or photo metadata or what, but it has never mistaken one for the other. I don't have a reasonable way of getting that tag information out of google photos (even though I'm paying for the service).
It might be time to revisit this, though.. I don't recall whether photostructure / photoprism try to do facial recognition, but even if they don't already, I bet they'll soon be roughly on par with google photos (or good enough that I can stop depending on google for this).
That's documents and photos / videos, what about music? For better or for worse, It's been at least a ~decade since I tried keeping my own music collection as files. Are there things these days like paperless / photoprism, a library system for music that has deeper integration with the content than just "files on disk"?
Depends a bit on the use case. For personal photos, mobile photos and the like, I use Google Photo as well. But it just isn't suitable for professional storage or sorting IMHO, tho the face recognition is certainly neat. The best I've came across was FotoStation, but it's many years since I worked with those sorta things.
You may want to try TagSpaces https://www.tagspaces.org/ or TMSU https://tmsu.org/ which provide mechanisms for managing tags of arbitrary files (not only EXIF or ID3 ones).
I've also been on the lookout for an AI assisted duplicate finder. Like, it'll compare pictures and then rate similarity between pictures on some scale, by appearance, colour or topic, perhaps with an auto-tagging feature as well. Ofc it should also compare dates, file sizes and the usual MIME type or hash based similarities as well.
Capital names for GUI things, lowercase names for CLI things. I'd prefer ~/documents, etc, but GUI people insist on capitalizing, so whatever. Very rarely I have to mix those, so it's not a big problem.
~/dotfiles is my dotfiles directory with git. I create ~/.zshrc -> dotfiles/zshrc, etc. I don't use any software to manage that, I just create symlinks. In the past I've used ~/.dotfiles but I think that making it visible makes more sense.
~/projects is my projects directory. ~/projects/test is throwaway test projects to check things out, etc. ~/projects/my is my personal projects. ~/projects/company is projects for company I'm working for. I'm kind of freelancer and sometimes work for different companies, so separation is necessary.
~/tmp is my throwaway directory for everything. I have shell function mkcdtmp which creates ~/tmp/240419 (current date) and cds into it. It's a wonderful way. I rarely clean it, because I prefer to buy big discs and keep trash around, somewhat organized. If I need something from yesterday or past month, I know where to find it. That's the most important thing I did for organizing my temp work which is plenty. Of course I can create ~/tmp/whatever if needed, it's all trash stuff.
Well, that's all, I guess. I don't use ~/Desktop. I rarely use ~/Documents, still need to find a way to organize it. Also I dump my short notes and stuff to my github repo which builds a personal website. I tried various notes software but ordinary website with markdown turned out the best way for me.
I never was able to organize my work in a very structured way, it's always piles of trash moving around which eventually turn into something usable, so instead of fighting myself I decided to make that trash organized.
Essentially my computer is disposable. Everything in ~/projects is in git. Everything in ~/tmp is not very important and more like a cache or discarded work. I'm trying to organize things in a way so restoring it from the clean state wouldn't take much time. I often reinstall OS from the scratch and I often switch between operating systems and laptops, that suits me best.
I personally would rather type "ThisIsSomeOtherProject" than "this-is-some-other-project" as advocated on the page. Approximately the same number of keypresses but fewer characters to read.
each year I make a year folder. And each project has a month + day prefix.
Sometimes I want long term projects to pop up on top, so I prefix them with 0000 (or only make the day 00).
It is simple, works on any OS. Although on Linux I do have some helper scripts.
And it is very easy to quickly make a directory and move files from downloads into this directory. Keeping the nesting only one level deep helps for the discoverability. (versus the YYYY/MM/DD pattern which uses an extra month level)
I do something similar for relevant folders, where I organize by time.
I have a top level dir where I usually have what I'm working on at the moment - think "this day" or "this week".
When it's time to archive it, outside that time window, I created a folder, using this format, for example for today:
041924
I move all the files I created on that day into there.
As a person with a typical human lifespan, I see no need to use a "2024" string - I don't think I'll live to see 2100, and anything before 2000 isn't relevant. "24" is just fine.
And that's it. Time keeps rolling on, so the number of 'archive' folders keeps increasing, but they're small in size and easy to locate.
This works really well when you're creating not-that-unique standard files every day, btw, which is my use case.
My experience with metadata is that a lot of applications don't care about its integrity but will never mess with the filename unless that is the purpose of the application.
Demo a sync program and find that all the creation dates have changed to today!
A lot of filesystems don't store file creation dates. Posix only requires last modification date (which for directories, is the last time a file/subdirectory was added, removed, or renamed), last access date, and last status change date.
Readit News
The one that upsets me the most is the default directory for go modules `~/go`. This frustrates me so much. I refused to install any go apps or use it for development for years. I've unfortunately had to give in, and it can at least be overridden by setting `GOPATH`, but it is a terrible, terrible default.
But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
I don't really ever program in golang, but whenever I write a Node.JS/Python tool that does need a user-global config file, I just write my own implementation of it:
[1]: https://pkg.go.dev/os#UserConfigDir> pollute
gah, reminds me of my pet peeve.
Insert a drive, it gets .Trashes ._Trashes .DS_Store .fseventsd .Spotlight-V100 and other nonsense everywhere.
And if you are ever trying to recover a failing drive, do not mount it on a mac.
Counterpoint:
I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in.
I got used to it and in HOME I put some of my manually installed tools like `AMD_AOCL` for the AMD AOCL libraries, `Android/android-studio` for Android Studio, `intel` for IPP/OpenAPI, etc.
Don't want it anymore? Easy rm -rf, no need to go digging for where it could be hidden.
Dead Comment
Perhaps this design makes more sense for those who prefer to keep several unrelated projects together in a single monorepo, but that's not my preference.
~/src/$host/$owner/$repo
organization structure makes a ton of sense for every project and as long as you organize all of your languages into this one tree, everything just works.
Unruly apps that benefit from UID/GID regimentation can be constrained in this way. The whole app assuming ~amtgo is also helpful.
EDIT: I also remembered this useful advice on prefixing private shell scripts in your path with a comma:
https://news.ycombinator.com/item?id=31846902
$HOME is the one directory which belongs to the user. In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
Additionally I symlink .vimrc and .gitconfig to a git repos. Then everything else in my home dir can be total garbage, it no longer matters. The big plus is that if a machine dies I can be back in business in minutes on another one.
Or absolutely moronic things like ~/.config/VSCod*/*Cache* -- really, cache in config?
If you ask me, this should all be built-in, but even then (because on many platforms, it is), people can’t help rolling their own.
Standards XKCD I guess.
In short, it scans all your programs and determines if you can configure it to respect the xdg standards. It doesn't work for everything, but most applications do seem to have such an option.
[0] https://github.com/b3nj5m1n/xdg-ninja
The `.config` folder is a big headache when trying to be strategic about backups due to apps putting gigs of session data there.
"session data" is not "config", consarnit! An app's "config" shouldn't be gigs large. Grumble grumble, grouse grouse, and so on. :^)
Thinking about multiple storage locations is effort and most users don't care about this too much anyway
My home directory is nearly empty, because all the files I work with are in OwnCloud (so the real question is: what is the directory structure in OwnCloud). Local Git repositories are on a completely separate partition.
Since KeepassXC now handles SSH keys, the keys from .ssh are now in the Keepass-file, which is on OwnCloud. That was a huge simplification, and now there is nothing in my home directory that I really care much about.
I need to be able to login and run a single command to sync to have the appropriate set of files for whatever /linux/work/personal/desktop/server I am on
For example, I never want certain environment variables to be exported to work systems, like my home vault endpoint or certain tokens.
I recently switched to home-manager from the NixOS project and it looks very promising. The Nix language is complicated but the abstraction to define all these different types of environments is exactly what I need, and git branches manage the work/personal split of file content
Pictures are better sorted with EXIF keywords. So, metadata stored in the picture itself, perhaps even in the MIME type. Thus, if a picture is family related, just tag it #family, or #personx and so on. This is why I store them in the same folder on a per-date basis. The rest is keywords edited with programs such as Adobe Bridge and so on.
As for document file name structure, text files and the like, I've used both `Date then Descrpition.txt` or `Keyword Title or Description and then Date.txt` with the date obviously being an ISO date such that `YYYY-MM-DD-hhmm` with `-hhmm` being optional, also for sorting reasons. Sometimes I like sorting on "topics" i.e. keywords or titles, but other times, such as in logs, I like the date first because it's more essential to know when you logged something, and not necessarily the topic.
You should think the date is superfluous since it's also stored in the system. However, my experience that when you move around a file, the date eventually changes, and certainly if you make a mistake along the way. Meanwhile a filename date doesn't change. Also it helps with list sorting where applicable.
I think photoprism and photostructure don't really care about your directory structure or organization, but paperless(-ngx, or whatever is the most current iteration) is "notorious" for being opinionated about organization / not wanting to respect your organization.
I used camlistore/perkeep for photos for a while, but google photos has an absolutely killer feature of knowing who is in every picture, even accounting for age: although two of my boys who are 8 years apart look really similar at similar ages, it knows who is who. I don't know if it uses facial analysis or photo metadata or what, but it has never mistaken one for the other. I don't have a reasonable way of getting that tag information out of google photos (even though I'm paying for the service).
It might be time to revisit this, though.. I don't recall whether photostructure / photoprism try to do facial recognition, but even if they don't already, I bet they'll soon be roughly on par with google photos (or good enough that I can stop depending on google for this).
That's documents and photos / videos, what about music? For better or for worse, It's been at least a ~decade since I tried keeping my own music collection as files. Are there things these days like paperless / photoprism, a library system for music that has deeper integration with the content than just "files on disk"?
I've also been on the lookout for an AI assisted duplicate finder. Like, it'll compare pictures and then rate similarity between pictures on some scale, by appearance, colour or topic, perhaps with an auto-tagging feature as well. Ofc it should also compare dates, file sizes and the usual MIME type or hash based similarities as well.
Capital names for GUI things, lowercase names for CLI things. I'd prefer ~/documents, etc, but GUI people insist on capitalizing, so whatever. Very rarely I have to mix those, so it's not a big problem.
~/dotfiles is my dotfiles directory with git. I create ~/.zshrc -> dotfiles/zshrc, etc. I don't use any software to manage that, I just create symlinks. In the past I've used ~/.dotfiles but I think that making it visible makes more sense.
~/projects is my projects directory. ~/projects/test is throwaway test projects to check things out, etc. ~/projects/my is my personal projects. ~/projects/company is projects for company I'm working for. I'm kind of freelancer and sometimes work for different companies, so separation is necessary.
~/tmp is my throwaway directory for everything. I have shell function mkcdtmp which creates ~/tmp/240419 (current date) and cds into it. It's a wonderful way. I rarely clean it, because I prefer to buy big discs and keep trash around, somewhat organized. If I need something from yesterday or past month, I know where to find it. That's the most important thing I did for organizing my temp work which is plenty. Of course I can create ~/tmp/whatever if needed, it's all trash stuff.
Well, that's all, I guess. I don't use ~/Desktop. I rarely use ~/Documents, still need to find a way to organize it. Also I dump my short notes and stuff to my github repo which builds a personal website. I tried various notes software but ordinary website with markdown turned out the best way for me.
I never was able to organize my work in a very structured way, it's always piles of trash moving around which eventually turn into something usable, so instead of fighting myself I decided to make that trash organized.
Essentially my computer is disposable. Everything in ~/projects is in git. Everything in ~/tmp is not very important and more like a cache or discarded work. I'm trying to organize things in a way so restoring it from the clean state wouldn't take much time. I often reinstall OS from the scratch and I often switch between operating systems and laptops, that suits me best.
Desktop, Dev, Downloads, Documents, Dropbox etc.
I thought about taking action about this, but as the author points about, many applications are quite opinionated on the matter.
It is simple, works on any OS. Although on Linux I do have some helper scripts. And it is very easy to quickly make a directory and move files from downloads into this directory. Keeping the nesting only one level deep helps for the discoverability. (versus the YYYY/MM/DD pattern which uses an extra month level)
I have a top level dir where I usually have what I'm working on at the moment - think "this day" or "this week".
When it's time to archive it, outside that time window, I created a folder, using this format, for example for today:
041924
I move all the files I created on that day into there.
As a person with a typical human lifespan, I see no need to use a "2024" string - I don't think I'll live to see 2100, and anything before 2000 isn't relevant. "24" is just fine.
And that's it. Time keeps rolling on, so the number of 'archive' folders keeps increasing, but they're small in size and easy to locate.
This works really well when you're creating not-that-unique standard files every day, btw, which is my use case.
I can accept YY instead of YYYY, but surely YYMMDD is preferable due to sort order.
Demo a sync program and find that all the creation dates have changed to today!