PSA: If you are using AWS, you should be using Session Manager for remote access to your EC2 instances if possible - it is free and trivial to enable. Azure offers Azure Bastion, which is exactly what it says. Both of these can keep hostile SSH connections completely away from your servers, and also help to aggregate access logs.
Readit Newssjellis commented on SSHGuard sshguard.net/... · Posted by u/tosh
sjellis commented on Shell script best practices, from a decade of scripting things sharats.me/posts/shell-sc... · Posted by u/sharat87
Shell and SQL make you 10x productive over any alternative.
Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes.
What Python is to Java, Shell is to Python. It speeds you up several times. I started using inline 'python -c' more often than the python repl now as it stores the command in shell history and it is then one fzf search away.
While neither Shell or SQL are perfect, there have been many ideas to improve them and for sure people can't wait for something new like oil shell to get production ready, getting the shell quoting hell right, or somebody fixing up SQL, bringing old ideas from Datalog and QUEL into it, fixing the goddamn NULL joins, etc.
But honestly, nothing else even comes close to this 10x productivity increase over the next best alternative. No, Thank you, I will not rewrite my 10 lines of sh into python to explode it into 50 lines of shuffling clunky objects around. I'll instead go and reread that man page how to write an if expression in bash again.
I expect nushell to massively change how I work:
It's a shell that is actually built for structured data, taking lessons learned from PowerShell and others.
And let's not forget to be grateful towards for-profit companies that contribute to open source.
A lot of the work required to get games working transparently on Wine was thanks to Valve, who released Proton as open source, along with heavy work on DXVK to simulate DirectX, etc.
Of course they did it because it benefits them, but still they could have kept it private and it would have boosted only Steam sales. Instead it's a win for them AND for the community, an example that should be followed by other companies.
What a time to be alive indeed!
> And let's not forget to be grateful towards for-profit companies that contribute to open source.
The main sponsor of WINE is CodeWeavers, who have been paying folks that work on WINE for a very long time, and are involved in Proton development:
They sell a supported version of WINE for Linux, macOS and ChromeOS, as well as providing engineering services to clients like Valve.
sjellis commented on Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet github.com/roapi/roapi/bl... · Posted by u/houqp
This is cool...Totally reminded me about several tools pop up on HN every now and then in the past for similar task so i did a quick search:
clickhouse-local - https://news.ycombinator.com/item?id=22457767
q - https://news.ycombinator.com/item?id=27423276
textql - https://news.ycombinator.com/item?id=16781294
simpql- https://news.ycombinator.com/item?id=25791207
We need a benchmark i think..;)
I am currently evaluating dsq and its partner desktop app DataStation. AIUI, the developer of DataStation realised that it would be useful to extract the underlying pieces into a standalone CLI, so they both support the same range of sources.
dsq CLI - https://github.com/multiprocessio/dsq DataStation desktop app - https://datastation.multiprocess.io/
Two alternative CLI tools that I looked at:
sq - https://sq.io/ octosql - https://github.com/cube2222/octosql
Honourable mentions to:
Miller - https://miller.readthedocs.io/en/latest/index.html Dasel - https://daseldocs.tomwright.me/
These don't use SQL.
VisiData is also amazing for data navigation, although it requires some effort to get the model:
sjellis commented on Making pip installs a little less slow pythonspeed.com/articles/... · Posted by u/zdw
Pip has had ton of development in the last few years, and continues to do so. I'd say stick with pip, unless there's a specific problem you're addressing with pipenv or Poetry.
However, like a sibling comment, I've also heard god things about Poetry. Probably worth giving it a spin somewhere and seeing if my initial thought still holds.
I've been experimenting with Hatch:
The USP of Hatch is that it uses the latest generation of Python standards and tech under the covers, so you have a unified tool that's less quirky than previous ones. Poetry and pipenv predate some of the improvements in Python packaging, so had to develop some things in their own way.
sjellis commented on Managing Python Versions with Pyenv thepythoncorner.com/posts... · Posted by u/EntICOnc
sjellis commented on Managing Python Versions with Pyenv thepythoncorner.com/posts... · Posted by u/EntICOnc
sjellis commented on Jeffrey Snower was originally demoted over PowerShell twitter.com/jsnover/statu... · Posted by u/nocture
Question for the more seasoned Mac/Linux folks on HN:
Powershell has been a gift in getting things accomplished in Windows. But one thing I've also come to enjoy using it for is quick and dirty interactions and scripting with REST APIs.
Things like Invoke-WebRequest, ConvertTo-Json/ConvertFrom-Json, Export-CSV, piping between almost everything, and the results being passed around like an object that I can use SQL-ish Select queries against, etc.
I find myself feeling like I'm missing something helpful when I'm on a Mac/Linux device. Is there a similar go to in the Mac/Linux world for this type of stuff? My current thinking is that I either have to start picking up Python or building a mental map of wget/awk/grep type commands that can be chained together for a similar effect, but I thought it best to try to learn from the wisdom of others first :-)
There are a few projects that take inspiration from PowerShell, but they haven't got mass adoption. I have hopes that Nushell will be a success when it gets to 1.0:
sjellis commented on Experience Report: 6 months of Go typesanitizer.com/blog/go... · Posted by u/eatonphil
My disappointment with Rust, that in no way diminishes how much I enjoy using it in my own time, is that I have a hard time recommending it for microservices, which are arguably an average project at an average company. The ecosystem just doesn't feel as fleshed out or complete as in Go. It's a shame because there's libraries in Rust that I adore like clap, serde, and diesel but when I last wanted to write something that integrates with AWS, I found a deprecated unofficial crate and a non-production-ready official crate from AWS.
I don't know whether to attribute this disappointment to the breadth of what Rust can be used for and the difficulty in doing all of them well, or a lack of wider/corporate buy-in for these use cases. It's a pity, because after getting past the initial learning curve I struggle to find anything wrong with the language itself.
FWIW, AWS now have an official SDK for Rust in beta:
https://aws.amazon.com/sdk-for-rust/
There are people at Microsoft working on a Rust SDK for Azure, but it's explicitly a volunteer effort with no support guarantees at the moment:
https://github.com/Azure/azure-sdk-for-rust
These days I probably would not look at any language that did not have SDKs for AWS and Azure, for similar reasons.
It is time to give a shout out to my favorite note taking app: Trilium https://github.com/zadam/trilium
Pros:
1. Data is saved in SQlite. I am at 33k notes and it springs open instantaneously.
2. Notes can be arranged into arbitrarily deep tree. Single note can be placed into multiple places in the tree. (Think soft-links)
3. WYSIWYG support (CKEditor)
4. Tags, advanced scripting features
5. Other ususal wiki stuff like backlinks, note-map etc
1. Electron.
2. Data is saved in SQlite, not plain text.I use Joplin, which has a similar feature set but also mobile apps:
It supports various types of file stores for syncing between devices. I've used OneDrive and WebDAV. The project has also recently launched a cloud service for people who want to sync between devices but don't want to set up a network file store.