Readit News logoReadit News
teh commented on Cue Does It All, but Can It Literate?   xlii.space/cue/cue-does-i... · Posted by u/xlii
solatic · a month ago
Is there anyone out there that has actually, in the real world, realized CUE's promise of bundling type safety + data/configuration + task running in such a way that does not require wrapping it in shell scripts? Can you set up your CI/CD pipelines so that it's literally just invoking some cue cmd, and have that cmd invocation be reasonably portable?

The problem is, once you have to wrap CUE, the loss of flexibility within a special-purpose language like CUE is enough for people to ask why not just bother writing the scripts in a general purpose language with better ecosystem support. And that's a hard sell in corporate environments, even ones that find benefit in type safe languages in general, because they can just pick a general purpose language with a static type checker.

teh · a month ago
I can't fully answer your question but I did once spend about a week porting plain internal configuration to cue, jsonnet, dhall and a few related tools (a few thousand lines).

I was initially most excited by cue but the novelty friction turned out to be too high. After presenting the various approaches the team agreed as well.

In the end we used jsonnet which turned out to be a safe choice. It's not been a source of bugs and no one complained so far that it's difficult to understand its behaviour.

teh commented on NixOS is not reproducible   linderud.dev/blog/nixos-i... · Posted by u/udev4096
_fsql · a year ago
NixOS is a terrible solution rather than a reproducible one. Adding a layer of complexity to simple text files that can be easily edited and having thousands of links that are links to other links is just stupid. Filling up disk space like crazy and using more machine resources on trivial tasks is not an elegant solution. Editing configuration.nix or any other module, reading all the idiotic variable definitions that were invented to change a simple variable in a text file, is boring.

I don't know what the Nix designer (NixOS) had in mind when they thought that putting layer upon layer of complexity was a great solution. If something goes wrong in NixOS at the bottom layer, the nixos-rebuild command will produce weird errors. NixOS is an effort to make Linux complex and, in my opinion, useless. I could go on listing the shortcomings of NixOS, but I'll stop here...

teh · a year ago
I feel you (like many people) got burned by the steep learning curve. Empirically some pretty high powered companies use nix successfully. It's of course always difficult to know the counterfactual (would they have been fine with ubuntu) but the power to get SBOMs, patch a dependencies deep in the dependency stack, roll back entire server installs etc. really helps these people scale.

nixpkgs is also the largest and most up to date package set (followed by arch) so there's clearly something in the technology that allows a loosely organised group of people to scale to that level.

teh commented on Ladybird browser spreads its wings   lwn.net/Articles/976822/... · Posted by u/signa11
blowski · 2 years ago
I hear so much about Ladybird on HN, that I’m equally excited and cynical.

What is the value proposition? Is it to be another general purpose browser, so there’s more competition with Chrome / WebKit? Or to be a niche browser, that could be an alternative to Electron?

How close is it to achieving that?

teh · 2 years ago
I feel the same way. Excited to see another attempt. But it's a c++ engine so not something I would want to expose to the internet really.
teh commented on Amazon Redshift re-invented   amazon.science/publicatio... · Posted by u/belter
temuze · 4 years ago
Back at my old job in ~2016, we built a cheap homegrown data warehouse via Postgres, SQLite and Lambda.

Basically, it worked like this:

- All of our data lived in compressed SQLite DBs on S3.

- Upon receiving a query, Postgres would use a custom foreign data wrapper we built.

- This FDW would forward the query to a web service.

- This web service would start one lambda per SQLite file. Each lambda would fetch the file, query it, and return the result to the web service.

- This web service would re-issue lambdas as needed and return the results to the FDW.

- Postgres (hosted on a memory-optimized EC2 instance) would aggregate.

It was straight magic. Separated compute + storage with basically zero cost and better performance than Redshift and Vertica. All of our data was time-series data, so it was extraordinarily easy to partition.

Also, it was also considerably cheaper than Athena. On Athena, our queries would cost us ~$5/TB (which hasn't changed today!), so it was easily >$100 for most queries and we were running thousands of queries per hour.

I still think, to this day, that the inevitable open-source solution for DWs might look like this. Insert your data as SQLite or DuckDB into a bucket, pop in a Postgres extension, create a FDW, and `terraform apply` the lambdas + api gateway. It'll be harder for non-timeseries data but you can probably make something that stores other partitions.

teh · 4 years ago
I've looked into this but saw hugely variable throughput, sometimes as little as 20 MB / second. Even if full throughput I think s3 single key performance maxes out at ~130 MB / second. How did you get these huge s3 blobs into lambda in a reasonable amount of time?
teh commented on Security Engineering – A Guide to Building Dependable Distributed Systems   cl.cam.ac.uk/~rja14/book.... · Posted by u/go-red-team
teh · 6 years ago
This is a great book!

After reading it I found it much harder to enjoy movies showing bad security though (such has heists, nuclear anything, ..).

E.g. from the book I learned about the IAEA recommendations for safekeeping nuclear material [1], and it's pretty clear that smart people spent some time thinking about the various threats.

Anyway, rambling. It's a great and very entertaining book, go read it!

[1] https://www-pub.iaea.org/MTCD/Publications/PDF/Pub1481_web.p...

teh commented on How Does Lazy Evaluation Work in Haskell? (2014)   apfelmus.nfshost.com/arti... · Posted by u/gbrown_
amelius · 6 years ago
teh · 6 years ago
I just spent some time implementing a lazy VM: Note also the push/enter VS eval/apply implementation change in GHC described in [1].

[1] https://www.microsoft.com/en-us/research/publication/make-fa...

teh commented on Over 150k botanical illustrations enter public domain   hyperallergic.com/541381/... · Posted by u/bookofjoe
supermatt · 6 years ago
And they will be on getty images with a €475 royalty free license in 5...4....3...
teh · 6 years ago
I think this is a common misunderstanding. The images are in the public domain. Nothing stops Getty (or you, or anyone) from selling them, even though you can just use them for free.

The value-add service that Getty offers is legal indemnification, i.e. they cover the legal costs if the image turns out to be copyrighted after all. To offer this service they spend some time and money upfront to research images' copyright status.

Whether you think that's good value for money is up to you.

teh commented on Guide to the London Startup Ecosystem   startupsoflondon.com/lond... · Posted by u/Ozzie-D
chrisseaton · 6 years ago
Have you looked jobs boards like Oxford Knight?

I often see cash compensation >300k on there.

Here's one for precisely 150k right now https://oxfordknight.co.uk/jobs/big-data-engineer/.

These aren’t junior jobs, and they often require you to be in central London so your money won’t go as far, but these jobs do exist.

teh · 6 years ago
More often than not recruiters (external and in-house) make up large numbers to get you to reply. Here is a verbatim quote from a mail I got last year:

> For the right candidate year 1 comp will be up to £500k.

After going through a three hour coding test, phone screen + onsite it turned out to be 120k + smallish bonus. Maybe I'm not the right candidate but most likely 500k was never on the table.

It's not the first time this has happened to me or people I know either. What I'm getting at is that I'd take the Oxford Knight compensation numbers with a pinch of salt. Those 300k jobs _do_ exist but nowhere near as many as in the US.

Additionally levels.fyi or glassdoor data doesn't support a large number of 300k jobs in London.

teh commented on Secure EcmaScript, a runtime for running third-party code safely   github.com/Agoric/SES... · Posted by u/rewq4321
teh · 6 years ago
The original SES doesn't seem to do anything to prevent meltdown/spectre attacks [1]

This version removed direct access "Date" [2] but I'm not sure I'd trust any code running in the same process space given how hard it is to fix spectre in general.

[1] https://github.com/google/caja/wiki/SES#current-date-and-tim...

[2] https://github.com/Agoric/SES/tree/master/demo#taming-dateno...

u/teh

KarmaCake day731July 5, 2010View Original