jiggawatts (u/jiggawatts)

jiggawatts commented on Why was Apache Kafka created? bigdata.2minutestreaming.... · Posted by u/enether

erulabs · a day ago

Kafka's ability to ingest the firehose and present it as a throttle-able consumable to many different applications is great. If you're thinking "just use a database", it's worth noting that SQL databases are _not well suited_ to drinking from a firehose of writes, and that distributed SQL in 2012 was not a thing. Kafka was one of the first systems that fully embraced the dropping of the C from CAP theorem, which was a big step forward for web applications at scale. If you bristle at that, know that using read-replicas of your postgres database present the same correctness problems.

These days though, unless I was at Fortune 100 scale, I'd absolutely turn to Redis Cluster Streams instead. So much simpler to manage and so much cheaper to run.

Also I like Kafka because I met two pretty Russian girls in San Francisco a decade back and the group we were in played a game where we described what the company we worked for did in the abstract, and then tried to guess the startup. They said "we write distributed streaming software", I guessed "confluent" immediately. At the time confluent was quite new and small. Fun night. Fun era.

jiggawatts · 9 minutes ago

> that SQL databases are _not well suited_ to drinking from a firehose of writes

Now I’m wondering if we’re all overthinking this when we could just use rendezvous hashing and a bunch of database servers with a heap table called “eventlog” and be done with it…

jiggawatts commented on Starship's Tenth Flight Test spacex.com/launches/stars... · Posted by u/d_silin

ivape · 30 minutes ago

When these rockets crash, is it because their digital simulations are inaccurate? Why do they need data from the actual test is the question instead of just relying on a bullet proof simulation?

jiggawatts · 16 minutes ago

Some aspects are just too complex to simulate. The fluid dynamics of nearly supersonic flows of cryogenic liquids that suddenly turn into white hot gas is… intractable.

The failures Starship had were often to do with simpler engineering bugs that they’ve been ironing out, such as: leaks in piping caused by violent shaking, explosive gases accumulating in closed spaces, filters getting clogged by ice forming in the cryogenic tanks, and burn-through of an experimental heat shield design at moving joints.

jiggawatts commented on Starship's Tenth Flight Test spacex.com/launches/stars... · Posted by u/d_silin

vFunct · 23 minutes ago

Seriously. NASA had a reusable orbital rocket 40 years ago. Space-X still only has reusable boosters.

I was mostly impressed by the materials science of the space shuttle tiles, even though they’re expensive.

jiggawatts · 19 minutes ago

Sure, but it was burning congressional pork as fuel and cost only the occasional human sacrifice.

jiggawatts commented on The cost of interrupted work (2023) blog.oberien.de/2023/11/0... · Posted by u/_vaporwave_

kaffekaka · 16 hours ago

Does this mean you as a pair were as productive as both of you individuals combined? Or that the pair was as productive as one individual?

Pair programming is twice as expensive so it needs to be twice a productive (quality, LOC, whatever) to make sense I guess.

jiggawatts · 15 hours ago

Two of us at one keyboard were as productive as the two of us separately combined.

I figured this was because typically while one person was coding the other would be researching. If you’re by yourself those are serial activities instead of parallel and the total workload is the same.

jiggawatts commented on The cost of interrupted work (2023) blog.oberien.de/2023/11/0... · Posted by u/_vaporwave_

SoftTalker · 20 hours ago

If only I wouldn’t prefer stabbing myself in the leg with a rusty knife over pair programming.

jiggawatts · 19 hours ago

When pair programming was a fad in the early 2000s, I tried it with a coworker for a security-critical piece of code that needed two pairs of eyes on it.

It felt horrendously unproductive to have two people at one keyboard but we compared commit rates and the surprising result was that we produced the same rate of changes as working separately.

jiggawatts commented on The contrarian physics podcast subculture timothynguyen.org/2025/08... · Posted by u/Emerson1

cauch · 2 days ago

I'm surprised by what you say, it is not at all my experience. Are you sure you are not over-interpreting what your friend said, or that your friend's experience was not unusual?

1) People at CERN publish papers in "normal" physics journals, which do the usual peer review. Few articles that I've myself per-reviewed were not from my own experiment. There is, of course, also an internal reviewing for each collaboration, but it is to improve the quality and something totally natural and obvious if you want to have a collaboration (by definition, a collaboration is a place where people read each other work and feedback to each others). But it is totally different from "the work is only reviewed by the collaboration".

2) I've worked ~5 years in one experiment, and ~5 years in another, and I did not notice any different terminology. In both experiments, I've very rapidly met and learned the name of people of other experiments working on similar subject. I don't know any workshop or conference where the invited scientists are not from different experiment. During these events, there are a lot of exchanges.

3) What is true, and it is maybe the reason of your misunderstanding, is that you are strongly advised to not share non-cross-checked material outside of the collaboration. The goal is to avoid biasing the independent experiments: if you notice a strange phenomena that will later turn out to be a statistical fluctuation or if you use a new methodology that will later turn out to have unnoticed systematical biases, if you mention this to the other experiment, you will "contaminate" them: they may focus their research or adopt the flawed methodology. But this is only for non-cross-checked and it does not make any sense to pretend that it has a negative impact (a lot of scientists, in collaboration or not, towards all history, don't like to share their preliminary results before they acquired a good confidence that what they saw it reliable).

4) Do you have example of things that one could not understand while it was done down the hall from them? I don't recall "not being able to understand" (the point of a publication is to explain, so people care about making it understandable). I do recall "harder to understand", but it was often from people from the same collaboration, and the reason was because of they needed to use some mathematical tools I did not know and that there was not really any other way.

I'm sure there are cases where two groups end up diverging and it makes the collaboration more challenging. But I really doubt it is not something exceptional, and that everyone in the collaborations will try to mitigate.

Your comment makes me wonder to which extend the outsiders of CERN don't have plenty of crazy myths totally disconnected from the reality. I guess it is a good example why people like Hossenfelder are a problem: they feed on these myths and cultivate them.

jiggawatts · 2 days ago

> journals, which do the usual peer review.

They don't though! They farm it out to expert physicists, which in the case of CERN research almost certainly also work at CERN.

> Few articles that I've myself per-reviewed were not from my own experiment.

But were they from CERN?

> Do you have example

This was a few years ago, it was a comment here on HN, but it would be hard to dig it up without an AI reading through everything.

jiggawatts commented on Does MHz Still Matter? ubicloud.com/blog/does-mh... · Posted by u/furkansahin

ytreem · 2 days ago

>These licenses cost millions per year for couple 16 core seats

The ROI on hiring a professional overclocker to build, tune and test a workstation is probably at least break even. As long as the right checksums are in place, extreme OC is just a business writeoff.

jiggawatts · 2 days ago

I had a conversation like this with a business that had been around for decades and suddenly grew 100x because some market they were in “took off”. They had built up decades of integration with a legacy database that was single threaded and hence they couldn’t scale it.

Given the urgency and the kind of money involved, I offered to set up a gaming PC for them using phase change cooling. Sadly they just made the staff work longer hours to catch up with the paperwork.

jiggawatts commented on The contrarian physics podcast subculture timothynguyen.org/2025/08... · Posted by u/Emerson1

gwerbret · 3 days ago

> There is a simple test the public can use for any scientific model: does it make accurate predictions, or not? You don't need to understand how a model works to test that.

It's quite obvious from your position on this matter that you're not a practicing scientist, so it's very unfortunate that your position is so assertive, as it's mostly wrong.

To understand the predictions, as it were, you do have to understand the experiments; if you don't, you have no way of knowing if the predictions actually match the outcomes. Most publications involve some form of hypothesis-prediction-experiment-result profile, and it is the training and expertise (and corroboration by other experiments, and time) that help determine which of those papers establish new science, and which ones go out with last week's trash. The findings in these areas are seldom accessible until the field is very advanced and/or in practical use, as with the example of GPS you gave elsewhere.

> The biggest problem I see with "establishment" science today is that it doesn't work this way. There is no mechanism for having an independent record that the public can access of predictions vs. reality.

There is; it's called a textbook.

jiggawatts · 3 days ago

An example of this ideal can go horribly wrong is CERN.

There's one apparatus (of each type) and each "experiment" ends up with its own team. Each team develops their own terminology, publishes in one set of papers, and the peer reviews are by... themselves.

I don't work at CERN, but that criticism was from someone who does.

They were complaining that they could not understand the papers published by a team down the hall from them. Not on some wildly unrelated area of science, but about the same particles they were studying in a similar manner!

If nobody else can understand the research, if nobody else can reproduce it, then it's not useful science!

Note that this isn't exactly the same as Sabine's criticism of CERN and future supercolliders, but it's related.

jiggawatts commented on Modern CI is too complex and misdirected (2021) gregoryszorc.com/blog/202... · Posted by u/thundergolfer

arunix · 4 days ago

Is there something about .NET that makes this easier?

jiggawatts · 3 days ago

It's like Java in that it tends towards the "build once, run anywhere" style.

Also, Windows has a consistent user-mode API surface (unlike Linux), so a .NET app that runs on a desktop will run on server almost always.

The same cannot be said for someone developing on a "UNIX-like" system such a MacOS and then trying to run it on Ubuntu... or RedHat. Alpine? Shit...

jiggawatts commented on AWS in 2025: Stuff you think you know that's now wrong lastweekinaws.com/blog/aw... · Posted by u/keithly

awongh · 4 days ago

This is exactly what I use LLMs for. To just read the docs for me and pull out the base level demo code that's buried in all the AWS documentation.

Once I have that I can also ask it for the custom tweaks I need.

jiggawatts · 3 days ago

Back when GPT4 was the new hotness, I dumped the markdown text from the Azure documentation GitHub repo into a vector index and wrapped a chatbot around it. That way, I got answers based on the latest documentation instead of a year-old LLM model's fuzzy memory.

I now have the daunting challenge of deploying an Azure Kubernetes cluster with... shudder... Windows Server containers on top. There's a mile-long list of deprecations and missing features that were fixed just "last week" (or whatever). That is just too much work to keep up with for mere humans.

I'm thinking of doing the same kind of customised chatbot but with a scheduled daily script that pulls the latest doco commits, and the Azure blogs, and the open GitHub issue tickets in the relevant projects and dumps all of that directly into the chat context.

I'm going to roll up my sleeves next week and actually do that.

Then, then, I'm going to ask the wizard in the machine how to make this madness work.

Pray for me.