Readit News logoReadit News
suntipa commented on Counterexamples in Type Systems: programs that crash, segfault or explode (2021)   counterexamples.org/intro... · Posted by u/nequo
posed · 2 years ago
What’s your product if you don't mind sharing? Curious.
suntipa · 2 years ago
The machinery is large-scale ML and discrete optimization over large/complex data structures, designed for a specific vertical industry. These algorithms run for hours and lazy eval was an engineering disaster. Our team had experienced Haskell developers who fought complicated space leaks night and day.

0/10 would not recommend.

suntipa commented on Counterexamples in Type Systems: programs that crash, segfault or explode (2021)   counterexamples.org/intro... · Posted by u/nequo
valcron1000 · 2 years ago
And what are you using now?
suntipa · 2 years ago
The heavy numerical work is in julia, python, R, and a little custom c++ where necessary. Team productivity has gone way up.
suntipa commented on Counterexamples in Type Systems: programs that crash, segfault or explode (2021)   counterexamples.org/intro... · Posted by u/nequo
kccqzy · 2 years ago
I don't doubt your experience, but in my experience judicious use of bang patterns and StrictData are enough to prevent those issues. It's no different from debugging and avoiding memory leaks in other languages. It just takes different skill sets that are more difficult to find.
suntipa · 2 years ago
Bang and StrictData are fine locally but things get enormously worse when that data flows into deeply-layered libraries that I don't own.

Then, I need to understand and modify the evaluation behavior of a deeply-layered external software stack, including its crazy type-level magic, all before a looming deadline.

Seriously, how is this good for my sanity or career?

suntipa commented on Counterexamples in Type Systems: programs that crash, segfault or explode (2021)   counterexamples.org/intro... · Posted by u/nequo
kccqzy · 2 years ago
So?

I tested my Python app on a list with ten elements and it works fine. But in production I unexpectedly received a list with a million elements! It crashes! Or more likely, it repeatedly receives a short list but still runs out of memory due to a memory leak I hadn't considered.

Python doesn't prevent a bad software engineer from writing bad code. Try to sell that.

suntipa · 2 years ago
Lazy eval opens the floodgates for complex space leaks. Suppose you have a complicated algorithm that runs until it reaches mathematical convergence. In a lazy language like Haskell, those cycles of iteration/recursion can accumulate thunks and crash. Thunk accumulation and crashing can depend on input that you can't control or know in advance.

How do I know this? Because our big Haskell app died this way on live customer data and became a fiasco for my employer. We dumped Haskell and never looked back.

suntipa commented on Counterexamples in Type Systems: programs that crash, segfault or explode (2021)   counterexamples.org/intro... · Posted by u/nequo
kccqzy · 2 years ago
There's a big difference between crashing as in segfaults and crashing as in I should have handled an exception but I chose not to. I can trigger a crash by dividing by zero and not handling it in almost any language.

"Nobody knew why" could mean bad code, communication breakdown or a host of things. You can make code puzzling in any language.

suntipa · 2 years ago
Lazy evaluation also implies that Haskell programs can crash based on their input. That doesn't imply big input, simply that the input caused a space leak the author hadn't considered.

Try selling that to your boss who runs a mission-critical app.

u/suntipa

KarmaCake day4June 6, 2023View Original