That's not how that works. Same reason we don't tax unrealized gains (unless you are Norway).
And The Netherlands
Readit Newsmarijnz0r commented on OpenAI Needs $400B In The Next 12 Months wheresyoured.at/openai400... · Posted by u/chilipepperhott
marijnz0r commented on Async await: the worst thing to happen to programming? andrewzuo.com/async-await... · Posted by u/thunderbong
I think the idea behind c# async await was that you could use multple threads without having to worry about the details. It seemed redundant to me to have awaits in a web server. That runs multiple threads already, why does my code have to be async now? I hated it too.
Although it seems async/await is based on multiple threads, this is not the case. To learn about this, read this blog post: https://blog.stephencleary.com/2013/11/there-is-no-thread.ht...
Please let me know what you think.
marijnz0r commented on Async await: the worst thing to happen to programming? andrewzuo.com/async-await... · Posted by u/thunderbong
I think it’s nice for server stuff but on a desktop app it’s a pain to deal with. MS definitely went overboard with a lot of APIs being async only. A lot of people don’t seem to understand that async/await is still multithreading so in a desktop app they tend to mess up. Not sure how it’s in mobile.
Although it seems async/await is based on multithreading, this is not the case. To learn about this, read this blog post: https://blog.stephencleary.com/2013/11/there-is-no-thread.ht...
Please let me know what you think.
Try and write any complex SQL as a series of semantically meaningful CTEs. Test each part of the CTE pipeline with an in.parquet and an expected_out.parquet (or in.csv and out.csv if you have simple datatypes, so it works better with git). And similarly test larger parts of the pipeline with 'in' and 'expected_out' files.
If you use DuckDB to run the tests, you can reference those files as if they were tables (select * from 'in.parquet'), and the tests will run extremely fast
One challenge if you're using Spark is that test can be frustratingly slow to run. One possible solution (that I use myself) is to run most tests using DuckDB, and only e.g. the overall test using Spark SQL.
I've used the above strategy with PyTest, but I'm not sure conceptually it's particularly sensitive to the programming language/testrunner you use.
Also I have no idea whether this is good practice - it's just something that seemed to work well for me.
The approach with csvs can be nice because your customers can review these files for correctness (they may be the owners of the metric), without them needing to be coders. They just need to confirm in.csv should result in expected_out.csv.
If it makes it more readable you can also inline the 'in' and 'expected_out' data e.g. as a list of dicts and pass into DuckDB as a pandas dataframe
One gotya is SQL does not guarantee order so you need to somehow sort or otherwise ensure your tests are robust to this
I really like this answer because it tests CTEs in a modular way.
One question I have is: how do you test a the CTE with an in.csv and out.csv without altering the original CTE?
Currently I have a chain of multiple CTEs in a sequence, so how would I be able to take the middle CTE and "mock" the previous one without altering the CTE actually used in production? I prefer not to maintain a CTE for the live query and the same CTE adapted for tests.
marijnz0r commented on Ask HN: Where to buy cheap but good quality glasses in Europe? · Posted by u/tsingy
https://www.aceandtate.com
is an option as well.
A book about this topic: Hans Rosling - Factfulness: Ten Reasons We're Wrong About the World – and Why Things Are Better Than You Think
https://www.goodreads.com/book/show/34890015-factfulness
I can highly recommend it.
marijnz0r commented on Good times create weak men (2019) tonsky.me/blog/good-times... · Posted by u/marijnz0r
Isn't this another variant of the trends/emergent phenomena that Peter Turchin's "Cliodynamics" also illustrates?
It might be, but the originally linked article is more about tech and not so much about politics on a national or global level.
marijnz0r commented on Good times create weak men (2019) tonsky.me/blog/good-times... · Posted by u/marijnz0r
See also the discussion of that meme at https://acoup.blog/2020/01/17/collections-the-fremen-mirage-...
Wow, that's quite a discussion. I'll be sure to make time to read it in its entirety.