Readit News logoReadit News
legorobot commented on Designing Software in the Large   dafoster.net/articles/202... · Posted by u/davidfstr
Warwolt · a month ago
While the tools you talk about sound interesting, to me this was more about an in-principle possible measurement rather than something we'd actually carry out.

I think stating that "more stuff" in the program code and in the spec leads to more stuff to keep track of, and so we want to minimize complexity to maintain tractability?

legorobot · a month ago
I think that's reasonable :) More stuff is more stuff, no matter how simple/complex the aspects of the code and reasoning for why it is that way.
legorobot commented on Designing Software in the Large   dafoster.net/articles/202... · Posted by u/davidfstr
Warwolt · a month ago
I mean, we can definitively talk about simplicity/complexity in a fairly easy way when it comes to mathematical structures or data structures in my opinion.

For instance, a binary tree that contains just a root node is clearly simpler than a binary tree with three nodes, if we take "simple" to mean "with less parts" and complex to mean "with more parts". Similarly, a "molecule" is more complex than an "atom".

This is a useful definition, I think, because when we write computer programs they always written in some programming language, with a syntax that yields some kind of abstract tree, so ultimately we'll always have _some_ kind of graph-like nature to the computer program, both syntactically and semantically, and surely graphs also permit the same kind of complexity metrics.

I'm not saying measuring the number of nodes is _the_ way of getting at complexity, I'm just pointing out that there's no real difficulty in defining it.

Complexity means more stuff, and we simply take it as a premise that we can only fit so much stuff in our head at the same time.

legorobot · a month ago
I think my issue with this generalization is assuming the code itself is where complexity is measured and applied.

For example, the Quake Fast Inverse Square Root[1] takes into account nuances in how floating point numbers can be manipulated. The individual operations/actions the code takes (type casts, bit shifts, etc.) are simple enough, but understanding how it all comes together is where the complexity lies, vs just looking at the graph of operations that makes up the code.

Tools like Rubocop for Ruby take an approach like you mention, measuring cyclomatic and branch complexity in your code to determine a mathematical measurement of the complexity of that code. Determining how useful this is, is another conversation I think. I usually find enforcing rules around that code complexity measurement against your code to be subjective.

Going back to the article, the visualization of with vs without abstractions can cover aggregating the mathematical representation of the code and how to tackle complexity. Abstractions lets you take a group of nodes and consider them as a single node, allowing you to build super-graphs covering the underlying structure of each part of the program.

> both syntactically and semantically

I do want to cover semantic program complexity at some point as a deeper discussion. I find that side to me to be quite interesting. How to measure it too.

[1]: https://en.wikipedia.org/wiki/Fast_inverse_square_root

legorobot commented on Designing Software in the Large   dafoster.net/articles/202... · Posted by u/davidfstr
Warwolt · a month ago
I found "A philosophy of software design" to be a well intended but somewhat frustrating book to read.

It seemingly develops a theory of software architecture that is getting at some reasonable stuff, but does so without any reference _at all_ to the already rich theories for describing and modeling things.

I find software design highly related to scientific theory development and modeling, and related to mathematical theories like model theory, which give precise accounts of what it means to describe something.

Just taking the notion of "complexity". Reducing that to _just_ cognitive load seems to be a very poor analysis, when simple/complex ought to deal with the "size" of a structure, not how easy it is to understand.

The result of this poor theoretical grounding is that what the author of A Philosophy of Software Design presents feels very ad-hoc to me, and I feel like the summary presented in this article similarly feels ad-hoc.

legorobot · a month ago
> Just taking the notion of "complexity". Reducing that to _just_ cognitive load seems to be a very poor analysis, when simple/complex ought to deal with the "size" of a structure, not how easy it is to understand.

Preface: I'm likely nitpicking here; the use of "_just_" is enough for me to mostly agree with your take.

Isn't the idea that the bulk of complexity IS in the understanding of how a system works, both how it should work and how it does work? We could take the Quake Fast Inverse Square Root code, which is simple in "size" but quite complex on how it actually achieves its outcome. I'd argue it requires comments, tests, and/or clarifications to make sense of what its actually doing.

How do we measure that complexity? No idea :) But I like to believe that's why the book takes a philosophical approach to the discussion.

I agree the arguments in the book largely "make sense" to me but I found myself finding it a little hand-wavey on it actually proving its points without concrete examples. I don't recall there being any metrics or measurements on improvement either, making it a philosophical discussion to me and not a scientific exercise.

legorobot commented on ClickHouse raises $350M Series C   clickhouse.com/blog/click... · Posted by u/caust1c
adrian17 · 3 months ago
The majority of our queries have joins (plus our core logic often depends on fact table expansion with `arrayJoin()`s) before aggregations and we're doing fine. AFAIK whenever we hit memory issues, they are mostly due to high-cardinality aggregations (especially with uniqExact), not joins. But I'm sure it can depend on the specifics.
legorobot · 3 months ago
Definitely agree with this, I think ClickHouse can do a lot with joins if you don't implement them naively. Keeping the server up-to-date is a part of it too.

They've made strides in the last year or two to implement more join algorithms, and re-order your joins automatically (including whats on the "left" and "right" of the join, relating to performance of the algorithm).

Their release notes cover a lot of the highlights, and they have dedicated documentation regarding joins[1]. But we've made improvements by an order-of-magnitude before by just reordering our joins to align with how ClickHouse processes them.

[1]: https://clickhouse.com/docs/guides/joining-tables

legorobot commented on Some Go web dev notes   jvns.ca/blog/2024/09/27/s... · Posted by u/tosh
tizzy · a year ago
There is definitely space for an opinionated set of libraries and boiler plate code for golang projects like this.

Having said that, I’d bet that the go community consensus is that you build one out yourself and reuse it. So most times I end up copy and pasting the same logic rather than recreating.

legorobot · a year ago
100% this. I have a set of commonly-used code in a repository we use at work. AuthN/AuthZ, code specific to our infrastructure/architecture, common http middlewares, error types, DB wrapper, API clients, OpenAPI Server generation, etc.

However, my personal projects have a different set of code I reuse (more CLI- and tool-heavy focus), and I'm sure other environments would have an entirely different set of reused code.

On the opinionated library side of things, I did follow LiveBud for a while, and Go-Micro but haven't really sat well with the experiences from those, given how they lock you in to different aspects of your stack.

legorobot commented on Mountpoint – file client for S3 written in Rust, from AWS   github.com/awslabs/mountp... · Posted by u/ranman
BrianHenryIE · 2 years ago
JungleDisk was backup software I used ~2009 that allowed mounting S3. They were bought by Rackspace and the product wasn't updated. Seems to be called/part of Cyberfortress now.

Later I used Panic's Transmit Disk but they removed the feature.

Recently I'd been looking at s3fs-fuse to use with gocryptfs but haven't actually installed it yet!

https://github.com/s3fs-fuse/s3fs-fuse

https://github.com/rfjakob/gocryptfs

legorobot · 2 years ago
We've used the s3fs-fuse library for a while at work for SFTP/FTP server alternatives (AWS wants you to pay $150+/server/month last I checked!) and it's worked like a dream. We scripted the setup of new users via a simple bash script and the S3 CloudWatch events for file uploads is a dream. Its been pretty seamless and hasn't caused many headaches.

We've had to perform occasional maintenance but its operated for years with no major issues. 99% are solved with a server restart + a startup script to auto-re-mount s3fs-fuse in all the appropriate places.

Give them a try, I recommend it!

legorobot commented on Show HN: Ory Kratos – Open-source identity server written in Go   github.com/ory/kratos... · Posted by u/nottennis
skrtskrt · 3 years ago
I learned so much about AuthN and AuthZ from reading Ory code and docs.

Sometimes it seems like Ory is the only web auth stuff on the internet that’s intended for you to understand how the whole system works, rather than telling you just enough to get you to use/buy their proprietary software (Auth0, Okta, etc).

I read a few books as well but they were extremely poorly done.

I eventually also read the entire OpenId Connect spec as well, which was enlightening.

I’d love to hear of other good resources - I am happy to pay for good stuff as long as it’s mot selling me service.

Thank you!

legorobot · 3 years ago
Same experience for me too. I learned a lot, and integrating OAuth into a platform was pretty painless, including importing and managing users.

It was the first time I felt like I can wrangle the OAuth spec and implementation, and really understand all the ways in which things can go wrong without proper care and expertise into how an OAuth server can be attacked, and how to mitigate those issues the right way.

legorobot commented on Go nulls and SQL   ente.io/blog/tech/go-null... · Posted by u/mnvrth
psanford · 3 years ago
The nice thing about the Null* type is they can reduce the number of allocations done on the heap and thus also reduce total GC your program needs to do.
legorobot · 3 years ago
I haven't thought about it that way! I've not hit the GC as a performance wall when it comes to accessing nullable DB values yet. Thanks for the insight.
legorobot commented on Go nulls and SQL   ente.io/blog/tech/go-null... · Posted by u/mnvrth
eknkc · 3 years ago
So, to avoid checking for null, you'll check for `NullString.Valid` now? The string pointer is a part of the language. You can pass it around, expose as part of a library etc. And it conveys the intent perfectly.

I have no idea what is the issue here?

legorobot · 3 years ago
I think the idea is `sql.NullString` can be used to have a SQL NULL still be an empty string in Go (just avoid checking the `Valid` field, or cast to string -- no check necessary).

It seems like the intent of a string pointer vs. `sql.NullString` is their goal with this type anyways[1]?

In practice I've used a string pointer when I need a nullable value, or enforce `NOT NULL` in the DB if we can't/don't want to handle null values. Use what works for you, and keep the DB and code's expectations in-sync.

[1]: https://groups.google.com/g/golang-nuts/c/vOTFu2SMNeA/m/GB5v...

legorobot commented on Ask HN: How deep do you go?    · Posted by u/martialg
code_runner · 3 years ago
this is what makes all of my side projects so much fun and so educational.

I get caught up in the most ridiculous yak shaving and never make any real progress.... but always have a blast.

legorobot · 3 years ago
I totally agree with this too! Sometimes its nice to have a project that's meant to go nowhere, but it teaches you a ton and you get a lot of that yak shaving out of the way.

By the end you might have so little to show, but I found writing down what I learned a great way to "mark" the project with a healthy milestone :)

u/legorobot

KarmaCake day11February 12, 2022View Original