Readit News logoReadit News
phlip9 commented on QUIC for the kernel   lwn.net/Articles/1029851/... · Posted by u/Bogdanp
zamalek · 7 months ago
What is done for that is userspace gets the network data directly without (I believe) involving syscalls. It's not something you'd do for end-user software, only the likes of MOFAANG need it.

In theory the likes of io_uring would bring these benefits across the board, but we haven't seen that delivered (yet, I remain optimistic).

phlip9 · 7 months ago
I'm hoping we get there too with io_uring. It looks like the last few kernel release have made a lot of progress with zero-copy TCP rx/tx, though NIC support is limited and you need some finicky network iface setup to get the flow steering working

https://docs.kernel.org/networking/iou-zcrx.html

phlip9 commented on Show HN: Whirlwind – Async concurrent hashmap for Rust   github.com/fortress-build... · Posted by u/willothy
phlip9 · a year ago
Benches look promising! My main concern is validating correctness; implementing good concurrency primitives is always challenging. Have you looked into testing against a purpose-built concurrency model checker like tokio-rs/loom [1] or awslabs/shuttle [2]? IMO that would go a long way towards building trust in this impl.

[1] https://github.com/tokio-rs/loom [2] https://github.com/awslabs/shuttle

phlip9 commented on Asterinas: OS kernel written in Rust and providing Linux-compatible ABI   github.com/asterinas/aste... · Posted by u/Klasiaster
phlip9 · a year ago
Super cool project. Looks like the short-term target use-case is running a Linux-compatible OS in an Intel TDX guest VM with a significantly safer and smaller TCB. Makes sense. This way you also postpone a lot of the HW driver development drudgery and instead only target VM devices.
phlip9 commented on Adapting Plan 9's listen to GNU Guix   the-dam.org/docs/explanat... · Posted by u/todsacerdoti
tmtvl · a year ago
> Network services are defined by the presence, in a directory watched by listen, of executable files whose name is of the form <protocol><port>.

That seems a bit silly to me as it means something like rot1376 is tricky to parse correctly. I'd think it would make sense for the protocol to need to be separated from the port by a non-numeric character, like a hyphen. Like that it would be trivial to parse: just grab all the numeric characters from the end until we find the non-numeric character.

phlip9 · a year ago
Exactly, it should use a separator. Consider a more realistic example, like http280 or h3443. Totally ambiguous.
phlip9 commented on Orphaning bcachefs-tools in Debian   jonathancarter.org/2024/0... · Posted by u/pabs3
rlpb · 2 years ago
Maintenance is much more practical if everything in your dependency tree uses the same version of every dependency. Achieving this in practice is much of the work distribution maintainers have to do.

Doing this has a couple of key advantages:

1. when a security vulnerability needs patching in a stable distribution release, the security team only have to patch one version instead of a dozen or more different versions bundled all over the place

2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version

In an ecosystem where it's considered acceptable to "pin" versions of dependencies and also call any system that doesn't use the pinned versions "unsupported", both of the above two cases become impractical.

Whether you use shared libraries or static libraries, the above matters still exist.

phlip9 · 2 years ago
Maintenance is much more practical when you use the versions upstream tests in their CI and not whatever mishmash of ancient/silently incompatible deps that each distro separately decides to combine together.
phlip9 commented on Eza: A modern, maintained replacement for ls   github.com/eza-community/... · Posted by u/alexzeitler
phlip9 · 2 years ago
Can definitely recommend eza (prev. exa). I've used it as an ls replacement for a long time with zero problems. If anyone's using nix home-manager, here's my config for inspiration:

    programs.eza = {
      enable = true;

      # In list view, include a column with each file's git status.
      git = true;
    };

    programs.bash.shellAliases = {
      ks = "eza";
      sl = "eza";
      l = "eza";
      ls = "eza";
      ll = "eza -l";
      la = "eza -a";
      lt = "eza --tree";
      lla = "eza -la";
    };

phlip9 commented on The sad state of property-based testing libraries   stevana.github.io/the_sad... · Posted by u/nequo
ruuda · 2 years ago
The major omission in this article is fuzzing. Not only is it practical and in wide (and growing use), it’s also far more advanced than QuickCheck’s approach of generating random inputs, because fuzzing can be _coverage-driven_. Property-based testing came out of academia and fuzzing came out of security research, initially they were not connected. But with the advent of in-process fuzzing (through libFuzzer), which encourages writing small fuzz tests rather than testing entire programs; and structure-aware fuzzing, which enables testing more than just functions that take a bytestring as input, in my view the two techniques have converged. It’s just that the two separate communities haven’t fully realized this yet.

One pitfall with non-coverage-driven randomized testing like QuickCheck, is that how good your tests are depends a lot on the generator. It may be very rarely generating interesting inputs because you biased the generator in the wrong way, and depending on how you do the generation, you need to be careful to ensure the generator halts. With coverage-driven fuzzing all of these problems go away; you don’t have to be smart to choose distributions so that interesting cases are more common, coverage instrumentation will automatically discover new paths in your program and drill down on them.

But isn’t fuzzing about feeding a large program or function random bytestrings as inputs, whereas property-based testing is about testing properties about data structures? It is true that fuzzers operate on bytestrings, but there is no rule that says we can’t use that bytestring to generate a data structure (in a sense, replacing the role of the random seed). And indeed this is what the Arbitrary crate [1] in Rust does, it gives tools and even derive macros to automatically generate values of your data types in the same way that QuickCheck can. The fuzzing community calls this Structure-Aware Fuzzing and there is a chapter about it in the Rust fuzzing book [2]. There are also tools like libprotobuf-mutator [3] that substitute fuzzers’ naive mutation strategies, but even with naive strategies fuzzers can usually get to 100% coverage with appropriate measures (e.g. recomputing checksums after mutation, if the data structure contains checksums).

I am using this extensively in my own projects. For example, RCL (a configuration language that is a superset of json) contains multiple fuzzers that test various properties [4], such as idempotency of the formatter. In the beginning it used the raw source files as inputs but I also added a more advanced generator that wastes less cycles on inputs that get rejected by the parser. The fuzzer has caught serveral bugs, and most of them would have had no hope of being found with naive randomized testing, because they required a cascade of unlikely events.

Structure-aware fuzzing is not limited to generating data structures either, you can use it to generate reified commands to test a stateful API, as you describe in the _Stateful property-based testing_ section. The Rust fuzzing book has an example of this [5], and I use this approach to fuzz a tree implementation in Noblit [6].

[1]: https://docs.rs/arbitrary/latest/arbitrary/ [2]: https://rust-fuzz.github.io/book/cargo-fuzz/structure-aware-... [3]: https://github.com/google/libprotobuf-mutator [4]: https://docs.ruuda.nl/rcl/testing/#fuzz-tests [5]: https://rust-fuzz.github.io/book/cargo-fuzz/structure-aware-... [6]: https://github.com/ruuda/noblit/blob/a0fd1342c4aa6e05f2b1c4e...

phlip9 · 2 years ago
Agreed. A while back I played around with fuzzcheck [1], which let's you write coverage-guided, structure-aware property tests, but the generation is smarter than just slamming a fuzzer's `&[u8]` input into `Arbitrary`. It also supports shrinking, which is nice. Don't know that I would recommend it though. It seemed difficult to write your own `Mutator`s. It also looks somewhat unmaintained nowadays, but I think the direction is worth exploring.

[1]: https://github.com/loiclec/fuzzcheck-rs/

phlip9 commented on Show HN: Brioche – A new Nix-like package manager   brioche.dev/blog/announci... · Posted by u/kylewlacy
phlip9 · 2 years ago
Congrats on the release! I love the focus on devex w/ typescript and autocomplete. That's probably one of my biggest pain points with Nix -- writing any non-trivial package always requires a ripgrep adventure through nixpkgs. Finding the right poorly documented and poorly discoverable derivation attributes is always such a chore.

What are your plans for cross-compilation or heavy package customization? One of nixpkgs coolest party tricks imo is that you can just change the stdenv and get a musl static binary or cross-compiled binary.

phlip9 commented on Viking 7B: open LLM for the Nordic languages trained on AMD GPUs   silo.ai//blog/viking-7b-t... · Posted by u/reqo
bangaladore · 2 years ago
I have had this question. How much better would common LLMs (Llama, GPTN) be if they were only trained in one language? I have to assume they would perform better, but I might be wrong.
phlip9 · 2 years ago
Interesting thought. Maybe an LLM would build deeper insight with only one training language. On the other hand, the model might overfit with just one language -- maybe multilingual models generalize better?

u/phlip9

KarmaCake day94September 28, 2013
About
github: https://github.com/phlip9

notes: https://phlip9.com/notes

twitter: https://twitter.com/phlip9

View Original