> It’s unprecedented for a language to be both garbage-collected and as squarely aimed at systems programming as Go is, because systems programmers have trouble tolerating the stop-the-world pauses that happen when the GC runs.
Only for those that never heard of Mesa/Cedar, Algol-68, CPL, Ada, Modula-2+, Modula-3, Oberon, Oberon-2, Active Oberon, Component Pascal, Oberon-07, C+@, Sing#, System C#, D.
I continue to be surprised by the sheer amount of writing we see about ntpsec. The project was started over two years ago, and yet progress reports seem very optimistic and self-congratulatory, despite an obvious lack of progress.
Sure rewriting "bits" of things, with more features, and adding my python sounds useful. But here we are 500+ days since the project started and there are only minor experiments in moving from C.
I get the feeling this is one of those projects which will never be complete. Perhaps not a bad thing, providing those involved are having fun, but so much verbiage to say so.
What progress are you looking for? There seem to be downloads here, posted pretty regularly: ftp://ftp.ntpsec.org/pub/releases/
I don't know if they build and work, but I'd suspect they do, based on the refactorings being discussed. The commits seem to be clicking along. A diffstat of roughly where I think they branched off of ntp vs. master takes a moment to poke through, because the raw stats are "1032 files changed, 126489 insertions(+), 143789 deletions(-)", but there's a lot of trading C code for test code and more docs in there.
I don't know exactly what metric you're using but this doesn't match my "obvious lack of progress".
They started with Mills ntpd, so there have been downloads available since t0. There's a significant amount of code you can lose from a large C project just by getting rid of ifdefs that nobody uses, so the +/- line count isn't a great metric either.
The true metric for success for something like ntpsec is the number of meaningful security problems ntpd has been vulnerable to since ntpsec's inception that ntpsec hasn't been.
At the point where they rewrite in a new language, they're embarking on a fundamentally different project with a different value proposition, which sort of moots the progress they've made (or not made) on hardening ntpd.
(Fair warning: I am both an ESR skeptic and an ntpsec skeptic; I genuinely do not like the idea behind the ntpsec project).
Downloads have always been available, as this project was started by taking an existing one - then trying to cut down on things that were insecure, bad, outdated, or similar.
In short this project is very like the forked openssl project. Progress has been made, I can see it, but I think taking two years to turn some existing code into something cleaner is .. surprising.
Especially now that the focus seems to be throwing it away, and starting anew in rust / go (though based on recent updates go seems to be the choice).
Moreover, at the point where you're rewriting ntpd in a new language, you're no longer really producing a hardened version of the original. There already are competing ntpd implementations, if that's all you're after, so this latest excursion into Rust and Go raises the question of whether the project still even makes sense.
I've been pondering this question too. NTP is a very important Internet service and it's great that folks are looking for a more secure alternative to traditional ntpd. I'm not sure if NTPsec is going to get there or not.
The best alternative out there right now seems to be Chrony. But it hasn't had nearly the inspection and usage ntpd has had, so it's hard to know for sure about its security. ntimed was also looking promising but seems stalled. openntpd doesn't keep time very well and systemd-timesyncd isn't really an NTP implementation.
I had the same thought. I will say that that lack of progress may be the exact reason why so much consideration is being put into porting to another language, so as to make faster progress.
> I checked this by writing loccount as a parallelized tree traversal. Each file in the tree gets a goroutine spawned to count its lines; the threads run in parallel and as each finishes its work it stuffs a statistics block into a channel.
In the past, this approach would blow up because Go would spawn a kernel thread for each blocking system call that was performed. I had to use a rate limiting channel to constrain the number of kernel threads. Is that still the state of affairs?
Yes, that is still the case. Cgo calls work the same. Note that the most common syscalls get special treatment though: file and network I/O use a thread pool and epoll/kqueue respectively.
Isnt there a way to limit the number of threads? Starting N threads each reads the same queue and executes the operation. The queue gets full when there are no workers are available. I though you can implement something like this in Go...
"I think writing toy programs - conscious finger exercises - are a terrible way to learn a new language."
whoa hold on there. everyone's learning style is different. Doing "dumb simple" things first and extracting reward from it is essential to growing progress, even for advanced programmers imo.
For me, the "learning curve" of Go was very small and that's the kicker. You can get up to speed quickly and get a 'feel' for the language quickly. It's been a great learning experience and I recommend it thoroughly.
I continue to be surprised by this argument. It seems to only matter if your job is to learn one language per week.
I don't need to get up to speed in a weekend. It's the speed that I'm getting up to that matters. Of course there is a limit to this principle, I grant you that.
What matters as well is how readable other people's code is, or even my own code after not touching it for a year. And I mean readable for someone as proficient in the language as the original author. This aspect sometimes seems underappreciated by proponents of "powerful/expressive" languages.
Learning curve is important because it means you can bring a new team member up to speed quickly--you don't need to find Go developers, you can make them. The learning investment will pay dividends very quickly. Also, Go can be mastered in 1-3 months, whereas other languages take years. Being able to quickly (i.e., cheaply) produce experts in the language seems advantageous.
I think what you are saying is that the readability of a language is more important than how easy a language is to learn. I heartily agree. Languages that encourage simple, readable code trump pretty much any other language out there, in my opinion. I can't tell you how many times I've looked at my overly abstract Ruby code and just thought, WTF?!
I like Go for this reason. It has more noise (err) than I'd like, but when I look at most Go code, I find it relatively easy to follow and modify. There's no other language I've used that achieves this so consistently.
That is the beauty of gofmt in my opinion. You can pretty much guarantee that your code will be almost the same style as someone else's code. Which in turn increases readability down the line for someone who knows the language.
I encourage anyone that hasn't learned Go, to do it as well. In fact, everyone should learn as many languages as they have time for! Each one has interesting decisions they've made that will possibly help you in your everyday language of choice/work.
That being said, Go was fun to learn, and taught me that deployment does have to be as painful as it's been in Java for so many years. But I encourage people to also spend some time with Rust, as it has made a very different set of choices. For me the type and generics system are the best I've ever used and I wish I had access to them in Java.
>It’s nice to see CSP in a language I can use for production, and very gratifying to find that it really is an effective tool for carving up concurrency problems. If there is one pitch the Go designers have knocked right out of the park, this is it. I’m already sure I’m going to miss it a lot in Rust.
Wait until he realizes that Rust has channels, too. It's not as pure as Go's implementation, but it's a superset.
So I have to wonder about the performance difference between his sloccount and loc (https://github.com/cgag/loc) which is so fast it always makes me think that it hasn't done anything (same as ripgrep).
I'm like 98% sure loc will be faster, but it would be interesting to know by how much.
Because they kind of are, even though they are targeted to different set of skilled developers and Go happens to be a bit spartan in features.
For any type of application, including user space drivers, that can be developed with the constraints of a sub-millisecond GC pauses, GC is a good language, in spite of its stance on generics.
As for kernel space, Go could eventually be used as well, in spite of GC hate, as academia including Microsoft have proven a few times.
Me personally, I prefer Rust, but I do see others would think otherwise.
- Both were invented by a major browser vendor, roughly around the same time.
- Both include an unsafe fun zone that lets you play with raw pointers.
Maybe some of that was inevitable just by following the trends in the field (safety and parallelism in particular), and we'll see other new languages making the same choices?
Safety is hardly a trend is it? GC'ing everything makes safety rather much easier. Go doesn't go beyond Java level safety or really any of many earlier "managed" languages. Does Go offer safety above, say, PHP?
Rust not only does all that, it provides data race safety, and does so without a runtime or tracing garbage collector or real performance sacrifices. All while maintaining C linkage for zero cost interop. I can use Rust basically anywhere I could use C, Go, not so much.
That seems to put it squarely in a different segment than Go.
No they're not. Go is very easy to pick up, Rust is not. That point alone should be enough. Without prior knowledge of both, no sane person would consider writing a browser-engine in Go, and yet that's exactly what they're doing with Rust. Just as no sane person would pick up Rust to write generic web services in that scenario.
Put Rust and Go in front of a python coder who wrote dozens of web services using flask - and I am pretty sure I can predict what tool he'll pick. Sure you could write web services in Rust, that's not what I'm arguing, learning Rust for writing stuff like that is simply not worth it for people having to write such services.
Rust's complexity is comparable with C++, and the main reason for this complexity is to address problems that are mainly seen in C/C++ applications. Rust's reason for existing is simple: modern low-level language with built-in (memory) safety with no performance trade-offs (implying concurrency support etc). That makes it squarely aimed at C++, where developers accept the complexity as a trade-off for performance.
Go's reason of existence and goals are completely different, it addresses the gap that existed between compiled and interpreted languages. It wants to be a simple, safe, easy to deploy language that offers more performance over classic scripting languages and thus is able to address more performance-critical issues. It's main target is clear: servers and networking services, where a GC's impact less important due to the I/O bottlenecks in most scenario's.
"Rust competes for mindshare with C++ and D for programmers who are prepared to accept more complex syntax and semantics (and presumably higher readability costs) in return for the maximum possible performance. For example, micro controllers, AAA game engines, and web rendering engines."
"Go competes for mindshare in the post 2006 Internet 2.0 generation of companies who have outgrown languages like Ruby, Python, and Node.js (v8) and have lost patience with the high deployment costs of JVM based languages."
Readit News
Only for those that never heard of Mesa/Cedar, Algol-68, CPL, Ada, Modula-2+, Modula-3, Oberon, Oberon-2, Active Oberon, Component Pascal, Oberon-07, C+@, Sing#, System C#, D.
http://www.ada-europe.org/manuals/Rationale_2012.pdf
Was also the source for many of Plan 9 UI ideas.
What is most successful operating system created in Go or Rust?
Sure rewriting "bits" of things, with more features, and adding my python sounds useful. But here we are 500+ days since the project started and there are only minor experiments in moving from C.
I get the feeling this is one of those projects which will never be complete. Perhaps not a bad thing, providing those involved are having fun, but so much verbiage to say so.
I don't know if they build and work, but I'd suspect they do, based on the refactorings being discussed. The commits seem to be clicking along. A diffstat of roughly where I think they branched off of ntp vs. master takes a moment to poke through, because the raw stats are "1032 files changed, 126489 insertions(+), 143789 deletions(-)", but there's a lot of trading C code for test code and more docs in there.
I don't know exactly what metric you're using but this doesn't match my "obvious lack of progress".
The true metric for success for something like ntpsec is the number of meaningful security problems ntpd has been vulnerable to since ntpsec's inception that ntpsec hasn't been.
At the point where they rewrite in a new language, they're embarking on a fundamentally different project with a different value proposition, which sort of moots the progress they've made (or not made) on hardening ntpd.
(Fair warning: I am both an ESR skeptic and an ntpsec skeptic; I genuinely do not like the idea behind the ntpsec project).
In short this project is very like the forked openssl project. Progress has been made, I can see it, but I think taking two years to turn some existing code into something cleaner is .. surprising.
Especially now that the focus seems to be throwing it away, and starting anew in rust / go (though based on recent updates go seems to be the choice).
The best alternative out there right now seems to be Chrony. But it hasn't had nearly the inspection and usage ntpd has had, so it's hard to know for sure about its security. ntimed was also looking promising but seems stalled. openntpd doesn't keep time very well and systemd-timesyncd isn't really an NTP implementation.
http://esr.ibiblio.org/?p=6820
In the past, this approach would blow up because Go would spawn a kernel thread for each blocking system call that was performed. I had to use a rate limiting channel to constrain the number of kernel threads. Is that still the state of affairs?
Think about something like goroutines reading and writing from pipes.
I don't need to get up to speed in a weekend. It's the speed that I'm getting up to that matters. Of course there is a limit to this principle, I grant you that.
What matters as well is how readable other people's code is, or even my own code after not touching it for a year. And I mean readable for someone as proficient in the language as the original author. This aspect sometimes seems underappreciated by proponents of "powerful/expressive" languages.
I like Go for this reason. It has more noise (err) than I'd like, but when I look at most Go code, I find it relatively easy to follow and modify. There's no other language I've used that achieves this so consistently.
That being said, Go was fun to learn, and taught me that deployment does have to be as painful as it's been in Java for so many years. But I encourage people to also spend some time with Rust, as it has made a very different set of choices. For me the type and generics system are the best I've ever used and I wish I had access to them in Java.
Then you should spend some time with OCaml and Haskell. :)
Wait until he realizes that Rust has channels, too. It's not as pure as Go's implementation, but it's a superset.
I'm like 98% sure loc will be faster, but it would be interesting to know by how much.
For loc the best time was: 55.425s and the worst was: 1m30.557s
For loccount the best time was: 42.345s and the worst was: 56.396s
loc supports more languages but I also believe it has more false positives. loc reporting seems nicer as well.
I see ~5 seconds for loc and ~34 seconds for loccount to count a freshly cloned linux repository.
edit: woops, that was user time. These both spin all the cores. loc takes ~0.7 secs and loccount ~6.6 secs real time.
For any type of application, including user space drivers, that can be developed with the constraints of a sub-millisecond GC pauses, GC is a good language, in spite of its stance on generics.
As for kernel space, Go could eventually be used as well, in spite of GC hate, as academia including Microsoft have proven a few times.
Me personally, I prefer Rust, but I do see others would think otherwise.
- Both compile to native code.
- Both care a lot about performance.
- Both care a lot about parallelism.
- Both care a lot about safety.
- Both were invented by a major browser vendor, roughly around the same time.
- Both include an unsafe fun zone that lets you play with raw pointers.
Maybe some of that was inevitable just by following the trends in the field (safety and parallelism in particular), and we'll see other new languages making the same choices?
Rust not only does all that, it provides data race safety, and does so without a runtime or tracing garbage collector or real performance sacrifices. All while maintaining C linkage for zero cost interop. I can use Rust basically anywhere I could use C, Go, not so much.
That seems to put it squarely in a different segment than Go.
We are already seeing that, .NET and Java are finally paying more attention to the AOT story on their standard toolchains.
And we also have Objective-C and Swift on iDevices.
Unless Rust wants to position itself as embedded-only it will compete with Go for higher level tasks.
Put Rust and Go in front of a python coder who wrote dozens of web services using flask - and I am pretty sure I can predict what tool he'll pick. Sure you could write web services in Rust, that's not what I'm arguing, learning Rust for writing stuff like that is simply not worth it for people having to write such services.
Rust's complexity is comparable with C++, and the main reason for this complexity is to address problems that are mainly seen in C/C++ applications. Rust's reason for existing is simple: modern low-level language with built-in (memory) safety with no performance trade-offs (implying concurrency support etc). That makes it squarely aimed at C++, where developers accept the complexity as a trade-off for performance.
Go's reason of existence and goals are completely different, it addresses the gap that existed between compiled and interpreted languages. It wants to be a simple, safe, easy to deploy language that offers more performance over classic scripting languages and thus is able to address more performance-critical issues. It's main target is clear: servers and networking services, where a GC's impact less important due to the I/O bottlenecks in most scenario's.
Taken from Dave Cheney's https://dave.cheney.net/2015/07/02/why-go-and-rust-are-not-c... ->
"Rust competes for mindshare with C++ and D for programmers who are prepared to accept more complex syntax and semantics (and presumably higher readability costs) in return for the maximum possible performance. For example, micro controllers, AAA game engines, and web rendering engines."
"Go competes for mindshare in the post 2006 Internet 2.0 generation of companies who have outgrown languages like Ruby, Python, and Node.js (v8) and have lost patience with the high deployment costs of JVM based languages."
But that is true for any fast-enough high-level language like Java, Scala, C#, OCaml, Haskell, F#, Kotlin, etc.
So it's not Rust vs Go, it's Rust (and Go) vs the world.