Very cool! I will definitely give this a try, I've been looking to build Go bindings to Mach[0] soon.
It looks like this would make cross-compiling CGO easier (no target C toolchain needed?)
Does this do anything to reduce the overhead of CGO calls / the stack size problem? IIRC the reason CGO overhead exists is at least partly because goroutines only have an ~8k stack to start with, and the C code doesn't know how to expand it-so CGO calls "must" first have the goroutine switched to an OS thread which has an ~8MB stack.
One reason I think Go <-> Zig could be a fantastic pairing is that Zig plans to add a builtin which tells you the maximum stack size of a given function[1], so you could grow the goroutine stack to that size and then call Zig (or, since Zig an compile C code, you could also call C with a tiny shim to report the stack required?) and then eliminate the goroutine -> OS thread switching overhead.
Contributor here: Purego doesn’t do anything to improve the overhead of calling into C. It uses the same mechanisms that Cgo does to switch to the system stack and then call the C code. Purego just avoids having to need a C toolchain to cross compile code that calls into C from Go.
I’ve actually been quite interested in Zig. If that built-in was added than it would likely be possible to grow the goroutine stack to the proper size and than call the Zig code. Very interesting stuff!
Makes sense! I also wonder (if you know): last I looked I recall that each CGO call requires switching to the system stack, but I can't recall what happens after. Does it switch back to a regular goroutine stack once the syscall has completed?
I wonder if a more tailored CGO implementation could pin a goroutine to a thread which is guaranteed to have a system stack available, so that each CGO call need not worry about that switching at all. Maybe that'd require runtime changes though?
I thought I had a piece of dust on my screen, but as I scrolled the dust scrolled: what do these 0xB7 characters do in the identifiers? Are they just "name mangling" to keep them from being exported or something?
"In Go object files and binaries, the full name of a symbol is the package path followed by a period and the symbol name: fmt.Printf or math/rand.Int. Because the assembler's parser treats period and slash as punctuation, those strings cannot be used directly as identifier names. Instead, the assembler allows the middle dot character U+00B7 and the division slash U+2215 in identifiers and rewrites them to plain period and slash."[0]
I did not know that this could reasonably be done. For some reason it did not occur to me that you could break the chicken and egg problem by simply linking to libdl dynamically; Go binaries are usually static and I didn't even realize it had a mechanism for dynamically linking like this.
This is pretty cool because you can already do this sort of thing on Windows (using the syscall package, since the Windows Loader is always available from kernel32 anyways) and I use it all the time. Probably the most consequential thing I've done with it is my WebView 2 bindings. But with this, you could probably do the same thing on Linux and Mac with GtkWebkit and ... WebKit, and get a native HTML window without CGo on Windows, Mac, and Linux. Perhaps this has already been done (haven't paid attention) but it would make a pretty nice way to get a UI going in Go. (It's not like I'm a fan of using HTML UIs for native apps, but it works pretty well if you don't overdo it, and using native widgets on a given platform does mess up predictability a bit, but it saves disk space at least.)
It loads the dynamic library at runtime, instead of linking against it, which means it makes cross-compiling with CGO easier as no target C toolchain is needed.
What slimsag wrote is correct. It makes cross-compiling code that needs to call C functions as easy a setting the GOOS and GOARCH and just building. This means no need to worry about building a C cross-compiler.
I do want to write an article about how purego works under the hood.
Interestingly it’s easy to do this with just standard library on Windows: syscall.NewLazyDLL plus NewProc are enough. (Of course in practice you should probably use golang.org/x/sys instead.) I’ve never thought about why dlopen isn’t offered in syscall on *nix until now.
Readit News
It looks like this would make cross-compiling CGO easier (no target C toolchain needed?)
Does this do anything to reduce the overhead of CGO calls / the stack size problem? IIRC the reason CGO overhead exists is at least partly because goroutines only have an ~8k stack to start with, and the C code doesn't know how to expand it-so CGO calls "must" first have the goroutine switched to an OS thread which has an ~8MB stack.
One reason I think Go <-> Zig could be a fantastic pairing is that Zig plans to add a builtin which tells you the maximum stack size of a given function[1], so you could grow the goroutine stack to that size and then call Zig (or, since Zig an compile C code, you could also call C with a tiny shim to report the stack required?) and then eliminate the goroutine -> OS thread switching overhead.
[0] https://github.com/hexops/mach
[1] https://github.com/ziglang/zig/issues/157
I’ve actually been quite interested in Zig. If that built-in was added than it would likely be possible to grow the goroutine stack to the proper size and than call the Zig code. Very interesting stuff!
I wonder if a more tailored CGO implementation could pin a goroutine to a thread which is guaranteed to have a system stack available, so that each CGO call need not worry about that switching at all. Maybe that'd require runtime changes though?
https://github.com/ebitengine/purego/blob/v0.2.0-alpha/sys_d...
I noticed another 0xB7 character in a comment, and sure enough it seems to be part of the identifiers: https://github.com/ebitengine/purego/search?q=runtime%C2%B7c...
[0] - https://go.dev/doc/asm
This is pretty cool because you can already do this sort of thing on Windows (using the syscall package, since the Windows Loader is always available from kernel32 anyways) and I use it all the time. Probably the most consequential thing I've done with it is my WebView 2 bindings. But with this, you could probably do the same thing on Linux and Mac with GtkWebkit and ... WebKit, and get a native HTML window without CGo on Windows, Mac, and Linux. Perhaps this has already been done (haven't paid attention) but it would make a pretty nice way to get a UI going in Go. (It's not like I'm a fan of using HTML UIs for native apps, but it works pretty well if you don't overdo it, and using native widgets on a given platform does mess up predictability a bit, but it saves disk space at least.)
It made sense after reading the comment about not needing a C toolchain for cross-compiling CGO but I didn't realize it immediately.
Neat stuff
I do want to write an article about how purego works under the hood.
This would be such a game-changer for server-side rendering Javascript in Go with V8.
I'd love to integrate this into Bud[1].
[1] https://github.com/livebud/bud