Readit NewsThe stand-alone code generator (a statically linked executable written in Go with no dependencies outside the Go standard library) generates stand-alone POSIX C code for the neural net, requiring only gcc to compile.
Also see Fabrice Bellard's LibNC:
C API. Small library, no external dependencies, available for Linux and Windows.
thinc.by the creators of spaCy https://github.com/explosion/thinc
nnabla by Sony https://github.com/sony/nnabla
LibNC by Fabrice Bellard https://bellard.org/libnc/
Dlib dnn http://dlib.net/ml.html#add_layer
Spend some time using Go, and you will be impressed by its performance.
You'll wonder, "Were all those haters on Hacker News misinformed?"
Deleted Comment
I've been out of the cracking scene for over a decade now, but I expect that to be none other than a challenge, having seen how far publicly available decompilers have progressed.
https://nn-512.com/browse/DenseNet121
Even if you had the C code available to you, you would have a hard time producing the input graph.
Good luck reverse engineering it after GCC has compiled it!
NN-512 has an incredibly flexible code generator. It can easily be tweaked to produce completely different code for the same convolution, so everyone can apply their own twist to defeat the reverse engineers ("the intellectual property thieves").
NN-512 generates custom code for all the operations, custom units of work for the threads, custom code around tensor edges, everything is fused and unrolled and customized. If they can deduce the network graph specification from the AVX-512 code, I will be astonished.
If you can do it, show me. But I know you can't.
Anyone who cares about model privacy will use their own variant of a tool like NN-512. It's security through obscurity, but that's the best you can hope for if you are distributing an executable.
Deleted Comment
If you need a little bit of inference (say, 20 ReNet50s per second per CPU core) as part of a larger system, there's nothing cheaper. If you're doing a small amount of inference, perhaps limited by other parts of the system, you can't keep a GPU fed and the GPU is a huge waste of money.
AVX-512, with its masked operations and dual-input permutations, is an expressive and powerful SIMD instruction set. It's a pleasure to write code for, but we need good hardware support (which is literally years overdue).