You don't need a degree to get working visa in the Netherlands (source: I don't have a degree). You can even use Dutch-US treaty to basically hire yourself.

With all the anger over illegal immigrants taking US jobs, as a European it surprises me that nobody in the US seems to even mention the idea of punishing the employer for employing illegal workers.

If I want to hire someone (local or remote) as an employer here, I better make sure the worker has a valid working permit. Fines for non-compliance towards the employer are huge, even for a single day of work. All paperwork has to be complete before any work is done. Even when hiring through intermediary companies who guarantee it's all legal, liability and fines remain in place for the ultimate employer if it turns out to be not so.

How much faster is it?

Are there any interesting modern applications of diff. equations in computer science outside physics simulators, and 3d vision? Or some adjacent areas that would benefit from skillset of working with diff. equations?

There is an example of using this with gradient-based optimization here: https://docs.sciml.ai/SciMLSensitivity/dev/tutorials/data_pa....

As an ODE solver, you wouldn't do nanoGPT with it though, you'd need to go back to KernelAbstractions and write a nanoGPT based on that same abstraction layer. Again, this is a demonstration of the cross-GPU tools for ODEs, but for LLMs you'd need to take these tools and implement an LLM.

We recently showed DiffEqGPU.jl generating customized ODE solver kernels for NVIDIA CUDA, AMD GPUs, Intel OneAPI, and Apple Metal, where for CUDA it matches state of the art (MPGOS) which is about 10x-100x something like Jax/PyTorch (where the performance difference comes from inefficiencies of using vmap vs actually writing and calling a kernel). It's all in https://arxiv.org/abs/2304.06835. So this stuff exists and people are using it. Of course the caveat here is this is the context of engineering applications so someone would need to do similar for LLMs to fully relate back to the article, but it shows the tools are ready to a large extent for someone to step up in the ML space.