> A hands-on workshop where you write every piece of a GPT training pipeline yourself, understanding what each component does and why.
I see in dependencies torch, so most likely tensors and backpropagation are not implemented, but rather taken as granted. Does it count then as writing "from scratch"?..
I did something similar (in Rust, AI assisted), but I restricted myself not to use any dependency, only standard library. As result, I have to implement much more things, such as tensor design, kernels concept, simple gradient descent optimizer and even custom json parser, cpu data parallelism abstractions similar to rayon, etc. It was quite fun when I got everything wired and working - soo sloooow, but working.
Hey all!
I've just released my qwen3-rs, a Rust project for running and exporting Qwen3 models (Qwen3-0.6B, 4B, 8B, DeepSeek-R1-0528-Qwen3-8B, etc) with minimal dependencies and no Python required.
- Educational: Core algorithms are reimplemented from scratch for learning and transparency.
- CLI tools: Export HuggingFace Qwen3 models to a custom binary format, then run inference (on CPU)
- Modular: Clean separation between export, inference, and CLI.
- Safety: Some unsafe code is used, mostly to work with memory mapping files (helpful to lower memory requirements on export/inference)
- Future plans: I would be curious to see how to extend it to support:
* fine-tuning of a small models
* optimize inference performance (e.g. matmul operations)
* WASM build to run inference in a browser
Basically, I used https://github.com/adriancable/qwen3.c as a reference implementation translated from C/Python to Rust with a help of commercial LLMs (mostly Claude Sonnet 4). Please note that my primary goal is self learning in this field, so some inaccuracies can be definitely there.
> The people that stay are those that have fewer opportunities to find something better elsewhere. So when you see an architect/principal engineer that has been there for 15 years, you know they know that they'd not be hired at the same level anywhere..
..or they are simply more enjoying life by maintaining a better life-work balance.
I didn't get in details, what kind of CS question should be eliminated in "non-whiteboard" interviews. However, I would definitely ask at least basics of algorithms, their analysis, and data structures. For example, in my carrier, I saw enough getter methods (e.g. getCustomers) which looks from outside simple/fast but internally do linear search on each invocation. It especially "nice", if you have a piece of code which iterates over multiple collections in nested loops on UI thread..
As magic script author, who completed the book step by step, I want to say, you won't get "intimately familiar with the OS" after completion. What you will learn:
* needed packages: purpose, how and in which order to build them
* really basics of OS configuration
* fault intolerance to mistakes
So, why I've built such script? I think it is needed for the next steps, where you actually might want to modify kernel/tools/configuration to gain the knowledge how things are (not-)working.
Yes, this library does not generate random shaped buildings or terrain regions. Instead, it generates them from real geographical data and allows you specify some noise parameters, e.g. :
canvas|z16 {
grid-cell-size: 5%; /* affects triangulation*/
max-area: 0.05%; /* affects triangulation refinement */
color-noise-freq: 100000; /* color of each vertex will use this noise value */
ele-noise-freq: 100000; /* height of each vertex will use this noise value*/
color: gradient(#dcdcdc 0%, #c0c0c0 10%, #a9a9a9 50%, #808080); /* color gradient*/
}
Also, some objects can be randomly generated for specific place, e.g. trees in forest. I found this idea where you have real geodata as input for randomization more interesting than having yet another planet generator with manual setting of various parameters.
I see in dependencies torch, so most likely tensors and backpropagation are not implemented, but rather taken as granted. Does it count then as writing "from scratch"?..
I did something similar (in Rust, AI assisted), but I restricted myself not to use any dependency, only standard library. As result, I have to implement much more things, such as tensor design, kernels concept, simple gradient descent optimizer and even custom json parser, cpu data parallelism abstractions similar to rayon, etc. It was quite fun when I got everything wired and working - soo sloooow, but working.