Posts

Why This Project In the previous post, I ended up running inference on a Tesla T10 picked up for €300 — a card that, in another life, streamed cloud gaming for GeForce Now. Fun, but let’s be honest for a second: running a model someone else trained is the bare minimum. Anyone with ollama pull can do that in thirty seconds. What was itching at me was the part before that. Where the model actually comes from. How you go from a pile of raw text to something that strings together sentences that hold up. When you use Claude, ChatGPT, or Mistral day to day, you type a prompt, out comes an answer, and everything in between is black magic to pretty much everyone who uses it. ...

Why This Project The Hailo-10H is the NPU Hailo sells for running LLMs on a Raspberry Pi 5 (and other SBCs). Unlike its vision-oriented cousins, this one is built from the silicon up for exactly that: 8GB of LPDDR4 on board! At first, I tested my chip with hailo-ollama, Hailo’s homegrown “Ollama-like” tool that drives their precompiled models. I had fun for about two minutes, and then got bored fast. The models on offer are slow, and worse, locked to one very specific list: ...

What’s the Point? Ever since I got into AI, I’ve enjoyed messing around with weird GPUs. After putting together An AI Setup Cheaper Than Mario Kart World?, I wondered whether decent AI models could run on genuinely cheap GPUs. Spoiler: yes, they can. With a few dark spots along the way. The Setup The setup is more or less the same as in the previous article. As a reminder: HP 700W PSU Intel Xeon E5-1620 v3 @ 3.50GHz 32GB RAM DDR4 ECC 2133MHz Samsung EVO 250GB SSD Quadro K2200 4GB GTX 1050 Ti 4GB But we’re adding 2 slightly suspicious cards to the mix: ...

🖥️ Why bother? Automating DNS record creation sounds fancy, sounds cool even, but what does it actually get you? When you’re running a Kubernetes cluster, one of the tools you really can’t do without is the ingress controller. It acts as a reverse proxy, which means it’s the one deciding where incoming HTTP/S traffic actually goes. graph LR subgraph DNS Entries A1[one.example.org] A2[two.example.org] A3[three.example.org] end RP[Reverse Proxy] subgraph Backends S1[Server One] S2[Server Two] S3[Server Three] end A1 --> RP A2 --> RP A3 --> RP RP --> S1 RP --> S2 RP --> S3 Ok, so if my reverse proxy already redirects traffic to the right application based on the DNS entry, why not just use a wildcard? ...

🔴 Why bother running AI at home? Fair question, right? Why run AI at home at all? And actually, let’s back up further: what even is AI? What is AI? According to Wikipedia: Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. In short, AI means using mathematical functions and operations to automate and speed up complex tasks by reproducing human behaviors like reading, seeing, writing, or understanding language. Thanks to these calculations, machines can learn to recognize images, understand text, make decisions, or even generate content — often faster and sometimes better than we can (though honestly, that’s debatable). ...

🧠 Why automate K3s updates? Keeping a cluster up to date is the bare minimum: security patches, bug fixes, better performance, all of it. But in real life, especially when you’re running a self-hosted cluster on a pile of heterogeneous hardware, you forget, you put it off, or you just don’t have the time for it. And before you know it you’re way behind on versions, which is exactly when the trouble starts. ...

🎯 Why This Project? At first, I thought the Hailo‑8L was some kind of magic chip — the kind where you could run LLMs, use it as a backend device for frameworks like Tensorflow, train models on it, all of it. Spoiler: absolutely not. Digging in a bit, I understood that the Hailo‑8L is built exclusively for inference, not training, and definitely not [Reinforcement Learning] or any kind of LLM business. ...