Odin Gets Its Own Box — John Rothfusz

The last Odin post was about the coach. Since then most of what I've done is hardware. Odin used to be a VM on one of my Proxmox servers, sharing a single graphics card. Now it has its own machine, a second graphics card, a bigger model, and a studio that makes pictures and video without anything leaving the house. This post covers all of that, plus the smaller stuff.

A small rounded box on the left holding a few glowing teal dots joined by thin lines, with one bright line arcing across the dark frame to a much larger rounded box on the right holding the same dots spread out

Its own machine

On the VM, the graphics card was passed through from the server that also runs my storage. If I broke a driver trying something new, the card could wedge and the only fix was power cycling the whole server, which took the NAS down with it. Odin's box only runs Odin now, so breaking it takes down nothing else.

It's an Intel Core Ultra 7 270K Plus with 32 GB of RAM and a 1 TB NVMe, running Ubuntu 26.04. No VM, just the machine. It sits in the rack in the basement with the rest of the gear.

A black open-frame server rack in a basement next to a pegboard of tools and a wooden desk with a monitor and printer. The rack holds network switches with blue and yellow cables, a blue rackmount case, a thin one-unit server, a CyberPower battery backup and a black box with a wood-slat front — The basement. Odin is the blue case.

Looking down into an open computer case at two graphics cards side by side, a black PNY GeForce RTX 5060 Ti on the left and a silver Gigabyte Radeon card on the right — The two cards. NVIDIA on the left, AMD on the right.

Top-down view inside the open case showing both graphics cards, the motherboard with a liquid cooler and G.Skill memory, a Toughpower GT 1200W power supply and a lot of black cables — Inside. Cable management is a work in progress.

Moving it

Before I moved anything I built a benchmark. It times how fast the model answers, how long until the first word, how long until the voice starts talking, and it runs 72 test requests that Odin has to get right. I ran it on the VM first so I'd have something to compare against.

The move itself was a final backup of the database, a restore on the new box, and a check that every table matched. All 101 did. Then I switched over. The 5060 Ti is the same card it was on the VM, so I expected the same speed, and that's what I got: 42.1 tokens a second against 42.3 on the VM. A token is roughly a word. It sits in a slower slot on this board and that doesn't matter, because how fast it writes depends on the card's memory. Typed requests got a little quicker. Time to first word dropped from 2.65 seconds to 2.41, and the voice started talking at 2.25 seconds instead of 2.52.

I also rebuilt the Proxmox server that runs my storage. Its boot drive had died for the second time, so I put a plain SATA SSD in it. Odin has a tool that redeploys a freshly installed Proxmox host, and I ran it for real for the first time on this one. It took a few runs. Each one that failed turned into a fix.

A second card, and a bigger model

The goal is for all of this to run on hardware I own, so no company can take it away. So the second card is an AMD R9700. It has 32 GB of memory.

The old setup ran Qwen3-14B on the 5060 Ti, next to the voice models and the memory search. It was tight. Under heavy load the voice started throwing errors. Now the R9700 holds the main model and nothing else, and the 5060 Ti does voice, memory search and pictures.

Two tall rounded boxes side by side on a dark background. The large one on the left glows teal and is filled with a web of small dots and thin lines. The smaller one on the right glows amber and holds a single brush stroke and a small picture frame, with a thin line and a small point of light connecting them

To pick the model I set up a bake-off that ran by itself after a reboot. It tried seven models I downloaded plus five more through Ollama, and ran the same graded tasks on each: picking the right tool, not calling a tool when none fits, reading a result back, a bit of math, and holding a long conversation in its head. Everything scored about the same on the easy quiz, so that wasn't what decided it.

What decided it was replaying 33 real conversations where Odin had messed up. The old 14B model got 13 of them right. Qwen3.8 27B got all 33, but it was 2.6 times slower. MiniCPM-V got 28, but in all three tries it turned off the living room TV when I'd said the family room. Ornith 1.5 got 27, never acted on the wrong thing, and was about three times faster than the old model. I went with Ornith.

It's a 35 billion parameter model and it uses about 25 of the card's 32 GB. It can hold 256K tokens of conversation at once, where the old setup ran 16K. Against the old model, the benchmark score went from 83% to 89%, time to the first word on a typed request went from 2.41 seconds to 1.60, and a whole turn from 3.44 to 2.24. Voice didn't change much.

One more thing came out of this. Photos of meals, photos of recipe ingredients, wine labels and screenshots are all read by the local model now. That used to go to a paid one. Now it costs nothing.

Pictures and video

The 5060 Ti has room left over, and I'm using it as an image studio. It runs ComfyUI, and Odin asks it for pictures the same way it asks for anything else. The rule is local first, and GPT only if I ask for it.

New pictures use FLUX.3 klein. I ran it side by side with the model I was using before and it was a lot better. It takes around 10 seconds.
Edits use Qwen-Image 2.1, 13 to 20 seconds each. It keeps the same person across a series of edits.
Video uses Wan 2.2, clips of 1 to 5 seconds. A 3 second clip took 108 seconds.

A thin white square canvas on a dark background with dots of teal light gathering inside it into the shape of a mountain and a sun, and a short film strip of three empty frames beside it

A job loads its model, runs, and frees the memory afterward. I ran 82 voice requests during one render and none of them failed. The local pictures come out at 1536 pixels on the long edge, so for something I want to print I still ask for GPT.

It can also train a LoRA, which is a small add-on that teaches the image model a particular face or style. Training doesn't fit beside voice on the 5060 Ti, so when I queue one it runs overnight. Voice pauses at 11 PM and is back by about 1 AM.

Everything else

After DNS went down on October 13, I moved Pi-hole and nginx onto a Raspberry Pi 5. It's powered by the network switch and doesn't depend on any of my servers, so DNS stays up when they don't. All 43 names resolved the same before and after the swap.

Backups got a second layer. The database dump still goes offsite every night. Now there's also a nightly copy of everything the dump and GitHub don't hold, including a bundle of every project repo so unpushed commits survive a dead drive. There's also one guide for recovering from any kind of failure, whether that's a drive or the whole setup.

One battery backup runs the servers and the network gear. When I installed it I measured about 29 minutes. If the power drops, Odin shuts itself off first, 4 minutes in. When power comes back, a script brings things up in order, DNS first and storage before anything that needs it.

The coach chat can see every ride from the last 7 days now, Zwift rides too, with power, heart rate and cadence. It used to only see weekly totals, and after a 38 minute Zwift ride it told me it couldn't see the ride.

That's everything since the last post.