Science & Technology

Building a home AI server from used parts

by Ice Tea Admin

📚 Main topics

Building the systemThe creator assembled an approximately $800 server from used parts, including a Ryzen 9 3900X, 16 GB of RAM, and two Intel Arc A770 cards with 16 GB of VRAM each. 1:04
Trying operating systems and toolsWindows and Ubuntu were installed for comparison. LM Studio repeatedly produced looping output, so the creator switched to running llama.cpp directly. 4:36
Choosing a GPU backendVulkan caused reliability issues, while llama.cpp’s SYCL backend ran models coherently and used both GPUs. 7:15
Benchmarking performanceLlama Bench was used to compare Windows and Linux, SYCL and Vulkan, and models running on one or both GPUs. 9:21

✨ Key takeaways

Usable local AI performanceSYCL reached up to 41 tokens per second on smaller models and up to 39 on larger models split across both GPUs. 9:54
Different OS strengthsLinux processed prompts faster, while Windows generated tokens faster—especially for larger models. 10:26
Vulkan’s potentialVulkan was faster than SYCL for token generation in most tests, but results were uncertain because of stability issues. 10:57
Budget and VRAMFor around $800, the system provided 32 GB of GPU VRAM and could run local models at decent speeds without subscriptions. 12:01

🧠 Lessons learned

Intel GPU software compatibilityLocal AI apps may not work reliably with Intel GPUs, so choosing compatible software and backends matters. 12:31
Understanding the stack helpsRunning llama.cpp directly provided more control and helped identify the backend behind the problems. 6:42

🏁 Conclusion/next steps

Room for improvementThe hardware showed strong potential, but backend and driver software still need to mature. 11:29
Planned upgradeThe creator plans to upgrade the server to run even larger models and invites viewers to follow along. 12:31

localai localllm homelab ai llm llamacpp intel intel arc intel arc a770 arc a770 a770

Transcript excerpt

0:00 Hello and welcome to my first YouTube video. In this video, I will show you how I built a home AI server with only used parts. The goal of this project was to build a budget friendly system that would still be powerful enough to run local AI models for coding, image generation, and other tasks. Like many of you, I had tried popular AI services like Chat GPT and Claude. But this was my first time trying to run AI on my own hardware. As you will see later on, some of the

0:32 choices I made caused a lot of headaches, but I learned a lot and in the end, I was able to run state-of-the-art local AI models at home without any subscriptions. I will show you the hardware and software that I used, what worked and what didn't, and some benchmarks at the end. So, without further ado, let's start by looking at the hardware. After browsing the used hardware market, I decided to build my system around AMD's AM4 platform. It has a large user base and because

1:04 people have started upgrading to AM5, they're now selling their old AM4 boards. I found a good deal on a bundle containing a motherboard, CPU, 16 GB of RAM, and an NVME SSD for around $400. The motherboard is the ROG Strix B550E Gaming which features three PCIe X16 slots and supports PCIe4 which is important for modern GPUs.

🔒 The full, searchable transcript is available with Pro.

Suggestions