📚 Main Topics

1. Project Overview

   • Building a budget-friendly home AI server using used parts.
   • Goal: Run local AI models for various tasks without subscriptions.

2. Hardware Selection

   • Chose AMD's AM4 platform due to availability of used parts.
   • Key components:
     • MotherboardROG Strix B550E Gaming
     • CPURyzen 9 3900X (12 cores, 24 threads)
     • RAM16 GB DDR4
     • Storage512 GB Samsung 960 EVO NVME SSD and a 120 GB SATA SSD
     • GPUsTwo Intel ARC A770 cards (16 GB VRAM each)
     • Power SupplyColink Continuum 1,200 W
     • CaseNCXT H710i

3. Operating System Installation

   • Installed both Windows and Ubuntu to compare performance.
   • Initial attempts to run AI models using LM Studio faced issues.

4. Software and Model Testing

   • LM Studio was found to be unreliable; switched to Llama CPP for better control.
   • Explored GPU backends: Vulcan and SLE (Intel's answer to CUDA).
   • Successfully ran models using the SLE backend after initial failures.

5. Performance Benchmarking

   • Conducted benchmarks using Llama Bench on both Windows and Linux.
   • Results showed:
     • Linux was faster for prompt processing.
     • Windows had better token generation speeds.
     • Vulcan backend was faster in most cases but less stable.

✨ Key Takeaways

• Building a home AI server can be done affordably with used parts.
• Understanding hardware and software compatibility is crucial, especially with Intel GPUs.
• Performance can vary significantly between different operating systems and software backends.
• The experience provided valuable insights into the workings of AI models and server setups.

🧠 Lessons Learned

• Don't rely solely on popular applications; sometimes, direct interaction with underlying software (like Llama CPP) is necessary for success.
• Be prepared for troubleshooting and learning curves when working with less common hardware configurations.
• Future upgrades may be necessary to run larger models effectively, highlighting the importance of scalability in hardware choices.

🏁 Conclusion

The project successfully demonstrated the feasibility of running local AI models on a budget, while also emphasizing the importance of understanding both hardware and software in achieving optimal performance. Future videos will explore further upgrades and enhancements to the system.