I just started messing with local LLMs and honestly I’m pretty impressed. I have a workstation laptop with an NVIDIA A1000 (6GB VRAM) and 96GB of RAM. I rarely used my gpu. Occasional CAD design or Machine Learning with OpenCV.

I ran llama3:latest and it ran pretty fast! I’m curious to see how Qwen would run on my system.