There’s a few more considerations:

- You can use the GPU for training and run your own fine tuned models

- You can have much higher generation speeds

- You can sell the GPU on the used market in ~2 years time for a significant portion of its value

- You can run other types of models like image, audio or video generation that are not available via an API, or cost significantly more

- Psychologically, you don’t feel like you have to constrain your token spending and you can, for instance, just leave an agent to run for hours or overnight without feeling bad that you just “wasted” $20

- You won’t be running the GPU at max power constantly