This is cool. Yet, there are levels of insanity and those depend on your inability to estimate things.
When I'm launching a project it's easier for me to rent $250 worth of compute from AWS. When the project consumes $30k a month, it's easier for me to rent a colocation.
My point is that a good engineer should know how to calculate all the ups and downs here to propose a sound plan to the management. That's the winning thing.
It goes further than this first order, though. If you're trying to build a business that attracts the types of talent who wants to know the stack up and down, starting with an AWS instance might give you a better shot at funding (and thus a better overall shot), but it's not clear that it gives you a shot a building the business you're aiming for. For the things that "don't make your beer better", sure, but we're talking about training ML models at an ML shop. Here it makes sense for this reason.
That last part is exactly it and I while I know the intro sentence nails it I don’t think compute resonates with people (everyone uses compute). If you are 24/7 running work at scale it absolutely makes sense past the initial first couple years to build out your own DC like this.
We’re past the point in history where most engineers get to make even a recommendation about which platform to use to management.
In 99.999999% of cases management has already decided and is just informing you, because they know better.
I work in a multi-billions dollars company and do not face what you describe
Perhaps an exception (yet so far, I've never encounter the situation you describe)