This would be true if the algorithm changes were limited to for-you feeds. But the larger problem is that the set of people willing to pay for X are boosted in replies. So if that set of people, which tends towards a certain political bias, is hostile towards a poster, that poster will be driven away from posting on X.
The net result is that X shows breaking news, in the same way that the (infamous) meme of bullet holes marked on the WWII plane only shows part of the story - the people who have departed the platform aren't posting, and thus X is only breaking news from a subset of people.
This might be fine for certain types of topics. For understanding the zeitgeist on culture and politics, though, you can't filter your way towards hearing from voices that are no longer posting at all.
I don't care about culture and politics on X, in fact it is something I actively block. By discussion I mean tech news and trends, ie how is someone using the latest AI model or what new project was created, that sort of stuff. The people I follow provide me that, not politics. If you're there for politics then I agree with your point, look elsewhere.