Arxiv isn't the solution. But i think computer science conferences are. These have the same scientific rigour and standards in the review process as journals in other scientific fields, but don't price gouge. Yes, conferences are also a bit expensive, but you get a lot for your money, and they usally aren't out to make a big profit.
The peer review process in CS is pretty bad, especially in conferences with a single round of reviews. When you combine this with high rejection rates, peer review becomes more about finding excuses to reject a submission than about trying to improve it.
Conferences also don't work that well as publication venues, as they often require that one of the authors must attend the conference physically. And it's not as much about money than about visa policies and travel restrictions. Even in the 2000s and 2010s, when international travel was easy, people from non-Western countries could often not get visas to attend conferences. And today the situation is much worse.
I've been to three international conferences in the past year. One was held in Europe. People from Russia and Israel had to present remotely, the former due to an ongoing war and the latter due to an unexpected war. Another was in the US, and there were fewer Europeans than usual, as many were not willing to take the risk. And the third one was in Japan. People from China could not attend due to increased tensions. People from Israel were there, but they were worrying if they would make it home before the next war. (They made it, and the war started a day or two later.)
Conferences can be truly wonderful, but not a universal replacement for publishing.
If you think journals are expensive, try sending your whole lab to a conference in another country. That may not let you in. Where some of the attendees have to fill out paperwork before talking to a foreign national. (does that ever make for awkward small talk...)
For all their many faults, journals provide access to a really wide audience, and- in theory- make it possible to form connections who wouldn't be able to meet directly.