Then the real question is whether this bespoke syscall mechanism will be needed going forward, especially as things like time_t adopt 64-bit values anyway. Can't we just define a new "almost 32-bit" ABI that just has 64-bit clean struct layouts throughout for all communication with the kernel (and potentially with system-wide daemons, writing out binary data, etc. so there's no real gratuitous breakage there, either), but sticks with 32-bit pointers at a systems level otherwise? Wouldn't this still be a massive performance gain for most code?

You could definitely do better than x32 did (IIRC it is a bit of an outlier even among "32-bit compat ABI" setups). But even if the kernel changes were done more cleanly that still leaves the whole software stack with the ongoing maintenance burden. The fact that approximately nobody has taken up x32 suggests that the performance gain is not worth it in practice for most people and codebases.

Defining yet another 32-bit-on-64-bit x86 ABI would be even worse, because now everybody would have to support x32 for the niche users who are still using that, plus your new 32-bit ABI as well.

But that maintenance burden has been paid off for things like 64-bit time_t on 32-bit ABI's. One couod argue that this changes the calculus of whether it's worth it to deprecate the old x32 (as has been proposed already) but also propose more general "ABI-like" ways of letting a process only deal with a limited range of virtual address space, be that 32-bit, 48-bit or whatever - which is, arguably, where most of the gain in "x32" is.