Realistically, most USB devices hiccup when they exceed their current limit, especially in 5V mode. And since caps don’t loose charge that quickly typically, you can usually charge the cap up enough to deal with this transient no matter how big your cap is (unless it’s something absolutely enormous… which wouldn’t be implemented for cost reasons)
I have seen this play out with host devices with robust current limiting outputs. Sometimes it works for the connected device and sometimes the device sabotages itself when its own regulator turns on at say 3.6 V, drawing down the partially charged input cap below the regulator's UVLO and turning itself off again. Then the cap starts charging again and the cycle starts over. Depending on the host implementation, the burp mode may not reset without a full disconnection, at best leaving the connected device to sip a low average voltage and low current or persist in the above loop forever. Making sure this doesn't happen is more difficult than just implementing inrush limiting.