I'm a musician who doesn't have absolute pitch, but does have very strong relative pitch. My understanding is that perfect pitch is neat party trick, but actually a hindrance instead of a help in most musical circumstances. Relative pitch, on the other hand, is incredibly useful (and fortunately you can train and develop it later in life).
Because most people don't have perfect pitch, (Western) music is built on the relationships between pitches rather than the absolute pitches. So with absolute pitch, you can play something by ear; with relative pitch, you can play something by ear in any key.
Learning to think of the notes you're playing relatively instead of absolutely is already a difficult leap for most musicians, and my understanding (though I don't have absolute pitch so I can't compare from experience) is that absolute pitch makes this skill significantly harder to acquire, since you have to retrain your ear in addition to your hands.
If I were offered a choice to trade my sense of relative pitch for absolute pitch, I most certainly would not take it. I know well the feeling of incongruity when my muscle memory is stuck in the wrong key, and absolute pitch would mean I'm stuck there all the time instead of being just able to shake my head, focus on the new key, and clear my mind of the old.
Somehow this makes me think of the differences between tonal and inflective languages. Learning a tonal language can be brutal for a person who's used to expressing emotion/inflections through tone, and there is nothing more frustrating than trying to speak to somebody when saying a word phonetically correct but in a slightly wrong tune and, to them, what you're saying is completely incoherent.
The tonal speaker hears a much wider and more precise range of tones, but that precision also kind of hinders them in a way because they can't not hear it. On the other hand speak with a tonal native speaker who's also learned a non-tonal language and they can understand your mistakes (in their native tongue) perfectly, because they essentially have already untrained the tonal instinct. But I'm sure hearing those tonal mistakes feels quite jarring to them nonetheless, like when you're listening to a musician who gets a chord wrong - it just hurts.
I guess you can get a bit of an idea of what this is like by listening to French people speak English. French doesn’t have phonemic syllable stress like English does and so French people often make mistakes with this. For a native English speaker the syllable stress is an integral part of the word and being phonemic it actually distinguishes between two words that otherwise sound the same. The commonest category of this would be words that change between verb and noun depending on the stress, like “reject”, “protest”, “transfer” etc. And there are other minimal pairs like “insight” vs “incite”.
The native English (many other Indo-European languages also have similar systems) speaker is very unlikely to make a pronunciation mistake in this manner but even very accomplished francophone speakers of English struggle with it even after being corrected.
For example here’s French cabinet minister Bruno Le Maire pronouncing “damages” as “daMAges” https://youtu.be/qKWFsg5uHKo
I think one difference is that to a native speaker daMages still sounds like damages. But in a language where 'ma' not only has a ton of different homophones, but entirely different sets of meaning based upon the tone, things get out of hand quickly. For instance in Chinese, "mother scolds a horse" = "ma ma ma ma" with the entire meaning determined by tones alone.
> in any key
I learned this is especially valuable when switching between instruments with different constrained ranges (you can just adapt), as well as your voice changing over time.