I've been hosting a ejabberd instance for a few people for almost a decade now. It just works, takes close to no resources and needs almost no maintenance - just some trivial config checkups when migrating to newer major releases.
Around the same time I tried hosting a Matrix server with Synapse, but quickly stopped. It consumed a lot of resources when doing almost nothing, and it stopped running after an upgrade with some non-obvious error message, so instead of reanimating it I just abandoned it and moved to matrix.org with my personal account.
Ejabberd is a bit overkill for a few people though. Prosody could be easily set up in 15 minutes or less. Also the config doesn't break when migrating to new releases.
For me too, ejabberd is the admin-friendlier/lower-effort one. Being more "monolithic", your calls will work straight out of the box because it ships a turn server properly configured out of the box, manage certificates over ACME for you, etc. Prosody isn't bad but has a reputation of needing attention to be paid to which incompatible modules not to enable together and overall more protocol knowledge. Both will run on a first gen RPi effortlessly.