There is a whole KIP that is “preview” on Kafka 4.1 to handle this use case natively: https://cwiki.apache.org/confluence/plugins/servlet/mobile?c...

Note: I haven’t had a chance to test it out in anger personally yet.

yep i mentioned that in another thread below. very excited to see it added, but it will be some time until its production ready / they implement the full feature set