The \pi_i in the paper is not the estimate of a latent parameter. It is the predictive probability of the event, which is a single number by necessity in a binary challenge. It's the integration of a distribution function which can contains very complex distributions: in my example something_you_believe can be a probability distribution.
So everything in the paper is distribution and when you forecast for a binary event, you give a number which is the expectation of that distribution. This is a probabilistic forecast.
If you were to give a probabilistic forecast for a continuous quantity, then yes you would give in a distribution, as in section 4.2
Emphasizing this response. Bayesian models can always produce simple probabilities if you ask them to. E.g., given this data, what is the probability that the next flip is heads?
The fact that the model is represented as a distribution over Bernoulli parameter p doesn't contradict this: you just integrate over the posterior.