Why would you need to pay $40 for a PDF of a paper published almost a hundred years ago? What makes the paper not public domain?

You don't. It is public domain. You pay that if you want to get it from them. This is the same as I can get a free pdf of "Linear algebra done right" from Sheldon Axler's website but if I want to get it from Springer I pay $50 or whatever it is.

That is a very confusing state of affairs!

Repackaging and selling free data is a very old business model.

In many cases, it's value-added, because the bundler may also do some curation and interpretation.

Not sure that's what's happening, here, though...

You can also buy licenses to use AV1, a royalty-free codec.