I wonder if populations have fully recovered from the whale-oil era.

Fish species in general have been badly affected by commercial fishing:

https://what-if.xkcd.com/33/