How can you generalize without facts? They are the foundation on which generalization is built. Like programming without memorizing the keywords. Unless you make a distinction between facts that let you generalize, and facts that do not, like random ID numbers.
We want the LLM to learn the multiplication algorithm not an incomplete set of tables. The algorithm might be smaller and will be more complete.
Honestly, our technology has outpaced our epistemology. So we don't really know what a fact is or isn't. Are facts what we call our supervised learning experiences? You think the sun rises, no the earth spins. Your belief that the sun rises helps you predict sunset and sunrise. Your belief would be quaint to someone born and raised on a space station. Apollos chariot moves the sun across the sky doesn't it?