Perhaps quite a bit more not impossible?

We only know of the symbol use (if they were symbols) that happened on media that lasted tens of millennia. If they even painted symbols on deeply underground cave walls, they likely had them on many less durable surfaces as well. There could have been a huge oral tradition augmented by drawn memorization aids on durable but not that durable media. That augmented oral tradition would then occasionally, every few dozen generations or so, due to some exceptional circumstances, spill over to the near-eternal medium of a cave wall.