Yes - the usual comment with these things is that you're not learning notes, you're learning movements.

Scales have specific fingerings and hand movements, including thumb over/under movements at specific locations.

They're not optional extras. They're essential for fluid playing.

More subtly there's also basic finger/hand positioning, which has to be a difficult combination of as-relaxed-as-possible but also firm and precise, so you get fine control over dynamics and timing.

If you don't learn the vocabulary of physical movements, you won't have the physical foundation you need to play notated music properly beyond the very basics.