You exactly describe my journey! After undergrad, I went right to industry doing hardware design. Given a complex PCB, I was confronted with this field of signal integrity and I fell in love with it. My undergrad didn't explicitly require EM courses, but I learned on the job (lots of reading plus guidance from some experienced RF engineers). Eventually I went back for my master's, and it was so nice knowing exactly which niche I wanted to focus on (and finally get a ton of EM fundamentals under my belt). It truly feels like EE is an infinite field with no limit to the possible type of work...