Of course they can. It doesn't take that long to learn CUDA/etc and hardware details, read through some manuals and performance guides, do some small projects to solidify that knowledge. What you need is talent, and some months. That's why at university I saw plenty of students pull off amazing projects, and there's nothing eyebrow-raising in the slightest about starting a PhD a year after getting a bachelor's and writing software more sophisticated than most programmers would write in their career.

I think the programming profession overvalues experience over skill. However, when I was young I had no appreciation for what the benefits of experience are... including not writing terrible code.