> because applications and windows are a separate concept

Is this the reason why "closed" applications still show up in cmd+tab?

Yeah the application is still loaded in ram.