It does happen-- which is why we're told to watch out for high CO2 levels in our living spaces and let fresh air in often

A slight increase in inspired CO2 levels might cause some mild symptoms but it won't kill brain cells.