Some of it. The air density is an important part of efficiency at higher altitudes, so every moment spent under like FL320 is wasted fuel.

So the entire climb "up", you are also wasting energy fighting the thick air. On the way back "down", that air again fights you, even though you are basically at idle thrust.

Your fuel reserves are calculated for cruise flight, so time spent doing low altitude flying is already at a disadvantage. "Two hours of reserves" is significantly less than that spent holding at a few thousand feet. Fuel efficiency while climbing is yet again dramatically worse