Several things are at play here. The first is that pre-Modernist architecture was heavily designed around natural lighting and cooling. Many buildings and houses designed and built before the 1920s used combinations of materials and elements that maximized the use of daylight. Transom lights and polished marble floors for example were often used to allow light to bounce through a room and to induce airflow. The light came in from a high place and bounced from the floor to fill normally unlit spaces, while hot air was pushed up and vented through the transom windows. If you've ever been in an office building built from the 1870s through the 1940s you'll notice every single transom only opens outwards towards the hall, as the hall created a high pressure space that pushed air into the low pressure rooms, an effect that is amplified when paired with ceiling fans. Meanwhile things such as mullions allowed the windows themselves to be set more deeply inside the wall by providing a strengthening member that allowed a larger opening. That larger opening with the windows set further in provided shade while still allowing in diffused light, and helped in preventing glare upon the glass while creating an overhang using the building's facade or cornices to allow the windows to be opened while it was raining because eaves are useless above two stories of height.
People don't understand these individual parts and don't understand how they all work together. But they definitely feel the cumulative effects, and those cumulative effects are much friendlier than Modernist and post-Modernist design. Modernism and all the schools that followed it rely on electric technology to fill in the gaps in the design, which makes them feel less accomodating to humans. Electric lighting, electric heating and cooling, and electro-mechanical traversal. Those things aren't as tactile, and many aren't even visible. The Modernist building does not present itself as something humans made, maintain, and interact with, but instead presents itself as a machine that you are temporarily inside of as a foreign entity because everything designed to outwardly accomodate humans is instead either streamlined down or fully away. The overhead lights shine light that you cannot influence, the hidden vents expel air from a place you cannot see or change, and the hanging facade glass walls provide no palpable separation from the outside world. Your mind tells you that this mid-century tower is a place where you do not belong.