> lidar and accelerometers can't tell cold from hot, lumpy from smooth.

Not those. There are other sensors. Tactile sensors exist. Lumpy and smooth can be distinguished. They are still rudimentary, but there is nothing fundamental blocking progress here. Roughness, squishiness, temperature, all well measurable.