My guess is they are only able to make the phones cheap if it’s sold at huge scale. If you make some niche small phone, the price goes way up and doesn’t look attractive.
Then you have to deal with the fact that the people with obscure requirements have a million other requirements. The person asking for a small phone then complains it doesn’t have a headphone jack, and AV1 decoding, and 16gb memory, and an unlocked bootloader, and whatever else.