Waymo does not believe that remote drivers are responsive enough to be able to safely operate. Safety drivers communicate with the self-driving system, and can set waypoints etc. for the navigation system, but the delays inherent make it unsafe, is what the Waymo people say publicly at least.
The reason that Waymo is slow to expand is that they have to carefully and extensively LiDAR map every single road of their operating area before they can open up service in an area. Then while operating they simply do a difference algo on what each LiDAR sees at the moment and the truth data they have stored, and boom, anything that can potentially move pops right out. It works, it just takes a lot of prep- and a lot of people to keep on top of things too. For example, while my kid's school was doing construction they refused to drop off in the parking lot, but when the construction ended they became willing. So there must be a human who is monitoring construction zones across the metro area, and marking up on their internal maps when areas are off limits.