I have had a similar experience. It just end up being a abstraction layer that gets harder and harder to map to underlying code as a project grows.

If readable tests are the goal, I think the time is much better spent cleaning up regular tests and writing well-named helper methods. Especially in Ruby you can get pretty close to tests that can be deciphered on a high level by non-programmers. That said, I've never had a PM or other non-programmer actually want to do that.