The modeness of vi does add friction in some cases, but it's a minor gripe, really.
Vi is incredibly well-suited and ergonomic for text editing, and switching between the modes is efficient, since often times you want to begin typing text as a consequence of some other normal mode operation you'd want to take anyway - e.g. add line below; change until end of line; change in parens; change up to character - and those operations put you immediately into insert mode afterwards anyway.
So I wouldn't let that feature (which enables making common operations more ergonomic) stop you from using vi-style binds.
Though modelessness can be quite nice, and I do see the appeal. Being able to select a region and immediately perform any desired operation is convenient.
What's great is that Emacs is what you make it - you can mix and match the two approaches. I, for example, use vi binds for text editing in Emacs, but maintain modeless binds I can always access when it makes sense. You can have the best of both worlds :)