What keeps me from using vi derivatives is knowing that the idea behind it is the result of a misunderstanding. As Joy said himself, “One of the good things about EMACS, though, is its programmability and the modelessness. Those are two ideas which never occurred to me.”
The modeness of vi does add friction in some cases, but it's a minor gripe, really.
Vi is incredibly well-suited and ergonomic for text editing, and switching between the modes is efficient, since often times you want to begin typing text as a consequence of some other normal mode operation you'd want to take anyway - e.g. add line below; change until end of line; change in parens; change up to character - and those operations put you immediately into insert mode afterwards anyway.
So I wouldn't let that feature (which enables making common operations more ergonomic) stop you from using vi-style binds.
Though modelessness can be quite nice, and I do see the appeal. Being able to select a region and immediately perform any desired operation is convenient.
What's great is that Emacs is what you make it - you can mix and match the two approaches. I, for example, use vi binds for text editing in Emacs, but maintain modeless binds I can always access when it makes sense. You can have the best of both worlds :)
Emacs is not really "modeless". Key-chords are kind of modes. Transients are kind of modes. Major and minor... well, just modes.
> What keeps me from using vi derivatives
That's correct. There's no such thing as "vim-mode". Every single Vim extension is full of glaring inconsistencies and shortcomings. Not a single VSCode extension, not IdeaVim for IntelliJ, not Sublime ones — none of them are really full-fledged vim integrations. Emacs one is the notable exception. Evil mode in fact is so good, it never feels like an afterthought, it feels like it was baked into the design of the editor from the get-go. It works extremely well. Emacs in fact can vim better than (Neo)vim.