If you use readline's Vim mode, then you might be interested in athame. This project is a patch for readline that embeds Vim, so you can use Vim's full editing capabilities rather than the more limited vi-mode subset that readline provides.
Because one of the advantages of Neovim is embedding, I wondered what a similar Neovim/readline fork would look like. However, it might be possible to approach the problem differently: use a Vim script to communicate with a Neovim
If you want to try out vi-mode in your terminal, try
set -o vi, type some text, and then press escape to start navigating using Normal mode commands.