vimdir.vim (GitHub: c0r73x / vimdir.vim, License: MIT) by Christian Persson is a plugin that allows you to manage files and directories in Vim. You can list files and folders recursively with
:VimdirR, and then edit the buffer to change the layout of the filesystem.
:Vimdir will list the files in the current directory. You can then use Vim's search tools to find a particular file and
dd to delete the line. Saving the buffer will cause vimdir to persist the changes. It'll even delete directories, so you should be careful when using it.
If you're totally taken by the idea of a buffer-based file manager, then you can even use the vimdir script to pull up Vim with vimdir when by typing
vimdir in the terminal.