Finding a good virtual assistant has worked for many businesses.
It does take time to on-board and like any other activity, there is a broad spectrum of competence.
Other ways to improve your situation is to do an audit of your external communications channels and advertising.
If you can laser target your communication to the right group of people and make sure it is easy to find the info they need, you can both attract more of the right people and deter the wrong people from calling.
This is a HUGE time saver.
Cheers
Paul