Jeremy, thanks for asking, and thanks for your kind concern. In general, I'd say don't worry much about the workload of the moderators. Conversion to a comment takes two button clicks, and I find that on most days there aren't that many such flags that need to be dealt with. 

My main request is that the not-an-answer flag be applied reasonably accurately. As a baseline, I encourage everyone to have a look at this thread: https://meta.stackexchange.com/questions/185073/how-do-i-properly-use-the-not-an-answer-flag. In particular, consider the advice of using a custom flag if you're not sure. 

The meta stackexchange explanation doesn't quite address the phenomenon of (clearly) crackpot answers that fall into the "not even wrong" category, but I myself just about never object if such are flagged as "not an answer", even if legalistically they don't fall under the stated SE criteria. Same with mathematically irrelevant responses -- most of the time, I feel those situations are tolerably clear and I'm happy to be notified of such through not-an-answer flags. But sometimes I see not-an-answer applied to items that are simply wrong, as opposed to not even wrong, and there I caution that not-an-answer is not really the right flag. Downvoting seems like a better response there. Finally, sometimes I see "not an answer" applied to items which seem pretty close to being an answer good and true, for example a reference to a book that looks relevant. Try to avoid that: in such cases the respondent should instead be encouraged to flesh out a more informative and self-contained answer. 

A final note: most of the time I view the users who spend a lot of time going through the review queues as very helpful in moderation activity, so thanks! There are occasions where I feel some users rush to judgment too quickly, and these situations can be time-consuming deciding how to deal with, so as a general rule I ask everyone to please be careful handling questions and answers outside your range of expertise.