First of all, all of us make mistakes and you need not be more than mildly embarrassed by past errors. The user with the most reputation points on the site, Joel David Hamkins, has also deleted on occasion. So you're in very good company here; all we can do is try our best to minimize our mistakes and just keep learning -- all of us here are lifelong ...
Let me digress a bit and respond just to this part of your question:
It is little embarrassing to know that my stupid answer/question is still visible to so many users.
At least in the case of the questions, they are not "that much visible". Users with 10k+ reputation points can see deleted posts provided they have the link.
So they can find and see ...
Such a system would be technically impossible since we can't delay posting a question until tags have been approved. However, this is not a serious problem:
Only users with at least 300 points can create new tags.
Users are quick to delete inappropriate tags from posts.
Unused tags get purged in 24 hours; single use tags get deleted after 6 months.