That's the trick exactly. If I understand correctly, you've basically got 3 coats to work with except where you sanded through. That should be plenty.
Take a little extra care where you know you have only 2 coats. If you're just knocking down orange peel you'll have plenty of paint. Once the specks are gone stop. Remember, you'll be cutting some more off when you buff. I don't know of any way to tell by looking when you're about to break through. The real problem is when you're trying to sand out a where a bug had to do the Funky Chicken before it died. When you sand through enough times you'll kinda learn what's just enough. Then it's a matter of "Can I live with that?"