Keeping suits and overcoats clean - brush or vac?

Unless we live in hermetically sealed bubbles, we all have to encounter the dust of building-site materials and fine particles of life's debris in the air as we go around our cities and transport systems, and our clothes are a magnet to all this, let alone what we have to breathe!

So I get paranoid about keeping all my suits and overcoats clean on a day-to-day basis (particularly the good ones). To that end I use a small hand-held vacuum to lightly draw off the dust rather than use a brush, but I am concerned that this method might be loosening fibres in the weave of the cloth without me knowing it. I have used a brush in the past but tend to think that it never really gets the dust out of the cloth and that it also loosens the fibres. What does anyone think of the vacuum approach?