I agree with others on their aversion to sweaters with zippers, although I had an argument with my brother about them a couple of years ago (he's also a clothes horse) and he was able to demonstrate to me that zippered sweaters went back much further in time than I had thought. I accept, though do not endorse, them as outdoor wear. They still have no place under a jacket.
I will disagree with those who say "v only" under suits or jackets. Several years ago, Paul Stuart ran a photo in their fall catalog showing a heathered-blue polo-necked sweater worn under a grey suit, over a shirt and tie. I thought it looked wonderful, as Paul Stuart's fall catalog photos usually do, and promptly copied the idea. I still do it a half dozen times each fall/winter, and have gotten a number of compliments. It's obviously a less formal look, but it has a nice layered compexity to it that appeals to me. I think it's sort of the sweater equivalent of lapels on a waistcoat.
FWIW, I generally eschew sweater vests. They tend to be made of bulky knits, added girth where I don't need it. I prefer tailored vests/waistcoats if I need a sleeveless garment, and a thin merino knit with sleeves if I need something softer/more casual. Besides, in America, sweater vests have rather risible Ned Flanders-type connotations.