What I see when I hear "dress wallet" is a long wallet - one designed for the breast pocket of a jacket - as opposed to a billfold for the hip pocket of one's trousers. I have black and brown in both styles, though I rarely use the black billfold.
If I were to narrow it down, I would go for brown. In the harness and saddle sales business there's the notion that "only the best leather gets dyed black." In the harness making business, however, it's better known that "only the leather that takes black dye best gets dyed black." Generally speaking, the better leather is left brown. It lasts longer, smells more like good leather and adopts an attractive patina.