When you're the illustrator, you can make anything work!
But melding an ensemble by tying it together with something above and below the waist is a useful tool. And the illustrator cheats further by making the browns, rich, "high" browns that tend to accessorize (And flatter!) more easily.
I've worn mid-brown sports jackets (flat hard cotton, rather than rich textured wool) with black or charcoal full-sleeve polos which are made of flat, soft pima cotton. A lighter colour in a pair of corduroy trousers (khaki, beige, rust) would combine well with this shirt and jacket. I think such combinations can work, but one must be careful about the shades and textures. Although I rarely wear black shoes, I imagine a pair of black loafers might go well with the ensemble I described above. In general brown, black and beige in various shades go well together, and here, playing with texture can create some lovely effects. Black shoes will also go with these mixtures.