I have a natural color panama style straw hat. It has a black hat band. In the summer, I do not wear too much with black. Such as a black belt and black shoes. Do you think it is a problem to wear light brown belt and shoes with a hat with a black hat band? I do wear it because I do not have a similar hat with a brown band. But I have always been a little self conscious about doing so. What do y'all think?