I completely understand the argument that God doesn't care what you wear, etc. And I agree with it, but I also think it's off point.
Dressing up for certain occasions is not (or should not be) about pride or - even worse - showing up others, but it is a way for a culture to say "this is an important event / this we respect."
So, IMHO, dressing for church is not some philosophical incongruity with Jesus' poverty (I've heard that one), but it is a way for a society to mark the specialness of going to church.
It signals to the younger members that this institution is important and it - even when we are (grumbling sometimes) putting on a suit - reminds the adults that there's a significance to going to church.
This holds - maybe with less theological meaning - for school and work and fine restaurants, etc. When we (used to) dress up for these places we were saying - as a culture / a society, "these are important institutions and activities and we reflect that by dressing in a serious and appropriate way."
And - in a nice restaurant, for example - we are saying to each other that we respect you (the other diners) and are dressing to keep the environment attractive for all. It used be thought disrespectful to dress sloppily in places where nice dress was expected.
Of course, any kind and well-intentioned society would never look down on anyone unable to afford - or not knowledgable in how - to dress for, say, church. A kind society welcomes them with open arms and, hopefully and respectfully, has practices and institutions to help those in need.
To be sure, we can live fine the way were are today where we have said, effectively, "dress how you want / your individual taste and comfort are paramount." But it is, IMHO, a sign of selfishness or, at least, self-centeredness where we don't want to voluntarily conform to a set of social norms that helps signal to all the importance of certain culture institutions and practices and that shows respect to all who attend.
I'm not a scold. This is our society today - the do-what-you-want society - and, my guess, if we take other things seriously, we'll be fine. But the societal norm around dress that proceeded ours wasn't simply for fussiness; it was a standard to acknowledge and reinforce our commitment and respect for certain institutions and practices - and for each other. It also served to educate the young and those who wanted to stray that these things mattered to the larger community.
Today, we've, essentially, said "this isn't necessary anymore." We've said - by dropping almost all standards for dress - that "we don't need these cultural signposts and guardrails." We've said that "arrant individual expression is more important than the social signaling of culturally accepted dress codes." We'll see over time how this all works out.