Can recall a similar scenario some years ago before I retired. Invited to a club by a member for lunch along with a colleague . The club having a dress code would at times allow non members to dine. As we sat, two gentlemen walked in to dine very casually dressed, jacket less, tee shirts and casual trousers. Our host immediately called over the Maitre De and directed him to refuse seating the pair. It wasn't so long ago that at many fine restaurants had a dress code, alas such may not be the case any longer.
I know of only one within driving distance from home and I book early.
No snob am I, very far from it but I will not succumb to what some millennials wish to dictate to pass for decent dress.