Button down or not is one choice. Oxford, pinpoint oxford or end on end broadcloth is another. If you want a lighter fabric then the end on end will probably be more satisfactory. Certainly BD shirts fit the bill if you want a more casual look (no tie) so that may be more important. So maybe go for end on end BD shirts?!