Help me build a small, versatile, non-business suit wardrobe

I need some help rounding out a small suit wardrobe, leaning more on the casual/cocktail side, not so much on the conservative/business side.

I like to wear suits for social events, weddings, shows, nice dinners, special events, etc., and don't wear a suit for work. I am in my mid 40s, in great shape and live a pretty social and active lifestyle. I live in Southern California where it's 70-80º F almost every day. It rains occasionally.

I currently own two suits:

1. Navy Suit: Navy two-piece with worsted medium weight wool, two button, notched lapel. It's a well fitting general purpose workhorse but feels a little too formal/conservative in some settings. Sometimes I wish I got a hopsack or another more open/textured weave.
2. Tuxedo: Black classic peak lapel tuxedo for the occasional gala, black tie event. Perfecto.

I'm stuck on the idea that I am missing one or two key suits, specifically in the cocktail/semi-formal/elevated casual realm. Khaki cotton? Linen? Mid-gray? Something with a pattern?

If you were me and didn't mind throwing a little money at the problem, how many additional suits would you get and which ones? Feel free to recommend color, fabric weight, pattern, texture, etc. Thanks!