Casual suits, while meant to be worn in a relaxed environment, can be tougher to pull off than a business setup. It can be hard to look good, but look relaxed doing it!
Most guys will say casual wear just means taking a regular suit and just not wearing the tie. That's doable, but doesn't always make it a good look. Even just removing a tie can change the whole structure. Your collar flops out and you're going to be showing off a whole lot more chest. This can be a good or bad thing depending on if you're in good shape there. You'll want to make sure your collar isn't spilling out of control. It's OK to show some chest but if you've got a wolfman thing going, you may want to think twice about it.
You might think that pinstripes are a more relaxed or fun look. It can work, but for some reason I think it looks a little out of place without a tie. I think a basic solid suit is a better idea though.
As far as a fabric, cotton or tweed can be seen as more leisurely. You can certainly stick with a wool, but perhaps a lighter weight. You might be out and about a little more so it will keep you a little bit cooler.
Navy blue and grey are known as conservative, "business" colors. Anything outside of that tends to be a little more casual in nature. A khaki or tan, or a nice brown can look nice. Black is also an option, but to me it says "club" more than anything. A nice peak label can look a little more relaxed, especially with no tie.