If you look up some of the other threads regarding nutrition I have replied to, you will notice a common theme, that echoes what abaya and paddy said: EAT MORE FIBER.
If it is a carbohydrate, and doesn't have fiber, you should not be sticking it in your face--or at least not often. Eat more whole grains, fruits, and vegetables. You should be able to get enough fiber from your diet without taking a fiber supplement. Buy bread with at least 3-4g of fiber a slice. Cereal should have a significant amount of fiber per serving. Read labels--there is a LOT of misleading advertising and labeling out there on boxes. A lot of foods claim to have whole grains but then only have 1g of fiber. As a guy, your goal should be to eat about 30g of fiber a day. It's actually easier than you think.
As for meat and potatoes--that's fine, so long as you're leaving the skins on the potatoes
Think of your plate in thirds--one third protein, one third whole grains, one third fruits and vegetables. A baked potato, in this case, would fit under whole grains (5g of fiber per small potato). So the other two slots would be meat (watch the portion size), and green beans/corn/peas/whatever. I also like to eat a side salad before I even eat my dinner.
Meat and potatoes is fine as long as you're not viewing a potato as a vegetable, and not counting it in your five a day. It's also fine so long as it's not the only thing you're eating--you need to eat a variety of foods.