I haven't been through rural Quebec, but Montreal and Quebec City both have a lot of what you're looking for, especially in their Old Quarters. If your budget is tight, then definitely consider it. Just remember that those who come from there are not "French" but Quebecois.
On the other hand, Quebec doesn't have the Eiffel Tower or the Louvre, or the Cote d'azure and on and on and on. I've been to France twice now, and believe it's a must see for any worldwide traveller, so if do try to get there someday.