I agree that the President (and Congress) get too much credit and too much blame for the economy, but the fact is that people vote their pocketbook and their personal financial situation.. and perception becomes reality.
I mentioned 2006 exit polls. Look at two Senate races, Missouri and Montana, where incumbent Republicans lost very close races that ultimately gave the Democrats a majority in the Senate:
Q - Which best describes your family's financial situation?:
Missouri: the less than 1/3 of voters who said they "feel they are getting ahead financial" voted overwhelmingly R and the more than 2/3 who said they "feel they are just maintaining their standard of living or falling behind voting overwhelmingly D:
http://www.msnbc.msn.com/id/14349811/
Montana: the less than 1/3 of voters who said they "feel they are getting ahead financial" voted overwhelmingly R and the more than 2/3 who said they "feel they are just maintaining their standard of living or falling behind voting overwhelmingly D
http://www.msnbc.msn.com/id/14349812/
Your views on economic issues may not be partisan, but it is partisan for many (most?) voters. Perceptions or a "feeling of getting ahead financially or not" can and does have a major impact on elections.