Just to say, I have yet to play either one. But I've played the hell out of 2003.
The only difference I can think of would be the extra courses, the Tiger Proofing (replayability), Legends Tour, and the extra features in GameFace. The core gameplay itself remains the same besides being able to move the position of the ball in your stance. It's been said that 2005 is just a tweaked and highly polished (perfected, even?) version of what 2004 set out to do.
Since you don't have the cash to spend, I'd go with 2004. You're still going to have a lot of fun with it, especially if you "dig a golf game." 2005 probably has better replayability, but while you're playing you should be having just as much fun with 2004.
Enjoy!
