That map that guy44 posted is basically what I've thought would be a successful plan for a while (which scares me if it's also what Barak Obama believes
).
You have a bunch of very-high-traffic areas, like Sacramento-to-San Diego (Pacific), Dallas-to-Miami (Gulf), Boston-to-Richmond (NEC), Buffalo-to-Milwaukee (Great Lakes), Minneapolis-to-New Orleans (Mississippi), and the like. Then you connect them with stretches that would see more sparse traffic and connect a few additional cities.
I've taken the normal Amtrak train from Baltimore to Philly and to Newark and it was pretty good even at that speed.
The problem is that there is no way that the track can be laid without major federal government money, since there would need to be brand-new track laid for this to be feasible.
Really, the entire rail system needs replacement. It's a pretty big issue here in Baltimore after a major fire in the tunnel under downtown that is the single way to get freight down the East Coast.