Well, two things. First of all, do you have the proper resistors in series with the LEDs? Otherwise they'll burn out pretty quickly.
Second, if you just connect a wire from the positive terminal of your power supply to the LED, and the negative terminal of the LED to position 2 on the switch, it should work fine...I'll sketch a circuit for you and see if I can find a spot to upload it.
Edit: There ya go...make sense? Your just switching which LED gets to see the complete circuit.
