Ideally we want very little current to flow when the switch is closed.
For the bipolar transistor, a darlington arangement using two transistors
would work well.
Then the 100k resistor could be increased to at least 1Megohm, I think.
Here is another circuit to play with. I found this circuit here:
Using a MOSFET as a Switch
Using Paint Shop Pro I flipped the positions of R1 and the switch.
I would change the 1M resistor (R1) to 5M or 10M and see if the circuit is still
stable. The project could still be powered by 3 volts but R2 should be a much
smaller value if the LED has to be bright. Perhaps 220 ohms.