Ya, cut down on the weed. The only time I have panic attacks is when I smoke weed. Well, too much weed. If your paranoia persists, go see a doctor. Maybe try other things like exercise or a hobby. Try to get your mind off thinking too much. You even said its not as bad as it seems. Try to do it the natural way. Sometimes medication isn't the best answer. It can change a person into something their not. But in other cases it can be positive. Go see a doctor as a last resort. Maybe try a therapist for now. Good luck man.