On the drivers side of the engine..just to YOUR right of the valve cover looking backwards, you will see the fuel rail with each injector connected to it.
Just under your head at the front of the engine nearest you on that rail is a round valve that the fuel return comes from. It is around 2%26quot; diameter and small black vacuum line goint to it.
That is the regulator.
I REALLY doubt this is a problem as you decribe. Had two leak in 15 years time,,but not do this as you describe..More concerned about the new pump!!!
Check with your auto parts store and you can perhaps borrow a fuel pressure gauge. The pressure test point is a Schroeder valve..Kind of like a tire port in the center of the fuel rail. DO NOT USE A TIRE Gauge!! While it MIGHT work,,it is dangerous due to leaking 39 psi fuel.
Engine at idle you should see around 30 psi,Disconnect the vacuum line and it should read right around 39 psi.
Good Luck