You don't need another pot - you only need to form a voltage divider with the pot that's already there. If your preamp has 50dB of gain, then you would would probably need to cut it down by 20dB. If your original pot is 20k, then 200k ohm resistors would do that very well, and not add noise or raise the output impedance of your line as external voltage dividers (attenuators) have to do.

It's impossible for a potentiometer to increase gain if it's wired correctly. Potentiometers operate by attenuation only.