Let the compression away for a while. There are some important considerations to do first:
Sound Pressure as perceived aparent volume depends on your cabinet's efficiency to convert the power of an Amp in air motion by a Loudspeaker. So, if you want more volume you need a better cabinet first and more powerfull amp second (the main factor still is your cabinet's sensitivity). Or you must have an individual bus out to feed your own stage monitor at least...
An Electric Bass doesn't makes sound, it produces an electric signal that translates (or describes) string's motion. That signal is very weak and needs to be amplified several times before being able to push a heavy loudspeaker's cone.
But this signal changes over time, first it pumps a very quick unorganized energy burst called Transient. It happens when we first release the string and it strongly starts to vibrate, expressing the initial attack of a note. It is so quick and shows so many stacked frequencies that we can't recognize pitch, tone or volume, just the snap or kick feel.
After that initial bounce, the movement gets more stable and we can finally recognize pitch and volume. In fact, we can't feel the exact power of each fluctuation on pressure since they occurs so quick, we perceive loudness by a kind of average of this oscilations called RMS (root mean square). So, in a Bass signal we have 2 meaningful data: peak level and average sound pressure level.
The first is important mostly to our gear, since excessive signal can harm some components, but clipping can chop off peaks and change the transients affecting an important part of the tone, too. On other hand, we're used to be more sensitive to loudness, so we focus on pushing volume up until sound pressure feels right.
The problem is that a Bass can produce Transients many times stronger than its average level and an audio gear must accomodate both. So, if the average is on a good level (nominal level - 0VU) the peaks will be probably got clipped, but if peaks are safely under clipping level the average will be most probably much under proper level (since too low levels always result in too much electrical background noise contamination during further amplifying steps). This is a headroom issue because headroom is how much signal one gear can deal with above its optimun level (0 VU) without clipping. A quality console can give you more than +20dB above 0VU to reproduce peaks properly, but cheap ones doesn't and this can be a real problem when reproducing a Bass.
This is the only reason to control dynamics, but you must be aware that any changing in wave format is distorting it, too. So, attenuating peaks means deform tone's envelope and tone. For example, if you use too long attack settings on your compressor, it will not affect transients and will increase level discrepances between peaks and average. If it is set too quick, it will rip off any peak and this'll make the bass sounds rubbery and floppy (compromising its percussive character). So, it is very difficult to properly adjust a compressor and very easy to mess it all.
Even knowing the B stuff are limited in many ways, it is not normal one having to choose between clipping or hearing. Did you ever tried without the GT10? Don't know but may be this compression set is worsening your problems. Extreme Eq settings always mess things up, too, when we try to equalize the room on the instrument instead on the PA/FOH. Many experienced musicians forget that the sound on stage usually is totaly different from FOH and we can't Eq directly on Instrument unless our reference response would be perfectly flat at our listening position (and they never are). Imagine yourself monitoring your bass tone in a tiny 10 speaker cabinet and trying to compensate its lack of bass, while sending that very same signal to a gigantic PA system equiped with thunderous 18 subwoofers. This mismatching always causes troubles and keeping things simple is a good way to avoid problems.
Try to test what happens discarding the compressor. Try swith off any amp simulation (you just need these when recording in a bad room). Don't adjust your monitor's tone on GT10 if it is not the PA, it must be done at the console aux or bus out (some small venues uses just the Amp to feed public, but in this case we must equalize from audience perspective, not at stage). If your problem is peaks, prefer to use a Limiter instead of a Compressor. Avoid using too fast attack, is important to keep at least some part of the transient to preserve tone. If GT10 doesn't helps you to control attack and release settings, try another modeled gear (different brands and models naturally have different attack and release times)
