Don't worry, your MxB appears to be in good health.
The Brute Factor is a very brutal thing... It'll demolish your house and take your head off if you're not careful
It's indeed about finding sweet spots. The exact result is highly dependent on filter mode and the other filter settings, as it is a feedback loop and thus highly reactive and kind of unpredictable, taking you into the realm of non-linearity and chaos. There is a lot of interplay between Drive, Resonance and Brute Factor.
As you have discovered in low pass mode, the BF can be brutal - although if you adjust it carefully, and work with Drive and Resonance as well, you can get nice controlled results. In high pass mode the effect is also quite big. With the Ladder in band pass mode, the BF hasn't got a very dramatic effect, but instead becomes subtle and takes away some of the resonance. With the Steiner in band pass mode, the BF is more dramatic and actually adds to the resonance.
With the filter routing in series mode, Bruting up the Steiner and then taming and shaping that with the Ladder can be effective. There are strange and unique sounds to be had with subtle adjustments. Sometimes I get wonderful formants more than distortion, yielding the sound of vocals and 'alien' instruments and so forth. There are worlds of sonic wonder to discover there.
One interesting thing to observe is that when you quickly move the BF knobs, the sound may change slowly (perhaps up to a second or so) even after you stop moving the knob. This is the complex effects of feedback causing the system to take some time to reach equilibrium. Adding modulation assignments into the mix there will reach into new territories of dynamic harmonic changes.
While the BF may be bewildering at first, it greatly expands the range of sounds to be had, and not necessarily in the form of extreme distortion.
