Hello Archibalgul
Yes you are right phantom power is only available when using XLR cables, not with jacks (TS or TRS).
" IIRC, phantom power could cause funny things while plugging in/out a TRS plug or using a TS plug then."
When Phantom power is on : NEVER, and i insist, NEVER EVER disconnect any cables.
You must first turn off the +48 V , using XLR or not, doing this is the way to ensure you won't fry anything.
Cheers.