I used to remove the dongle when my PC was off and then plug the dongle back in before turning it on again. However, one time I did this the license disappeared from my dongle. Fortunately I did a system restore and my license reappeared.
I haven't been too keen on unplugging the dongle since then. So, yeah, should one unplug and plug in the dongle with the computer on or off?