Hi Sinisa, and welcome on this board.
Sorry you're encountering this issue.
You're right, in the minilab MKII manual, details about relative modes are not explained, we have to update the manual for that, thanks for the info!
Actually the 3 relative modes are the same as the minilab mk1, here is how they work :
Relative 1:
The knob will send values 61-63 when turned in a negative direction and values 65-67 when turned in a positive direction.The turn speed determines the parameter response.
Relative 2:
The knob will send values 125-127 when turned in a negative direction and values 1-3 when turned in a positive direction.
The turn speed determines the parameter response.
Relative 3:
The knob will send values 13-15 when turned in a negative direction and values 17-19 when turned in a positive direction.
The turn speed determines the parameter response.
Please note that a "0" value will be sent between each steps. i.e in relative mode 1 : moving (slowly) right will send 65/64/65/64/65 etc...
This addition of this "0" value has been made to prevent browsing issues in Ableton Live. This parameter was called "knob fix" in Midi Control Center, but has been hard implemented on last firmware versions.
Please note that each DAW has it's own way to handle everything midi-related. I.e Relative mode in Ableton Live equals realtive mode 2 in the minilab.
As i don't know which version of Studio one and OS version you are using. I Unfortunately can't really help you more on this case.
In the meantime, suggest you contact Presonus support to ask them how relative modes are handled in studio one.
Hope it helped a bit anyway.
Best,