I did remove the MIDI Driver, is there anything else I could remove manually? I'm a software developer so I know my way around, if you have any other manual steps I could take, deleting files, etc, let me know.
Also, being a developer, this smells a lot like a software issue, not hardware, maybe it's a MacOS problem, I'm testing the AudioFuse in a Windows box as I type and it works perfectly fine (so far I just listened some MP3s using foobar2000). So I think this is not a hardware issue.