Finally, after a few days, I figured out what's wrong with my 3D 'engine'
I was either storing the rotation of an object as a Matrix4 or as a Quaternion, but really all rotational properties an object has are the x, y and z angles
@SimonAndréForsberg I found some more interesting posts related to HearthStone