See my answer here: https://physics.stackexchange.com/a/819681/128186
I end up having to use the same facts about $\sigma$ and $J$ as were used in your answer. However, instead of taking derivatives, I use the ad, Ad, $\exp$ relationship. But, I believe, your method of taking derivatives is how the ad, Ad, $\exp$ relation is **proven**. So in that sense the two answers are nicely tied together.