@Thorgott Potential counterexample: There is a 4-manifold with boundary $(M, \partial M)$ and an involution $f : \partial M \to \partial M$ such that $M \cup_f M = S^4$ and $f$ extends to a homeomorphism of $M$ (but not a diffeomorphism).
Let $D^5$ be a ball bounding this $S^4$, treat it as a manifold with corners whose boundary faces are the two copies of $M$ and the corner is $\partial M$ where they intersect. Let $X_1, X_2$ be two copies of this 5-ball with corners. Glue $X_1$ and $X_2$ along a pair of boundary face $M$ by the extension of $f$ to $M$. Then this union $X_1 \cup X_2$ seem…