Given your formulation, the Galilean structure by definition extends the affine structure of the space, so any transformations preserving the Galilean structure ought to preserve the affine structure, too. Let $\phi:\mathbb{R}^3\times\mathbb{R}\rightarrow\mathbb{R}^3\times\mathbb{R}$ be a Galilean transformation. Let $R\in\mathrm{O}(3)$, $\tau\in\mathbb{R}$ and $\mathbf{v},\mathbf{y}\in\mathbb{R}^3$. We can show that
$$\phi: \begin{pmatrix}
\mathbf{x} \\
t
\end{pmatrix}\mapsto \begin{pmatrix}
R & \mathbf{v} \\