@MikeMiller I'm not looking at the fiberwise kernel, I'm looking at the kernel bundle.
@DenisNardin I think the category of vector bundles over $X$ and then over $Y$ are the correct settings: we start by forming the kernel bundle in the category of bundles over $Y$ and then base change to obtain the vertical bundle, which is over $X$.
I tried to be detailed in my linked question :)