After investigating for a while now, my guess is that the discontinuities in the IF are causing the unwanted peaks as @SteveSh suggested. However "inside" these discontinuities the oscillations seems to represent the target/targets infront of the antennas, but these oscillations does not show up in the FFT due to the discontinuities. Why these discontinuities occur is a mystery to me. I've tried using a potentiometer to manually tune the VCO which gave some strange results as well

If I'm correct i should only observe a constant DC at the IF when slowly increasing the frequency of the VCO. However the DC that is observed at the IF seems to increase / decrease depending on the tuning voltage to the VCO, which is probably the reason the discontinuities show up.

These discontinuities are an artifact of using the stepped chirp waveform. You are trying to stitch together phase and amplitude information from one chirp to the other. In order for this to work properly, the IF output step-to-step needs to be continuous.

You need a more intelligent VCO to do what you want. In addition to just changing the frequency, you need to be able to control the starting amplitude and phase of the VCO's output, so that of chirp n ends at 1V and 235 deg, chirp n+1 starts at 1V and 235 deg.

This is a common problem, solved with complicated frequency synthesizers, when you're trying to synthesize a wide instantaneous bandwidth (IBW) waveform (say 2 GHz) with hardware that only supports a fractional IBW of that range, say 200 MHz, which wold require 10 chirps each with a different frequency slope.

@SteveSh Yeah, i've came to the same conclution that it is the phase that causes this problem. I have tried using a triangular chirp instead which gave some promising results, the phase missmatch is still there but much less prominant. Will be doing an update as soon as i get better data, thank you for leading me in the right direction!

@Allfadern Are you really trying to stitch together multiple sweeps in range? Normally, one would de-chirp and DFT one sweep at a time.