hm. USB2 FS would still be at a nominal clock rate of 12 MHz, so I'd expect the strongest harmonic to be at 36 MHz, not at close to 40 MHz. There might be some intermodulation of USB and some 1 – 2 MHz clock going on, or your USB is downconverting some 100 MHz clock, maybe? In both cases, adding low-pass behaviour might help, and so "it's just other frequencies" isn't quite helpful – you can plug many holes in a boat, true, but only those below the waterline really matter. Other frequencies don't contain the power you need to stop from escaping.

Can you post all your layers of the layout individually?

Are you constrained to use a 2-layer PCB? A 4-layer PCB with data lines in inner layers would reduce the external emissions.

Yes Rahmany, it has to be 2 layers

the thermal relief on the ground pins/pads is a bit extreme. Try to make it wider (or even better: get rid of it). Why is the ground of J1 connected to the actual GND through a 0R resistor? Which is then connected to GND with narrow tracks (thermal relief)?. To me, this looks like a power integrity problem.

Noted on the thermal relief and potential power integrity issue. The idea behind the 0 ohm in series after the bead connection of shield and ground was taken straight from a Razer mouse: imgur.com/3VoJM1Q Computer mice have similar connectors and product specifications (usb powered, plastic enclosure etc)

@MarcusMüller The bandwidth of the signal is determined by the rise/fall time, not the clock rate. You can have a 100kHz clock signal driven by a modern small-die CMOS part and it'll happily radiate in the 300MHz+ range on every transition due to the Tr/Tf being short.

@Polynomial but I'm not talking about bandwidth. The PSD shows clear peaks in the spectrum – something periodic is happening there.

@MarcusMüller The cut-off of the (33Ohm, 47p) filter is 102.6MHz. Isn't that too high for the 12MHz clock rate you mentioned. I want to plug those holes below the waterline. Should I increase R to 100 Ohms for a 33Mhz cut-off?

@MarcusMüller For sure, but you can't assume that the frequencies of concern are always directly matched to the clock rate. My bet would be that the rise and fall times on the USB transceiver are a little faster than the bare minimum needed to meet the timing requirements, hence the upward shift in frequency.

@Polynomial as you identified, the edge speeds are responsible for the bandwidth, but the absolute symbol rate for any shift (think about it as convolving a rectangular wave, which consists of a sum of sines at odd multiples of the fundament frequency, with a low-pass filter: convolution is linear, so convolving with a sum is the same as summing up the convolutions with the summands, and convolving with a sine is mathematically a frequency shift. The filter is always a low-passy one, so it's never shifting up – instead, it forms the bandwidth that's being shifted up.)

@Polynomial no harm done! That was a fine refresher for me as well. Also, an excellent pun that you just made!!

First of all, I only have questions. If you can answer them, maybe you find the reason. 1) You have a captive cable. Are you sure it is rated for 12 MBps speed? 2) The cable groubing is dubious. Why resistor to GND plane, why ferrite to shield? Should shield even have a DC path to GND plane? 3) The 2-layer board likely has completely wrong impedance for USB differential data pair. 4) The data pair is not matched in length, but likely not a huge issue. 5) Which MCU or chip is that? Why it needs a RC filter on pins, is that from appnote? Is it well bypassed, especially if it has USB supply pins?

@Justme I dig the socratic method. 1) Not sure if it's rated for this speed. We used the circuit and connector of a computer mouse which was the closest to our application mechanically. Maybe mice work with low speed. 2) Same. 3) I have to use 2 layer board I'm afraid. 4) Made the D+D- tracks as short as possible from connector to MCU. 5) STM32F411RCT6. R,C values copied from EMIF02-USB01F2 datasheet (EMI filter). Seemingly very well bypassed. Not sure what you mean by USB supply pins. There's a USBFS_VBUS pin on the MCU connecting directly to '5V' of the above schematic.

@Justme found your answer here: electronics.stackexchange.com/questions/502245/… Very helpful tips. What capacitor value do you recommend for the shild ground connection?

@Geo It might be best to read USB implementation appnotes, as it depends on a lot of things how the grounding should be made. But at least STM32 appnotes say the USB IO pins already contain embedded matching resistors. Adding external RC filter might make things worse. If you need a filter, a USB specific common mode choke could work. And ESD protection might be a good idea, you never know if an ESD event happens via data pin.

@Geo you may be constrained to a 2-layer PCB, but the trace width and separation will still affect its characteristic impedance. What's on the other layer in the same area where the USB D+/D- traces are?

@BenVoigt There is nothing on the bottom layer in the same area, what you see is the footprints of the resistors and caps which are on the front layer, I just didn't know how to hide them for the screenshot.

If and when you slap a ferrite on the cable, which you should definitely do, you want to look for a ferrite that has substantial impedance or attenuation at the frequencies where you are failing the worst. Some ferrites don't have much attenuation below 50 MHz. You need decent attenuation between 30 and 100 MHz. So look for that.

What USB host were you using for the test? Does it pass with some other device that has similar cable? Especially many laptops seem to be quite borderline when it comes to radiated emissions with cables attached. Specialized test hosts can be used instead.

I think this is a really cool EMI question.. Everyone has different and conflicting advice 😂. Your task is to carefully read all the comments and answers, watch for similarities and disagreement and try to understand what is right and wrong and why 😊

Like all mythical EMI issues, they can be either solved by understanding and precise action, or by incremental steps (that may or may not improve anything) of shielding, clumping ferrites and adding more components 😉 Choose your weapon

In addition to the stuff here, I recommend you watch some YouTube talks of Rick Hartley

@tobalt Thanks for the asnwers

@jpa The host was a laptop, placed on a table outside the chamber connected through a 10m USB extender cable. The laptop has not itself been tested to see how it radiates. It may well be that the laptop is the source of some of the noise? I should ask to test the laptop on its own next time.