Well, the main issue is that all those images are delivered to you from a CDN ("content delivery network") that has servers all over the world, so when your browser downloads things like the favicon, it doesn't have to call New York (were our datacenter is), but can call a CDN server (called "edge server") closer to you, so you get the images faster.
These edge servers cache the images for 24 hours, so when the first user requests "favicon.ico", the edge server has to ask New York itself, but for everyone else it can just remember what it sent previously. That only turns into a problem when…