Interesting. I do not know so much about the implementation details of the tool bar. But, from what I read up to now I think the tool bar uses directly some OS stuff. AFAIK the tool bar under Mac OS supports some nasty features like lenses. Maybe it scales the icons all to squares for some reason.
Could you run `M-: (nth 5 (cdr tool-bar-map)) in the Messages buffer? Under Linux that returns (separator-1 "--") . It would be interesting what they map to nothing under MacOS. I assume it is also (separator-1 "--") for you.
... At least that would be consistent with your statement that the separator in the AUCTeX tool bar vanishes when you replace (nth 7 (cdr tool-bar-map)) with (separator-1 "--").
My son woke up and I have to play with him... So I will continue later on. Would you be happy with the vanishing separator? That would at least be consistent behavior under MacOS.
We could try to replace the xpm image with an imageimage image. Would that be an interesting test for you? Depending on your answer I would post an answer in some hours from now.
You can test whether your Emacs has imagemagick support by (imagemagick-types). If that list is empty you don't have imagemagick support. Another interesting test is: (memq 'XPM (imagemagick-types)). That should return a list if xpm is supported by imagemagick. With imagemagick you can set the image width to 2 and the height to 24. That is what I am after.
@Tobias Both (imagemagick-types) and (memq 'XPM (imagemagick-types)) return an error like this. Running (image-type-available-p 'imagemagick) gives nil. I guess my Emacs for Mac OS X doesn't have the ImageMagick support.
Missing imagemagick support is a pity. Do you accept the vanishing separator? (Just stopping by at my computer on the way to the next play with my child.)