Notice as a point of orthography that 'topos' is a French word, formed from 'topologie.' and not a Greek word. In writing, Grothendieck always forms the plural according to the French rule for words ending in 's,' so it is invariant—'les topos.' So the English plural ought to follow the English rule—'toposes.'
Freyd. a confessed lover of classical endings and the inventor of cosmoi and logoi among other types of categories, says he heard that Grothendieck spoke of 'topoi' in Buffalo. I regard this as biased hearsay which can not stand against the published record.