Have you considered that most of the early buildings in the FC were basically built of mud brick, a material not really noted for longevity? The buildings fall down (or are torn down), people build on top of the rubble. After a few centuries or millenia of habitation, the old buildings have become a tell: en.wikipedia.org/wiki/Tell_(archaeology)

@jamesqf An interesting Wikipedia page. What is the criteria for damaged buildings subsequently restored? (Thinking especially of the Aula Palatina, which was extensively damaged in WWII.)

To the OP, note that if by "civilization" you have in mind large-scale civilizations like Sumer, then the answer is simple: That sort of civilization is not required for building, and those early extant buildings from Europe pre-date large-scale civilization (not just in Europe but anywhere on earth). If buildings from the first years of Sumer as a large scale civilization were extant, they would still be roughly a millenium younger than the Cairn of Barnenez.

To the OP, note the title of the WP page: "List of oldest known surviving buildings", which is not exactly how you characterize it. The difference is very important, yet you constantly ignore it through your question...

@C Monsour: We should note that the list the OP links to is of things considered to be buildings - basically roofed structures. It thus excludes the thousands of stone circles in Britain & adjacent Europe. Their existence argues that there must have been a fairly large-scale civilization capable of constructing them.

@jamesqf No, it doesn't. (a) It does not require a political organization larger than the size of a single tribe or town to build a stone circle. (b) Stonehenge is way younger than both the Cairn of Bernenez and is even younger than large-scale civilization in the Fertile Crescent. The Cairn of Bernenez dates to about 4850 BC. The beginning of large-scale civilization on earth dates to about 4000 BC. The stone structure at Stonehenge dates to about 2500 BC. Chronology matters.

@C Monsour: To build one small stone circle, perhaps a town is sufficient, just as a small town today can build a church. But building Stonehenge or Notre Dame takes rather more than a town. And building thousands of stone circles or churches to a similar plan presumes a shared culture. WRT chronology, there is also a similarity to the chambered tombs &c that predate Stonehenge &c, thus a shared culture.

@jamesqf You must be really confused to be bringing medieval churches into this discussion

@jamesqf Also, large-scale civilization implies a lot more than a shared culture. Cultural elements disperse far beyond the extent of political control! And it's fortunate that that is the case, or farming would never have spread to present-day France by 5000 BC, but obviously it had.

@CMonsour Farming did not spread into Europe it was brought in by neolithic immigrants. It took over 2000 years for farming to spread across Europe.

@Daniel Yes, brought by immigrants, but by immigrants from small scale neolithic farming communities, not immigrants from great civilizations. My point is that the neolithic predates the existence of massive political entities by several millenia.

@C Monsour: Well, you must be pretty dense not to understand that it's an analogy: two different societies constructing both small public structures and large, elaborate ones to the same general pattern/purpose.

what about this, they are not buildings? en.wikipedia.org/wiki/G%C3%B6bekli_Tepe

@Luiz Whether they were buildings depends on whether they originally included walls and a roof. They don't count as surviving buildings since they no longer have those features. However, that wikipedia article does rather make the same point as my point about Stonehenge--and then some--since it speculates that these might be structures built by hunter/gatherers (i.e., paleolithic), whereas I was at least allowing that that needed neolithic, though not large-scale civilization...

@CMonsour This definition of "building" underscores the selection bias built into the Wikipedia list.