Specially on why R1b1b2a1 (the bulk of Y-DNA R1b and the Western European specific subclade) cannot be Neolithic. It comes up in all discussions and I really feel the need to explain: to create a generic post that serves as reference.
For that purpose I created a simplified map of European Neolithic cultural flows (excluding mostly Eastern Europe that anyhow had its own distinct processes):
Click to expand
The Early Neolithic
As you probably know, European Neolithic (defined by the existence of farming and animal husbandry, and also often of pottery) began by all accounts in Thessaly, Greece (Sesklo Culture). It's characterized by a pottery painted using mostly the colors red and white and with those colors it spread through the Balcans.
More intriguing are the origins of the Cardium-Imprinted Pottery culture, specific of the Western Balcans, who did not use color in their pottery and instead decorated them with patterned impressions often made with the shell of a mollusk of the genus Cardium (hence the name). Some early pottery of this kind has been found at the Thessalian site of Otzaki, along with finer pottery of Sesklo style, however it is only known as distinct culture once it became dominant in the Western Balcans.
This Balcanic duality defines the general duality of early European Neolithic because Cardium Pottery Culture would spread by the Mediterranean shores, largely by boat, while the main Balcanic Neolithic would eventually give rise to the Central European one, often known as Danubian Neolithic.
The Cardium Pottery Culture spread with some colonization but mostly by acculturation of the indigenous peoples, as is evidenced in the toolkit continuity in most sites. They were specialized in fishing and sheep/goat herding but also carried the full package of cereals and at least one legume: lentils. They built villages but also inhabited in caves. After the initial coastal expansion local expansions took place as well already under the tag of Epicardial, most importantly along the Rhone and Ebro rivers.
Meanwhile in NE Hungary (curiously the easternmost reaches of Epi-Magdalenian culture) an ill explained cultural shift happened: the Balcanic pottery style was replaced by an engraved one. It is the Eastern Lineal Pottery Culture, which had a very limited expansion on its own (mostly to Transylvania).
However a derived culture arose then in Western Hungary, Moravia and Eastern Austria: it is the Western Lineal Pottery Culture, more commonly known as Danubian Neolithic. This one had a massive spread through Central Europe reaching into Belgium and Northern France (where it shared the territory with other minor local Neolithic cultures for some time) by the West and into Moldavia by the East, where a previous local Neolithic culture was assimilated as well. It even influenced the area of Vallachia-Bulgaria at a later time, generating which was maybe the first large European state: the Karanovo VI-Gumelnita culture, older than dynastic Egypt but short-lived.
Now it seems also clear that late Danubian peoples from Northern France brought agriculture to Britain, simultaneously to Megalithic peoples from Western France.
But I'm going too fast.
The Late Neolithic
A key an often ignored region in the second Neolithic phase is SW Iberia (mostly southern Portugal). Here there was also an important cultural shift: the custom of clannic (collective) burial in megalithic tombs known as dolmens or trilithons appeared just a few centuries after the arrival of the first Neolithic influences. Dolmenic Megalithism in this area is older than anywhere else by at least a thousand years.
Only much later, c. 3800 BCE, this custom appears in Brittany and other areas of Western France. In the following centuries it would spread through all Atlantic Europe and many parts of the Western Mediterranean, as well as penetrating in parts of the late Danubian cultural area.
It has been speculated that cod fishing may have been related to this expansion, however there must be more than just a mere economic activity: Megalithism had clearly a cultural, probably religious, element to it.
Many areas of Atlantic Europe only reached Neolithic with the arrival of Megalithism or a few centuries before it.
Another key region is more obscure: Denmark and neighboring areas. Here there was an important seagoing culture known as Ertebölle, which has been claimed to be Neolithic... only to see others rejecting that claim. This is a key issue that should be clarified in order to understand the Nordic Neolithic.
Related to this issue is that of Funnelbeaker Culture (often named by its German acronym of TRBK), which is clearly Neolithic in the North and West, where it is also Megalithic but is not in the East (Mecklenburg, Northern Poland). Some had theorized that Funnelbeaker began in Denmark, derived from Ertebölle after the arrival of Eastern influences related to Pitted Ware (another allegedly hunter-gatherer culture of Neolithic times, seemingly rooted in Eastern European Neolithic), but this is contested by others who think it's derived from some northern offshoots of Danubian Neolithic.
The issue is very murky so I prefer to leave it as it is: a mere anotation of unsolved complexity.
All this sums up pretty well the essence of Neolithic in Europe (excluding the East), I believe. In order to synthesize I had to ignore many local groups, often interesting but that add little to the global picture.
Can R1b1b2a1 be Neolithic?
Many people seem to believe that, in spite of this haplogroup being most concentrated in the westernmost reaches of Europe and decaying towards the East.
The most biased ones love to oversimplify and ignore R1b1b2a1 altogether, talking instead of R1b1b2. This was the case with the recent paper by Balaresque et al. which produced the following haplotype structure, which I have duly annotated to indicate what is not R1b1b2a1 and that way unveil the truth:
Other means West-Central-North Europe
Click to expand
It is obvious that the star-like expansion of R1b1b2 happened already at the level of R1b1b2a1 and happened already in Europe: either in Central or West Europe.
It is obvious that it does not show two distinct centers of expansion nor two different waves as should be the case would it have any relation with European Neolithic spread but instead shows one and only one general expansion affecting essentially to West and Central Europe without any kind of pattern, at least not one that we can easily spot (thank Balaresque for that lousiness).
It is obvious that the center of expansion is not in the Balcans either.
The R1b1b2a1 star-like structure has 15 basal branches, of which at least half show a clear strong presence in Iberia (sample that does not include Basques, as the only Basque sample used by Balaresque was from the North and hence included along all French in the Other category). This alone would suggest that the origin of the expansion should not have been too far from Iberia.
Oddly enough, this graph clearly supports the hypothesis of R1b1b2a1 having expanded with Magdalenian culture from the Franco-Cantabrian refuge in the late Upper Paleolithic.
However I do not bet all my money to this hypothesis because the limited haplotype diversity data I manage rather suggests slightly higher diversity in Central and even Northern Europe (this last only with small samples).
But I do bet all my money to the haplogroup having expanded before Neolithic. There is simply no way that the rapid expansion that star-like structure so clearly indicates could have been caused by Neolithic cultural flows. These can explain the scatter of other lineages like E1b1b1, J2b, G2a, T and even some subclades of I maybe. But that's about all.
[Note: what follows is an update, some 10 or 12 hours after the first publication]
What about Megalithism?
The only possibility for R1b1b2a1 to have expanded within Neolithic would be within the frame of Dolmenic Megalithism. This would fit reasonably well with the haplogroup's spread area, with some notable exceptions and would allow for Portugal to act as a transition zone between R1b1b2a* (that has high diversity in this country, comparable to that of Turkey or Italy) and R1b1b2a1a.
However this poses several problems:
First, while Megalithism may be associated to the origins of agriculture in some areas, it is certainly not the case in most. Second, Megalithism shows high cultural diversity and appears related to many different local cultures: it is not a monolithic phenomenon at all, what should be the case if it was essentially one of demic expansion. Third, some crucial areas do not fit well: East and Central Iberia were never Megalithic but are high in R1b1b2a1, instead North Africa was and shows very low levels of the haplogroup (these are just two examples: Central Germany, Austria and Italy also contradict the pattern). Fourth, Portugal has rather low levels of diversity for R1b1b2a1.
The mtDNA control
As mentioned, R1b1b2a1 has a marked star-like structure, meaning rapid expansion from a single center. Several European mtDNA lineages also have such structures. The most notable one is H, which is the second largest star-like structure in all the human mtDNA tree, after M, having 34 basal sublineages. Its descendant H1 also has a noticeable star-like structure with 15 basal sublineages. Less impressive but still meaningful are their ancestor HV (6 branches), their cousin V (9 branches), H3 (7 branches), H1b'f'g'k'q (5 branches), H2a (5 branches). In the U haplogroup there are also several with presence in Europe: K1a1 (9), K2a (6) and U5b3 (6). Other haplogroups important in Europe with some star-like structure are: T2 (7) and its descendant T2b (6), as well as I (5).
But only one group of those mtDNA star-like structures seems to be parallel in geography and dimensions to that of R1b1b2a1: H and its descendants.
In my opinion, H must have spread in Europe early on, probably with the colonization of the continent by our species in the early Upper Paleolithic. Only that can explain the the massive star-like structure, the high diversity in Central Europe and the known presence of this lineage in Paleolithic Portuguese and Moroccans. I also get those time frames using a control region mutation count. However there is one significant difference with R1b1b2a1: H is also found in Eastern Europe and Central Asia in large amounts, while R1b1b2a1 or R1b1b2 is not (and only R1b1b1 is in in Central Asia). This situation also happens in North Africa, where SW Europe-derived mtDNA H and V is very common (c. 25%) while Y-DNA R1b is very rare.
One possible explanation could be that there has been Y-DNA sweeps in those areas which did not affect so much to mtDNA. This could be because of Capsian Culture in North Africa (Epipaleolithic and Neolithic - Afroasiatic languages) and because of Kurgan migrations in Eastern Europe (Chalcolithic - Indoeuropean languages).
However it is still a weak point, admittedly.
So are there other options? I have flirted with the expansion of mtDNA K, which is a much recent expansion than that of H. But K only amounts to 6% of all Europeans, while R1b1b2a1 makes up more than 50% in all Western Europe. They are simply not comparable. Same for the other potential candidates mentioned above.
So at the moment I think that the expansion of R1b1b2a1 must have happened within that of mtDNA H, what implies a Paleolithic time frame.
Isn't a Paleolithic time frame too old for R1b1b2a1?
As you probably know, I strongly distrust molecular clock age estimates, favoring instead an holistic logic, inclusive of all genetic and archaeological data available.
The earliest Eurasian ancestors of R1b (F, IJK, K and MNOPS) must have been there in the time of the main Eurasian expansion. F, IJK and K surely coalesced in South Asia, while MNOPS did in SE Asia. Y-DNA P represents the only known back-migration to South Asia of this macro-haplogroup.
Y-DNA R and probably R1 as well surely coalesced in South Asia then but R1b already did in West Eurasia (either in West Asia or Italy). This R1->R1b migration must have happened within the colonization of West Eurasia by H. sapiens, which happened in the 50-40 Ka time frame by all accounts. So I presume that R1b is 50-40 Ka old.
Then R1b split into R1b1a and R1b1b, which must have existed in the Eastern Mediterranean arch (Italy-West Asia).
Then R1b1b split into a minor Central Asian haplogroup (R1b1b1) and the main West Eurasian one (R1b1b2). This one shows a clear origin in Anatolia-SW Caucasus, where it is most diverse, specially once excluded R1b1b2a1 (however notice that Italy and Portugal have similar diversity levels, what again makes me wonder about the exact role of Italy in particular on the spread of this lineage), and where the main R1b1b2a node seems to have coalesced on light of its non-R1b1b2a1 scatter.
And then R1b1b2a1 spread in a very quick expansion, not from Anatolia but from somewhere in West or Central Europe. This might have happened either in the early colonization of Europe (Aurignacian culture, c. 40 Ka ago), in the second wave of Gravettian culture (Cro-Magnon type, c. 30 Ka ago) or in the post-LGM recolonization from the Franco-Cantabrian refuge (Magdalenian culture, c. 15 Ka ago). It is really difficult to determine which wave but, if R1b1b2a1 has to correspond with mtDNA H, then the latter has to be discarded.
And that's what I can say on this matter. Really, without a time machine or accurate aDNA testing, we cannot say much more.