Given that individuals have the right to own property, erect fences, and exclude outsiders from entry, would groups of people also have rights to collectively annex a large territory and exclude others?

It is a common misconception that people have the legal right to own property. The Canadian Charter of Rights and Freedoms, for example, does not include any such "right". Canada is certainly not unique in this regard.

What do exist are laws and traditions that recognize the concept of private property and govern what an individual may or may not be allowed to do with their personal property. In most societies, no individual has the legal right to acquire and own land. There may be few limits on their freedom to acquire property, but that is substantially different from having a legal right to own land. Freedoms and rights are quite different. The rights and freedoms the state - the collective - has in this regard are usually broader than those of individual members of the collective (common good).

As far as a group of persons holding land is concerned, that is the very nature of the modern nation state. The state is a collective of individuals who determine what form and structure their society should take. This is done by establishing certain legal rights and freedoms within a regulatory framework, as well as taking into account traditions. Hence, we have the very different nations of Canada and US, for example, separated by an imaginary line.

With respect to excluding others from property that one owns, there are laws and traditions that allow a property owner to exclude those considered to be trespassers. There are also legal limits to what a property owner can do in this regard - one cannot simply kill or physically attack a person for setting foot on another's property. There are laws that regulate what the individual can or cannot do with regard to trespassers. There are also laws that regulate what the state can do to protect its territory from interlopers (e.g., invading armies) or trespassers (e.g., illegal immigrants).

As far as a group of persons acquiring land and excluding all others from it, there is a word for this process: secession, which often leads to revolution.

Your use of the word annex does not imply that the property has been purchased. It does imply that it has been seized, by whatever means, which would likely entail some form of conflict with the owners of the property, be they individuals or the state.