Northern peatlands typically develop through succession from fens dominated by the moss family Amblystegiaceae to bogs dominated by the moss genus Sphagnum. How the different plants and abiotic environmental conditions provided in Amblystegiaceae and Sphagnum peat shape the respective moss associated microbial communities is unknown. Through a large-scale molecular and biogeochemical study spanning Arctic, sub-Arctic and temperate regions we assessed how the endo- and epiphytic microbial communities of natural northern peatland mosses relate to peatland type (Sphagnum and Amblystegiaceae), location, moss taxa and abiotic environmental variables. Microbial diversity and community structure were distinctly different between Amblystegiaceae and Sphagnum peatlands, and within each of these two peatland types moss taxon explained the largest part of microbial community variation. Sphagnum and Amblystegiaceae shared few (< 1% of all operational taxonomic units (OTUs)) but strikingly abundant (up to 65% of relative abundance) OTUs. This core community overlapped by one third with the Sphagnum-specific core-community. Thus, the most abundant microorganisms in Sphagnum that are also found in all the Sphagnum plants studied, are the same OTUs as those few shared with Amblystegiaceae. Finally, we could confirm that these highly abundant OTUs were endophytes in Sphagnum, but epiphytes on Amblystegiaceae. We conclude that moss taxa and abiotic environmental variables associate with particular microbial communities. While moss taxon was the most influential parameter, hydrology, pH and temperature also had significant effects on the microbial communities. A small though highly abundant core community is shared between Sphagnum and Amblystegiaceae.