Protein O-mannosyltransferases (PMTs) are conserved endoplasmic reticulum membrane embedded enzymes responsible for the transfer of mannose from dolichol phosphate-mannose (Dol-P-Man) to serine/threonine-rich protein substrates or unfolded proteins. PMTs from three subfamilies form obligate dimers with different substrate specificities, and require the concerted action of their transmembrane domains (TMDs) and a luminal MIR domain for catalysis. Here, we present structures, native mass spectrometry and structure-based mutagenesis of the Chaetomium thermophilum and Saccharomyces cerevisiae Pmt4 homodimers. The core fold of the TMDs and MIR domain is conserved with the Pmt1-Pmt2 heterodimer, indicating a shared catalytic mechanism. Distinct to Pmt4, the MIR domain interacts in cis with the TMDs of the same subunit and has a beta-hairpin insertion required for O-mannosylation of substrates. We further identify a cytosolic binding site for substrate Dol33 P-Man within the Pmt4 TMDs, which is conserved amongst PMTs and important for in vivo activity. Thus, we provide a framework to understand the substrate specificity and regulation of the Pmt4 homodimer.