RNA polymerase IV (Pol IV) transcribes transposable elements (TEs) into precursors for small interfering RNAs (siRNAs) that guide DNA methylation to silence these potentially dangerious mobile elements in plants. Pol IV is recruited to TEs via SNF2-like CLASSY (CLSY) proteins. How Pol IV evolved to partner with the CLSYs is unknown. Using phylogenetics, we identified a conserved CYC-YPMF motif that is specific to Pol IV’s largest subunit. Modeling predicts that this motif coordinates a zinc ion and is exposed near Pol IV’s surface in a domain contacting downstream DNA. Via immunoprecipitation and mass spectrometry we found that this motif is essential for the association of all four CLSYs with Pol IV and hereby designate it as a CLSY-docking motif. Consistent with these biochemical results, mutations in the CLSY-docking motif or in neighboring sites of the Pol IV clamp domain, phenocopy pol iv null mutants and display a near-total loss of siRNA biogenesis, resulting in defects in DNA methylation and TE silencing. Together, our findings provide structural and functional insights into a critical protein feature that distinguishes Pol IV from other RNA polymerases, allowing Pol IV to target, transcribe and limit the expression of TE regions, thereby protecting genome integrity.