Nonsense-mediated RNA decay (NMD) is a highly conserved and selective RNA turnover pathway that depends on the endonuclease SMG6. Here, we show that SMG6 is essential for male germ cell differentiation in mice. Germ-cell conditional knockout (cKO) of Smg6 induces extensive transcriptome misregulation, including a failure to eliminate meiotically expressed transcripts in early haploid cells, and accumulation of NMD target mRNAs with long 3’ untranslated regions (UTRs). Loss of SMG6 in the male germline results in complete arrest of spermatogenesis at the early haploid cell stage. We find that SMG6 is strikingly enriched in the chromatoid body (CB), a specialized cytoplasmic granule in male germ cells also harboring PIWI-interacting RNAs (piRNAs) and the piRNA-binding protein PIWIL1.