Advanced prostate cancer initially responds to hormonal treatment, but ultimately becomes resistant and requires more potent therapies. One mechanism of resistance seen in 10% of these patients is through lineage plasticity, which manifests in a partial or complete small cell or neuroendocrine prostate cancer (NEPC) phenotype. Here, we investigate the role of the mammalian SWI/SNF chromatin remodeling complex in NEPC. Using large patient datasets, patient-derived organoids and cancer cell lines, we identify SWI/SNF subunits that are deregulated in NEPC, demonstrate that SMARCA4 (BRG1) overexpression is associated with aggressive disease and that SMARCA4 depletion impairs prostate cancer cell growth. We also show that SWI/SNF complexes interact with different lineage-specific factors in prostate adenocarcinoma and in NEPC cells, and that induction of lineage plasticity through depletion of REST is accompanied by changes in SWI/SNF genome occupancy. These data suggest a specific role for mSWI/SNF complexes in therapy-related lineage plasticity, which may be relevant for other solid tumors.