Fifteen percent of breast cancers are invasive lobular carcinomas (ILCs), a specific histological subtype. Although ILCs have distinct clinicopathological features and pathognomonic CDH1 loss, no specific therapies exist because ILCs are underrepresented in clinical trials and preclinical models are lacking. We developed intraductal ILC xenograft models to test the hypothesis that the pan-LOX inhibitor (LOXi), PXS-5505, used in patients with myelofibrosis can be repurposed to target ILC-specific matrix dependency. LOXi blocks ILC progression in clinically relevant settings and alters collagen fiber organisation and cell matrix composition. In vitro screens identify ITGAV and ITGB5 loss as synthetic lethal in cells with CDH1 loss. Consistent with ILC specific dependency on ITG-mediated matrix interactions, expression of downstream factors CCN1, MYC, NFKB and AP-1 are increased in ILCs and downregulated by LOXi. We identify collagen fibre density and alignment, and MYC/AP-1 signatures as translatable endpoints of LOXi activity enabling needed clinical trials on ILCs.