The novel Traditional Chinese Medicine Ramulus Mori (Sangzhi) alkaloid tablets (SZ-A) are approved by The China National Medical Products Administration for the treatment of type 2 diabetes mellitus (T2DM). However, the extensive pharmacological characteristics and the underlying mechanism are unknown. This study investigated the mechanisms by which SZ-A ameliorates glucose metabolism in KKAy mice, an animal model of T2DM. Diabetic KKAy mice were treated intragastrically with SZ-A once daily for 8 weeks, after which glucose levels, lipid metabolism, gut microbiome, systemic inflammatory factors, luminal concentrations of short-chain fatty acids (fecal samples), and ileal proteomic changes were evaluated. The ileum tissues were collected, and the effects of SZ-A on pathological inflammatory damage were evaluated by hematoxylin and eosin staining, immunofluorescence, and immunohistochemistry. The mRNA and protein expression levels of various inflammatory markers, including monocyte chemoattractant protein-1 and phosphorylated nuclear factor kappa B p65, were detected in the ileum tissues. SZ-A improved glucose metabolism with enhanced insulin response and elevated glucagon-like peptide 1 (GLP-1) during the glucose tolerance test in diabetic KKAy mice. Gut microbiota analysis demonstrated that SZ-A administration elevated the abundance of Bacteroidaceae and Verrucomicrobia, reduced the levels of Rikenellaceae and Desulfovibrionaceae; and increased the concentrations of fecal acetic and propionic acids compared to the diabetic model group. Additionally, SZ-A markedly improved ileal inflammatory injury and pro-inflammatory macrophage infiltration and improved intestinal mucosal barrier function in diabetic KKAy mice. SZ-A also attenuated the levels of circulating endotoxin, pro-inflammatory cytokines, and chemokines in the mice sera. Collectively, SZ-A ameliorated the overall metabolic profile including glucose and lipid metabolism in KKAy mice, which may be associated with an improvement in GLP-1 and insulin secretion, at least in part by modulating the gut microbiome and relieving the degree of ileal and systemic inflammation.