Poyang Lake is an essential natural wetland in the Yangtze River basin and plays a vital role in maintaining the ecosystem function and ecological security in the middle and lower reaches of the Yangtze River. However, the relative importance and spatial heterogeneity of the impacts of human activities and land use changes on ecological security needs to be further explored. Here, we analyzed the habitat quality level around Poyang Lake in 2022 and explored the factors of habitat quality change from a geographical perspective. The land use structure changes around the Poyang Lake basin from 2000 to 2022 were quantitatively analyzed, and then the relative importance and spatial heterogeneity of each factor on ecological security changes were investigated using geographic probes. The results show that (1) The worst quality habitat (0–0.1) consists mainly of construction land (1624.9 km2) with an area of 1634.64 km2; (2) Construction land continues to increase with the most significant change, and the dynamic land use attitude is 0.47. Grassland and mudflats have the greatest decrease. The increase in cultivated land in different periods is mainly due to the shift of water surface and forest land; (3) Wetland land use change drivers are more influenced by the interaction of socioeconomic factors. The explanatory degrees of the interaction between population density and total year-end population and population density and administrative area are greater than 0.84. The data are greater than the explanatory degrees of every single factor, indicating that the land use change is mainly coupled with population density, total year-end population, and administrative area. These results reveal that human activities influence the degradation of wetlands around the Poyang Lake area. This study has significant reference value for coordinating human–land relationships in Poyang Lake, optimizing land management policy, and improving the sustainable development of cities