Lake has an essential role in Earth’s hydrologic cycle. The changing of lake water areas shows the ecological health of lakes by instant feedback. Remote sensing monitoring reflects the changes in lake water intuitively. This study, based on remote sensing, combined with the hydrological data, meteorological data, and land use survey, investigated the Poyang lake water area changes from 1977 to 2021. Examined the relationship between the lake water area and the impact factor and analyzed the influence of river lake interaction on the lake water area. The results showed that: (1) From 1977 to 2017, the water area of Poyang Lake in flood season (June to September) and dry season (November to Next February) showed a significant downward trend by years. (2) In flood season, runoff, sediment, and precipitation significantly correlate with a water area. In the dry season, sediment significantly correlated with a water area. (3) The Three Gorges Dam water impoundment at the upper reaches of the Yangtze River weakened and eliminated the block and reverse flow of the Yangtze River on Poyang lake, causing the water area to decrease in advance at the end of the flood season after 2005, and decrease rapidly in the dry season after 2003. (4) The change in lakebed elevation caused by sedimentation and human activities also accelerated the shrinkage of the water area. Results demonstrated that the variation of river lake interaction is the primary cause of the Poyang Lake water area’s dramatic change.