Integrated water-energy management is crucial for balancing socioeconomic and environmental objectives in multi-reservoir systems. Multipurpose reservoirs support clean energy production, recreation, navigation, and flood protection but also disrupt natural water flows and fish migration. As hydropower's role evolves with grid decarbonization, managing these tradeoffs becomes increasingly complex. An integrated model combining economic and environmental factors is essential to inform how to adapt hydropower operations effectively to complement decarbonization of the electric grid. However, existing literature lacks such comprehensive models. This study introduces an integrated water-energy optimization model using the Columbia River Basin (CRB) and Mid-Columbia (Mid-C) energy market as a case study. The model couples a simulation of operations of 48 CRB reservoirs with a unit commitment/economic dispatch model of the California and West Coast Power system (CAPOW). We employ Direct Policy Search (DPS) and a multi-objective evolutionary algorithm (MOEA) to optimize four objectives: maximize economic benefits from energy production, minimize fossil fuel electricity generation, minimize environmental flow violations, and minimize peak flood levels. Our findings reveal that the integrated model discovers superior operational strategies compared to existing rules, with some policies outperforming current operations on all objectives simultaneously. Insights from the optimized policies include strategies for improved coordination of reservoir operations using storage and inflow data, and the strategic timing of water releases to ensure increased hydropower production leads to less fossil fuel dependence and greater revenue. These results highlight the potential of integrated models to enhance the sustainability of hydropower operations amid a transitioning energy landscape.