It has been proposed that conservation laws might not be beneficial for accurate hydrological modeling due to errors in input (precipitation) and target (streamflow) data (particularly at the event time scale), and this might explain why deep learning models (which are not based on enforcing closure) can out-perform catchment-scale conceptual and process-based models at predicting streamflow. We test this hypothesis at the event and multi-year time scale using physics-informed (mass conserving) machine learning and find that: (1) enforcing closure in the rainfall-runoff mass balance does appear to harm the overall skill of hydrological models, (2) deep learning models learn to account for spatiotemporally variable biases in data (3) however this “closure” effect accounts for only a small fraction of the difference in predictive skill between deep learning and conceptual models.