Machine learning techniques have seen a tremendous rise in popularity in weather and climate sciences. Data assimilation (DA), which combines observations and numerical models, has great potential to incorporate machine learning and artificial intelligence (ML/AI) techniques. In this paper, we use U-Net, a type of convolutional neutral network (CNN), to predict the ensemble covariances in the Ensemble Kalman Filter (EnKF) algorithm. Using a 2-layer quasi-geostrophic model, U-Nets are trained using data from existing EnKF systems. The U-Nets are then used to predict the flow-dependent covariance matrices in U-Net Kalman Filter (UNetKF) experiments, which are compared to traditional 3-dimensional variational (3DVar) and EnKF methods. The performance of UNetKF can match or exceed that of 3DVar, or EnKF with ensemble sizes up to 80. We also demonstrate that trained U-Nets can be transferred to a higher-resolution model for UNetKF, which again performs competitively to 3DVar and EnKF, particularly for small ensemble sizes.