Iterative learning control (ILC) is an advantage control strategy widely used in batch systems. Nevertheless, designing an effective iterative learning control scheme is still a critical problem for complex batch processes with non-repetitive nature and model mismatch. In this paper, we propose a two-dimensional iterative learning control-reinforcement learning (2D ILC-RL) control scheme composed of a two-dimensional ILC controller and a two-dimensional DRL compensator. Based on the 2D system theory, the 2D ILC controller is proposed to ensure the primary control performance and its stability and convergence are verified. Meanwhile, the DRL compensator counteracts the negative impact of the model mismatch and the non-repetitive nature. In addition, we proposed a real-time implementation scheme to guarantee the safety of the practical batch systems compared to the conventional online training method. Finally, the simulation results in two chemical batch processes demonstrate the proposed control method’s effectiveness, significant control performance, and strong robustness.