Iterative learning control (ILC) offers an effective learning control scheme to solve the control problems of the batch processes. Although the control performances of ILC systems can be improved batch-by-batch, the convergence still strongly depends on the repeatability of the process and thus lack of robustness. Meanwhile, the data-driven-based deep reinforcement learning (DRL) algorithms have good robustness due to the generalization of the neural network, but it has lower data efficiency in training. In this paper, we propose a complementary control scheme for the batch processes by employing a DRL guided by a classical ILC, termed as the IL-RLC scheme. This scheme has higher data efficiency than the DRL without guidance and better robustness than the ILC, which are demonstrated by the numerical simulations on a linear batch process and a nonlinear batch reactor. This work provides a way for the application of DRL algorithm in the batch process control.