Optimization of production processes is a critical focus in manufacturing, with machine and deep learning approaches increasingly applied across various domains. This paper introduces a novel application of Deep Reinforcement Learning (DRFL) for optimizing the sequential ordering of products in Surface-Mounted Device production. The approach addresses the overlap of components in assemblies, thereby minimizing changeover times. A Deep Neural Network (DNN) is employed to estimate production times, and a secondary DNN, integrated with DRFL, determines the optimal sequence of product assembly. This setup models the problem as a variant of the Traveling Salesman Problem, where products are represented as nodes in a graph with unique properties. The production time model, derived from both simulation and a Regression Deep Learning Network, feeds into the DRFL agent for optimization. Experimental results demonstrate that DRFL significantly outperforms heuristic methods such as Nearest Neighbor, City Swap, and Simulated Annealing, achieving approximately 5% time savings over heuristic approaches and 30% over manual or random product ordering. Furthermore, DRFL shows superior computational efficiency, making it a viable solution for real-time production environments. This paper underscores the potential of DRFL to enhance production efficiency, offering substantial cost savings in manufacturing operations.