Emerging 5G-Advanced and 6G wireless networks are anticipated to support a wide array of services, including further enhanced mobile broadband (FeMBB) and extreme ultra-reliable low-latency communications (eURLLC), to meet diverse communication needs. The radio access network (RAN) slicing is a pivotal technology for enabling the delivery of these services on shared infrastructure, playing a particularly important role in 6G, where FeMBB and eURLLC services have different blocklengths. To meet varying quality of service (QoS) demands in next-generation networks, innovative multiple access techniques are required to improve interference management and optimize spectrum efficiency. Rate-splitting multiple access (RSMA) is an effective approach for achieving these objectives. This paper investigates the problem of Energy-efficient joint Resource block (RB) allocation and Power control (ERP) for the coexistence of FeMBB and eURLLC services in RSMA-based green communication networks. In this ERP problem, each FeMBB user is guaranteed a minimum data rate, while each eURLLC user must satisfy latency and reliability constraints. To address the ERP problem, we introduce a sub-optimal algorithm (SO-ERP) based on convex optimization. However, the SO-ERP algorithm has high computational complexity and requires approximations to convexify the original ERP problem, potentially moving the solution away from the optimum. To overcome these limitations, we propose a hybrid deep reinforcement learning (HDRL-ERP) algorithm that employs a dueling double deep Qnetwork for RB allocation and a deep deterministic policy gradient for power control. Simulation results are presented to illustrate the performance of the HDRL-ERP algorithm.