Aiming at the problem of huge energy consumption in the Fog Wireless Access Networks (F-RANs), the resource allocation scheme of the F-RAN architecture under the cooperation of renewable energy is studied in this paper. Firstly, the transmission model and Energy Harvesting (EH) model are established, the solar energy harvester is installed on each Fog Access Point (F-AP), and each F-AP is connected to the smart grid. Secondly, the optimization problem is established according to the constraints of Signal to Noise Ratio (SNR), available bandwidth and energy harvesting, so as to maximize the average throughput of F-RAN architecture with hybrid energy sources. Finally, the dynamic power allocation scheme in the network is studied by using Q-learning and Deep Q Network (DQN) respectively. Simulation results show that the proposed two algorithms can improve the average throughput of the whole network compared with other traditional algorithms.