Ultra-short-term power load forecasting is beneficial to improve the economic efficiency of power systems and ensure the safe and stable operation of power grids. As the volatility and randomness of loads in power systems, make it difficult to achieve accurate and reliable power load forecasting, a sequence-to-sequence based learning framework is proposed to learn feature information in different dimensions synchronously. A CNN_BiLSTM network is built in the encoder to extract the correlated timing features embedded in external factors affecting power loads. The parallel BiLSTM network is built in the decoder to mine the power load timing information in different regions separately. The multi-headed attention mechanism is introduced to fuse the BiLSTM hidden layer state information in different components to further highlight the key information representation. The load forecastion results in different regions are output through the fully connected layer. The model proposed in this paper has the advantage of high forecastion accuracy through the example analysis of real power load data.