Through the comprehensive analysis of ab initio and experimental results of a large number of diatomic systems, the systematic deviation of ab initio method in vibrational energies prediction caused by physical/mathematical simplification is located. A joint ab initio and machine learning method based on information across molecules is proposed to deal with the problem. Starting from an ab initio model, and then systematically modifying it through machine learning, the vibrational energies prediction of many diatomic systems (SiC, HBr, NO, PC, N2, SiO, O2, ClF, etc.) have been improved, and significantly surpassed the more complex ab initio model. In addition to the improvement of accuracy, the new method also greatly reduces the computational expense, and is applicable for the systems without experimental data.