In order to count soybean seeds quickly and accurately, improve the speed of seed test and the level of soybean breeding, a method of soybean seed counting based on VGG-Two (VGG-T) was developed in this research. Firstly, in view of the lack of available image dataset in the field of soybean seed counting, a fast target point labeling method of combining pre-annotation based on digital image processing technology with manual correction annotation was proposed to speed up the establishment of publicly available soybean seed image dataset with annotation. Only 197 min were taken to mark 37,563 seeds when using this method, which saved 1592 min than ordinary manual marking and could reduce 96% of manual workload. At the same time, the dataset in this research is the largest annotated data set for soybean seed counting so far. Secondly, a method that combined the density estimation-based and the convolution neural network (CNN) was developed to accurately estimate the seed count from an individual threshed seed image with a single perspective. Thereinto, a CNN architecture consisting of two columns of the same network structure was used to learn the mapping from the original pixel to the density map. Due to the very limited number of training samples and the effect of vanishing gradients on deep neural networks, it is not easy for the network to learn all parameters at the same time. Inspired by the success of pre-training, this research pre-trained the CNN in each column by directly mapping the output of the fourth convolutional layer to the density map. Then these pre-trained CNNs were used to initialize CNNs in these two columns and fine-tune all parameters. Finally, the model was tested, and the effectiveness of the algorithm through three comparative experiments (with and without data enhancement, VGG16 and VGG-T, multiple sets of test set) was verified, which respectively provided 0.6 and 0.2 mean absolute error (MAE) in the original image and patch cases, while mean squared error (MSE) were 0.6 and 0.3. Compared with traditional image morphology operations, ResNet18, ResNet18-T and VGG16, the method proposed improving the accuracy of soybean seed counting. In the testset containing soybean seeds of different densities, the error fluctuation was small, and it still had excellent counting performance. At the same time, compared with manual counting and photoelectric seed counter, it saved about 2.493 h and 0.203 h respectively for counting 11,350 soybean seeds, realizing rapid soybean seeds counting.