Wood structural cracking datasets obtained by computer vision techniques can help researchers and managers understand the status of ancient wooden structures. The complex imaging processing will be involved. Herein, the current mainstream technique for computer recognition is Deep Neural Networks (DNNs), which typically requires robust training datasets. However, lacking large, publicly available datasets for ancient wooden structures hampers its applications in this area. In this paper, the Ancient Wooden Building Cracks (AWBC) dataset was developed, consisting of 10,528 images collected from ancient buildings of various ages, types (house, temple, bridges), and configurations (beams, columns, handrails, doorframes, etc.), annotated with four common types of cracks. To fulfill the requirements of both target recognition and instance segmentation tasks in visual inspection, different annotation files were extracted using various annotation programs. The typical DNN detector was introduced and adapted for the established dataset. The results show that the AP can reach 73% for the more obvious cracks, and the dataset can be applied to most of the current mainstream deep learning training programs and be a tool for detecting cracks on ancient wood structures. A large-scale open image dataset for crack detection on wooden structures of ancient buildings associated with the deep learning framework has been provided for the AEC industry.