Diversification through crop rotations, the third principle defining Conservation Agriculture (CA), is key to supporting more diverse above-ground arthropod populations. Furthermore, due to the higher demand for crop production, CA farming systems must focus on supporting natural services that can support yield and decrease the demand for external agrochemical inputs. The crop rotation sequence could play a role in supporting different groups of organisms, thereby promoting the sustainability of whole cropping systems. The objectives of this study were to investigate two crop rotation system treatments (i.e., A: Wheat-Wheat-Wheat-Wheat and G: Canola-Medics-Wheat-Medics) as part of a larger long-term trial on a CA farming system located in a wheat producing region of South Africa. Each phase of the crop rotation sequence was represented each year for crop rotation system G. Arthropods were sampled using two sampling methods (i.e., pitfall and sweep-net sampling) 90 days after plant emergence. The results indicated that diversification through crop rotation positively affected the diversity of above-ground arthropods, depending on the specific sampling method used. Arthropods collected with pitfalls revealed less distinct differences between crop rotation sequences. The sweep-net sampling method indicated more crop-specific differences, which could relate to the specific resources each phase provides within the crop rotation sequence. By investigating each arthropod order separately, the results indicated that diversification through crop rotations had a variable effect on the different arthropod groups. Therefore, our study highlights that different arthropod groups must be investigated to draw conclusions regarding diversification within CA farming systems.