Conservation agriculture (CA) systems represent a set of three soil management principles that include minimal soil disturbance, permanent soil cover and crop rotations whereas the CA-based systems in this study add the bed and furrow tillage structures as integral elements of CA. This study aimed at investigating the long-term (2005-2013) influence of CA-based systems on soil health and crop productivity in northern Ethiopia. The treatments include two types of CA-based systems (permanent raised bed PRB and contour furrowing CF) and conventional tillage (CT). The experimental layout was arranged in a randomized complete block design. Soil samples were collected at 0-10 cm soil depth to assess soil health. Wheat root samples were used to measure arbuscular mycorrhizal fungi (AMF) colonization percentage using grid line intersect method. Piecewise structural equation modeling (PSEM) was used to understand linkages between management practices, soil health and crop productivity. Higher soil microbial biomass carbon (SMBC), AMF spore abundance and root colonization were recorded in PRB followed by CF as compared to CT (P < 0.05). Carbon sequestration rate, nutrient availability, plant available water capacity and air capacity were significantly higher in PRB and CF compared to CT. Outputs of the PSEM highlighted two pathways in which CA-based systems contributed to improved productivity: (1) via higher density of bacteria and improved hydraulic conductivity, and (2) via higher density of fungi and increase soil organic carbon content in the topsoil. The study concludes that CA-based systems have the potential to improve crop productivity through improved soil health.