Recent research has indicated that the relationship between age-related cognitive decline and falling may be mediated by the individual’s capacity to quickly cancel or inhibit a motor response. This longitudinal investigation demonstrates that higher white matter fibre density in the motor inhibition network paired with low physical activity was associated with falling in elderly participants. We measured the density of white matter fibre tracts connecting key nodes in the inhibitory control network in a large sample (n=414) of older adults. We modelled their self-reported frequency of falling over a four year period with white matter fibre density in pathways corresponding to the direct and hyperdirect cortical-subcortical loops implicated in the inhibitory control network. Only connectivity between right Inferior Frontal Gyrus and right Subthalamic Nucleus was associated with falling as measured cross-sectionally. The connectivity was not, however, predictive of future falling when measured two and four years later. Higher white matter fibre density was associated with falling, but only in combination with low levels of physical activity. No such relationship existed for selected control brain regions that are not implicated in the inhibitory control network. The direction of this effect was counterintuitive and warrants further longitudinal investigation into whether white matter fibre density changes over time in a manner correlated with falling, and mediated by physical activity.