Agricultural robots offer a viable solution to the critical challenges of productivity and sustainability of modern agriculture. The widespread deployment of agricultural robotic fleets, however, is still hindered by the overall system’s complexity, requiring the integration of several non-trivial components for the operation of each robot but also the orchestration of robots working with each other and human workers. This paper proposes a topological map as the unifying representation and computational model to facilitate the smooth deployment of robotic fleets in agriculture. This topological abstraction of the system state results in an efficient representation of large-scale environments, but also offers the scalable and efficient operation of the entire fleet and allows for ex-situ modelling and analysis of operations. The practical use of the proposed framework is demonstrated in a horticultural use case with a fleet of robots supporting the work of human fruit pickers. The critical components of the system are analysed and evaluated in deployment in both realistic digital twin and real-life soft fruit farms of different scales, demonstrating the scalability and effectiveness of the proposed framework. The presented framework is general and should be easy to adopt in other multi-robot/multi-human scenarios such as warehouse logistics, cleaning and maintenance of public spaces.