Pollination is a key step of plant reproduction, allowing individual plants to produce offspring as father, mother or both. However, few studies exist that consider together male and female pollination success. This implies studying both mating system, through paternity analyses, and seed set, by measuring the percentage of flowers giving a seed. Studying these two processes together is needed as they are not independent: gaining fitness advantage through one sex can incur fitness costs through the other due to various tradeoffs including direct sexual interference. Hence, we developed the first spatially explicit mixed-mating model integrating these two interactive processes, by coupling a mating model with a fruit set model, therefore jointly exploring pollen export and import. We used as model an insect-pollinated tree species, chestnut. We carried out a paternity analysis based on nearly exhaustive sampling of potential pollen donors in an intensively studied plot of 273 trees belonging to three interfertile chestnut species and including both male-fertile and male-sterile individuals. We collected a large dataset of 1924 mating events. We further performed fruit set measurements for 216 trees. Our process-based model predicts fruit set with great accuracy, but only if we account for self-pollen interference and associated ovule discounting, a form of sexual interference. This model represents an important step forward for fundamental pollination studies aiming at comprehensively exploring pollen emission, transport and reception in a single study, thus clarifying the consequences of pollination on male and female fitness.