Inbreeding depression, i.e., the reduction of health and vigour in individuals with high inbreeding coefficients, is expected to increase with environmental, social, or physiological stress. Differences in the strength of sexual selection are therefore predicted to usually lead to higher inbreeding depression in males than in females. However, sex-specific differences in life history may reverse that pattern during certain developmental stages. In salmonids, for example, female juveniles start developing their gonads earlier than males who instead grow faster during that time. We tested whether the sexes are differently affected by inbreeding during that time. To study the effects of inbreeding coefficients that may be typical for natural populations of brown trout (Salmo trutta), and also to control for potentially confounding maternal or paternal effects, we sampled males and females from the wild, used their gametes in a block-wise breeding design to produce 60 full-sib families, released the offspring as yolk-sac larvae into the wild, caught them back 6 months later, identified their genetic sex, and used microsatellites to assign them to their parents. We calculated the average inbreeding coefficient per family based on a panel of >1 million SNPs. Juvenile growth could be predicted from these inbreeding coefficients and the genetic sex: Females grew slower with increasing inbreeding coefficient, while no such link could be found in males. This sex-specific inbreeding depression led to the overall pattern that females grew on average slower than males during the time of gonad formation.