Animal social structure is influenced by demographic processes such as survival and recruitment. Yet, how the gain and loss of social associations among individuals scale up through levels of social organization is poorly understood. To address this gap, we conducted multiscale analysis of long-term social network dynamics over twelve winters for a migrant passerine, the golden-crowned sparrow. We found consistently high levels of stability in pairwise associations among returning sparrows, but this social fidelity was not the strongest driver of higher-level properties such as community cohesion and network modularity. Instead, these were increased more by the clustered social associations of newly immigrating individuals. Thus, the interplay of distinct processes---social fidelity among older birds and social cohesion among newer birds---jointly drive social structures at different levels of social organization. Our results demonstrate the different effects of demographic changes on social structure across pairwise, community, and population levels.