The distributed and privacy-preserving characteristics of fine-grained smart grid data hinder data sharing, making federated learning an attractive approach for collaborative training among data owners with similar load patterns. However, malicious models can interfere with training in the federated learning aggregation process, making it difficult to ensure the accuracy and safety of the central model in load forecasting. Therefore, we propose a secure aggregation federated learning method for distributed load forecasting based on similarity and distance (Fed-SAD), which effectively eliminates the interference of malicious models by securely aggregating models, thereby ensuring accurate and safe distributed scenario prediction. Experimental results demonstrate that Fed-SAD maintains high accuracy and robustness in both the presence and absence of malicious models, while maintaining data and model security.