Multi-center heterogeneous data is a hot issue in federated learning nowadays. The data of clients and centers do not follow the normal distribution, which brings great challenges to learning. Based on the assumption that the client data with multivariate skewed normal distribution, we improve the DP-Fed-mv-PPCA model. We use a Bayesian framework to construct prior distributions of local parameters, and use Expectation Maximization (EM) algorithm and pseudo-Newton algorithm to obtain robust estimates of parameters. Then, the clipping algorithm and Differential Privacy (DP) algorithm are used to solve the problem that the model parameters do not have the display solution and achieve the privacy guarantee.