In the Industrial Internet of Things-Digital Twin (IIoT-DT), real-time mapping and collaborative modeling between physical entities and twins are achieved. The high concurrent transmission of massive sensor data from industrial electronic equipment leads to communication delays. Meanwhile, multimodal data has the characteristic of non-independent and identically distributed (non-IID). There are difficult to integrate. This letter proposes DT-SL framework in IIoT, which uses fully decentralized swarm learning (SL) to improve the efficiency of updating local twin models. We designed weighted multi-matrix singular value decomposition (SVD) to improve the consistency and efficiency of multimodal data fusion in the framework. Experimental results show that the proposed method outperforms the baseline method in model communication overhead, convergence rate and performance.