This letter proposes a cross-domain WiFi-based gesture recognition system (WiCross) based on a dynamically weighted multi-label generative adversarial network. Most existing WiFi-based gesture recognition systems are user, orientation, and environment sensitive, which limits the application of WiFi sensing. Compared with the influence of users and environments on WiFi sensing systems, the influence of orientation on WiFi sensing systems is more difficult to remove. To alleviate the confusion caused by the orientation more effectively, we arrange the transmitting and receiving antennas according to the characteristics of the Fresnel region. We propose to dynamically weight different links according to users’ orientations and use a multi-label generative adversarial network to obtain domain-independent features. More importantly, WiCross can use domain-independent features to classify some unknown gestures without modifying any code or data set. Lightweight computing resource consumption allows WiCross to respond in real-time. The experimental results show that WiCross can achieve an in-domain recognition accuracy of 93.54% and a cross-domain recognition accuracy of 93.11%