Standard convolution is difficult to provide an effective fog feature for visibility estimate tasks due to the fixed grid kernel structure. In this paper, a multiscale deformable convolution model (MDCM) is proposed to extract features that make effectively sampling discriminating features from the atmospheric region in foggy image. Moreover, to enhance performance we use RGB-IR image pair as observations and design a multimodal visibility range classification network based on the MDCM. Experimental results show that both the robustness and accuracy of visibility estimate performance are raised beyond 30% compared to standard convolutional neural networks (CNNs).