loading page

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self
  • +2
  • Guangming Wang,
  • Chaokang Jiang,
  • Zehang Shen,
  • Yanzi Miao,
  • Hesheng Wang
Guangming Wang
Department of Automation, Shanghai Jiao Tong University
Author Profile
Chaokang Jiang
Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, School of Information and Control Engineering, Advanced Robotics Research Center, China University of Mining and Technology

Corresponding Author:3373363335@qq.com

Author Profile
Zehang Shen
Department of Automation, Shanghai Jiao Tong University
Yanzi Miao
Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, School of Information and Control Engineering, Advanced Robotics Research Center, China University of Mining and Technology
Hesheng Wang
Key Laboratory of System Control and Information Processing of Ministry of Education, Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Engineering Research Center of Intelligent Control and Management, Department of Automation, Shanghai Jiao Tong University

Abstract

3D scene flow presents the 3D motion of each point in the 3D space, which forms the fundamental 3D motion perception for autonomous driving and server robots. Although the RGBD camera or LiDAR capture discrete 3D points in space, the objects and motions usually are continuous in the macro world. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the Generative Adversarial Networks (GAN) is utilized to self-learn 3D scene flow with no need for ground truth. The fake point cloud of the second frame is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on KITTI scene flow dataset show that our method realizes promising results without ground truth. Just like a human observing a real-world scene, the proposed approach is capable of determining the consistency of the scene at different moments in spite of the exact flow value of each point is unknown in advance.
  Corresponding author(s) Email: wanghesheng@sjtu.edu.cn
01 Oct 2021Submitted to AISY Interactive Papers
01 Oct 2021Published in AISY Interactive Papers
Apr 2022Published in Advanced Intelligent Systems volume 4 issue 4 on pages 2100197. 10.1002/aisy.202100197