Endoscopic renal surgeries have high re-operation rates, particularly for lower volume surgeons. Due to the limited field and depth of view of current endoscopes, mentally mapping preoperative computed tomography (CT) images of patient anatomy to the surgical field is challenging. The inability to completely navigate the intrarenal collecting system leads to missed kidney stones and tumors, subsequently raising recurrence rates. We propose a guidance system to estimate the endoscope positions within the CT to reduce re-operation rates. We use a Structure from Motion algorithm to reconstruct the kidney collecting system from the endoscope videos. In addition, we segment the kidney collecting system from CT scans using 3D U-Net to create a 3D model. We can then register the two collecting system representations to provide information on the relative endoscope position. We demonstrate correct reconstruction and localization of intrarenal anatomy and endoscope position. Furthermore, we create a 3D map supported by the RGB endoscope images to reduce the burden of mental mapping during surgery. The proposed reconstruction pipeline has been validated for guidance. It can reduce the mental burden for surgeons and is a step towards our long-term goal of reducing re-operation rates in kidney stone surgery.