Background and Objectives: The role of neoadjuvant chemotherapy (NAC) in treating initially resectable colorectal cancer liver metastases (CRCLM) remains controversial. Methods: We retrospectively compared outcomes between patients who underwent upfront surgery and those who received NAC followed by curative CRCLM surgery. Factors associated with early relapse were also investigated. Fifty-three patients were included (upfront surgery: n = 31, NAC: n = 22). Results: The 3-year overall survival (OS) and relapse-free survival (RFS) rates were comparable between the upfront surgery and NAC groups (OS: 83.9% vs. 81.8%, P = 0.9, hazard ratio [HR] = 1.07; RFS: 58.1% vs. 36.4%, P = 0.1, HR = 1.87). However, among metachronous CRCLM cohorts, the 3-year OS rates were 88.2% and 90.0% (P = 0.9, HR = 0.85), and the 3-year RFS rates were 76.4% and 30.0% (P = 0.015, HR = 4.03), respectively. Patients with higher computed tomography values in CRCLM had significantly better RFS than those with lower values (P = 0.04). Conversely, a higher number of CRCLM after NAC was significantly correlated with shorter RFS (P = 0.007). Conclusions: Routine NAC for initially resectable metachronous CRCLM should be avoided because of the high risk of early relapse.