We demonstrate an automated two-step tumor segmentation method leveraging color information from brightfield images of fresh core needle biopsies of breast tissue. Three different color spaces (HSV, CIELAB, YCbCr) were explored for the segmentation task. By leveraging white-light and green-light images, we identified two different types of color transformations that could separate adipose from benign and tumor or cancerous tissue. We leveraged these two distinct color transformation methods in a two-step process where adipose tissue segmentation was followed by benign tissue segmentation thereby isolating the malignant region of the biopsy. Our tumor segmentation algorithm and imaging probe could highlight suspicious regions on unprocessed biopsy tissue to guide selection of areas most similar to malignant tissues for tissue pathology whether it be formalin fixed or frozen sections, expedite tissue selection for molecular testing, detect positive tumor margins, or serve an alternative to tissue pathology, in countries where these services are lacking.