Once labeled the images, we manipulate the images to change the color balance, contrast and add noise to the images.  In the future, images could come from other sources such as street cameras or photos taken by project participants. In conducting these manipulations, our hopes were that our model would be able to handle to find features in the data other than color and position and handle a wider variety of images.  Color balance is simply the change of intensity of the colors in an image.
To change color balance, we drew three random numbers from a Gaussian distribution and multiplied the RGB values by that amount. Color contrast is the difference between colors. We took a random uniform distribution of 5 numbers from .4 to 1 and 5 more from 1 to 1/.4. We then squared all the pixels in the image by each value and normalized it by multiplying by 255 which is the maximum pixel amount and dividing by the maximum pixel value in the image. Noise was added to the images by using an approach called "salt and pepper".  This is when an image has sparsely occurring pixels in the data that is either white or black. Five salt and pepper images, five color balance images, and 10 contrast images where created for each image in our dataset. This gave us a total of 2,280 images. To make sure the images weren't too similar to one another, we removed images that had a similarity score of .95 or higher.  The similarity metric used is known as the structural similarity index (SSIM). SSIM compares the luminance, contrast, and structures of images. This reduced the total number to 1,201.