In the textile industry, manual fabric inspection is challenging and can lead to errors that affect product cost and quality. Deep learning has introduced effective machine learning algorithms for image classification and analysis, but issues like complex training, large data requirements, and poor generalization remain. There is a need for a fast, accurate automatic algorithm for real-time industrial use. This research developed a simple Convolutional Neural Network (CNN) to address these challenges effectively. The algorithm’s performance was evaluated using two image sizes: 150 x 700 and 245 x 345. It was evident that image size significantly affects the model’s performance. The dataset’s imbalance negatively impacted the model due to insufficient training and overfitting. To improve this, various sampling techniques were tested. The CNN model performed best with smaller images (245 x 345) and the SMOTEENN sampling technique. The model achieved impressive accuracy, precision, recall, and F1 scores of 98.00%, with modeling and prediction times of 1.57 seconds and 0.09 seconds, respectively. A method to deploy the algorithm for automating textile quality inspection was also proposed.