Chromosomes, which contain genetic information crucial for the human body, exhibit a thread-like structure within the cell nucleus. Analyzing chromosomes, a process known as karyotyping, is essential for identifying abnormalities within them. Despite the development of various methods for detecting abnormalities, existing approaches often suffer from inefficiencies such as being time-consuming and ineffective in feature extraction. To address these challenges, a novel Visual Geometric Transformer-based Mantis search (VGT-MS) algorithm is proposed that detect abnormal conditions of chromosomes. However, these images often contain extraneous elements that need to be removed. Subsequently, the VGG-16 model is employed to extract features and Vision Transformer is utilized to identify chromosome abnormalities. The parameters are tuned and optimized by Mantis Search Algorithm that validated the performance of the model. The effectiveness of the developed model is evaluated using metrics such as F1-score, accuracy, recall, ROC, and precision. The results reviewed that the proposed achieved superior performance boosting an accuracy, precision, recall and F1-score is 98.0%, 97.2%, 96.2%, 97.6%, all while maintaining lower execution time. Overall, the VGT-MS algorithm presents a robust solution for chromosome abnormality detection, effectively overcoming the limitations of previous approaches and providing enhanced performance metrics.