Selective catalysis, particularly when differentiating substrates with similar reactivities in mixture, is a significant challenge. In this study, anomaly detection algorithms---tools traditionally used for identifying outliers in data cleaning---are applied to catalyst screening. We focus on developing catalytic methods to selectively oxidize cyclic alkanes over linear alkanes in mixtures such as naphtha. By inserting cyclohexane oxidation data one by one into a database of n-hexane oxidization, we used several anomaly detection algorithms to evaluate whether the inserted cyclohexane oxidation data could be considered anomalous. Conditions identified as anomalies imply that they are likely not suitable for n-hexane oxidization. However, these anomalies come from conditions for cyclohexane oxidation. As a result, they are promising conditions for selective oxidation of cyclohexane while leaving n-hexane unaltered. These anomalies were thus further investigated, leading to the discovery of a specific catalytic approach that selectively oxidizes cyclohexane. This application of anomaly detection offers a novel method to search for selective catalyst for chemical reactions involving mixed substrates.