Four supervised machine learning algorithms were used in building social engineering attack prevention models i.e. Support Vector machines, Logistic Regression, Bayesian Network and Random Forest. The efficacy of the models was ascertained by the help of three datasets (one synthetic and two real-life). The performances of the machine learning methods were compared using F1-score, accuracy, Matthew’s Correlation Coefficient, Precision and Sensitivity. The usefulness of each dataset was also compared per machine learning technique to draw relevant conclusions. Results from the experimental set-up showed that SVM demonstrates greater resilience in preventing mobile money fraud attacks as compared to the other algorithms. Again, even though real datasets are complex and utilize greater system resources during model training and testing, they help in building more effective social engineering attack prevention models than other datasets that are obtained from synthetic sources.