Can you beat logistic regression? The performance of four binary classifiers predicting premature birth

Abstract

This paper presents a comparison of four predictive models for classification tasks: logistic regression, support vector machines (SVM), Gaussian process (GP) classification, and a model based on partial exchangeability. The models were evaluated using a dataset with 7 binary covariates, where 70% of the sample was used for training and the remaining 30% for testing. Predictive performance was assessed through three metrics: the area under the receiver operating characteristic curve (AUC), Brier score, and logarithmic score. Results show that, while the SVM model performed distinctly, the other three models exhibited similar performance, with logistic regression demonstrating the best overall results, though not by a large margin. Theoretical insights from de Finetti’s representation theorem for partially exchangeable data suggest that logistic regression can be seen as a specific case of a more general model, which captures the influence of covariates on the outcome probabilities. Furthermore, we explored the Bayesian counterpart of logistic regression, using a flat prior on the model coefficients, and found that its predictive performance was almost identical to the frequentist approach. Although logistic regression was the most effective model for this dataset, the paper discusses the advantages of more flexible models, like SVM and GP classification, which may be better suited for capturing complex nonlinear relationships in data. The results indicate that while logistic regression is a reliable, fast, and interpretable model, alternative models may outperform it in cases where complex interactions between covariates and the target variable exist.