Unintended consequences of machine learning in medicine?

Machine learning (ML) methods are currently being applied in a wide range of fields. Theoretically, the ability to extract meaningful relations from large datasets holds a great promise for health care and could potentially offer new, unexpected insights into disease and recovery.

However, more critical voices have emerged warning of potential issues surrounding the use of ML and a number of points were addressed in the original paper by Cabitza et al. The authors of the current F1000 paper (McDonald et al) indicate they consider the issues raised as justified but call for a more balanced discussion regarding ML use in medicine. Indeed, Cabitza et al’s view is mainly negative. McDonald and colleagues correctly argue that it is often bad study design that leads to unreliable or unwanted outcomes and not the technique that is used. While a valid point, given how common bad study design or misuse of more common statistical techniques is, it is not unlikely that such issues will also arise from use of ML techniques in health care. Given the far reaching implications of unreliable AI-based decision support systems in health care careful scrutiny of the techniques is necessary.

In their discussion of the reduced sensitivity of radiologists evaluating mammograms, McDonald and colleagues bring to the defense of ML techniques that the model that led to unwanted outcomes was old and that it might work better or worse depending on which type of cancer is being studied. In addition they state that it was not clear how the model was trained and evaluated and that having appropriate feedback loops in place can improve the accuracy of a model considerably. These are obviously important points that should always be taken into account both when evaluating a specific model as well as for the evaluation of the use of ML in medicine in general. The quality of the algorithm but also the quality of available data will ultimately decide how useful a ML approach is for a given medical problem. This only goes to show that ML models should be applied with care and does not negate the original concerns put forward by Cabitza et al.

McDonald and colleagues also propose the use of more transparent ML techniques to prevent potential confounding variables that would not show in a black-box model. However, the predictive power of ML models often increases with their complexity, making transparency either very difficult or even impossible to obtain. As stated by the authors, users of ML models should have a good understanding of the limitations of the techniques. More often than not, ML models will be implemented in a collaboration between technical experts without medical knowledge and medical experts with limited technical expertise. Ensuring an optimal level of transparency of the models combined with the right amount of clinical expertise is therefore vital and while the authors mention this, there are no specific recommendations proposed to actually achieve this. With the increased application of ML models in health care there is a need for guidelines on how to optimally make use of the most technically advanced techniques while making sure bias in the data and confounding variables are accounted for. One recommendation the authors could have made is the need to bring the two worlds (medical and ML-technical) together, bridging the gap by educating people to become professionals with a (bio)medical and technical expertise, as well as training medical doctors in critically working with computerized predictions.

Another point that could have been addressed is how to deal with the differences in experience between medical doctors. Superior performance of the most-experienced doctors could be used to improve the computerized models, while less-experienced doctors may likely gain expertise by reviewing the computerized predictions –provided the ML tool allows for a clear interpretation how it did come to its diagnosis. This property of ML algorithms should be further explored and developed, because it can and, in fact, should lead to further insight in the underlying mechanisms of diseases.