Machine learning for pattern discovery in management research

Prithwiraj Choudhury, Ryan Allen, Michael G. Endres

Supervised machine learning (ML) methods are a powerful toolkit for discovering robust patterns in quantitative data. The patterns identified by ML could be used for exploratory inductive or abductive research, or for post‐hoc analysis of regressionresults to detect patterns that may have gone unnoticed. However, ML models should not be treated as the result of a deductive causal test. To demonstrate the application of ML for pattern discovery, we implement ML algorithms to study employee turnover at a large technology company. We interpret the relationships between variables using partial dependence plots, which uncover surprising nonlinear and interdependent patterns between variables that may have gone unnoticed using traditional methods. To guide readers evaluating ML forpattern discovery, we provide guidance for evaluating model performance, highlighthuman decisions in the process, and warn of common misinterpretation pitfalls. An online appendix provides code and data to implement the algorithms demonstrated in the paper.

Managerial Summary

Supervised machine learning (ML) methods are a powerful toolkit that might help managers and researchers discover interesting patterns in large and complex data. We demonstrate this by using several ML algorithms to investigate the drivers of employee turnover at a large technology company. We evaluate the performance of the models, and use visual tools to interpret the patterns revealed. These patterns can be useful in understanding turnover, but we caution not to confuse correlation with causation. These methods should be viewed as "exploratory" and not conclusive proof of relationships in the data. Our guidance can be helpful for managers evaluating analysis conducted by data scientists in their organizations.

This article is protected by copyright. All rights reserved.

You might also like

A note on knowledge discovery and machine learning in digital soil mapping

Alexandre M.J-C. Wadoux, Alessandro Samuel-Rosa, Laura Poggio, Vera Leatitia Mulder
8 months ago

ML4IoT: A Framework to Orchestrate Machine Learning Workflows on Internet of Things Data

Jose M. Alves, Leonardo M. Honorio, Miriam A. M. Capretz

Never Miss Important Research

Researcher is an app designed by academics, for academics. Create a personalised feed in two minutes.
Choose from over 15,000 academics journals covering ten research areas then let Researcher deliver you papers tailored to your interests each day.

  • Download from Google Play
  • Download from App Store
  • Download from AppInChina

Researcher displays publicly available abstracts and doesn’t host any full article content. If the content is open access, we will direct clicks from the abstracts to the publisher website and display the PDF copy on our platform. Clicks to view the full text will be directed to the publisher website, where only users with subscriptions or access through their institution are able to view the full article.