Notes on the Fringes of the Replication Crisis in Management Science

• Autorzy: Przemysław Hensel

Warianty tytułu

Uwagi na marginesie kryzysu replikacji w naukach o zarządzaniu

• Języki publikacji: EN

Abstrakty

EN

The text aims to provide an introduction to the replicability and reproducibility issue in management science while at the same time highlighting possible problems associated with replication research. The study is based on the review of the current literature. Three central issues are highlighted: a) the scarcity of replication attempts is caused by the incentive structure faced by management scholars, b) since the majority of published replications (interstudy and intrastudy) are authored by the researchers who conducted the original study, their results can be affected by the same incentives that affect the results of the original study, c) the popularity of research findings seems to be unaffected by failed attempts to reproduce them. This introductory treatment of the issue suggests that further examination of the relationship between the authorship and replication results is warranted. Increasing the number of replication studies requires a significant change in the incentive structure to which scholars are exposed. (original abstract)

Celem tekstu jest przedstawienie wprowadzenia do problemu replikowalności i reprodukowalności wyników w naukach o zarządzaniu, z jednoczesnym podkreśleniem wyzwań powiązanych z realizacją badań replikacyjnych. Tekst oparto na przeglądzie współczesnej literatury tematu. Wskazano na trzy główne kwestie: a) rzadkość badań replikacyjnych wynika ze struktury zachęt, z którą mają do czynienia badacze zarządzania; b) ponieważ większość opublikowanych badań replikacyjnych została przeprowadzona przez autorów replikowanego badania, ich rezultaty mogą być pochodną tej samej struktury zachęt; c) publikacja badań, w których nie udało się ponownie uzyskać wyników z oryginalnego badania wydaje się nie wpływać na popularność opracowań, których nie udało się powtórzyć. Tekst ma naturę wstępnej analizy zagadnienia, dlatego konieczne są dalsze badania poświęcone temu problemowi. Zwiększenie liczby badań replikacyjnych będzie wymagać zmiany struktury zachęt, na którą wystawieni są naukowcy. (abstrakt oryginalny)

Słowa kluczowe

EN

Management sciences; Research methods

PL

Nauki o zarządzaniu; Metody badawcze

• Czasopismo: Problemy Zarządzania
• Rocznik: 2019
• Tom: 17
• Numer: nr 4 (84) Challenges in Developing Management as an Academic Discipline
• Strony: 52--64

Twórcy

autor

Przemysław Hensel

• Uniwersytet Warszawski

Bibliografia

• Adair, J.G. (1984). The Hawthorne effect: A reconsideration of the methodological artifact. Journal of Applied Psychology, 69(2), 334-345.
• Aguinis, H., & Solarino, A.M. (2019). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal, 0(0).
• Ames, E., & Reiter, S. (1961). Distributions of Correlation Coefficients in Economic Time Series. Journal of the American Statistical Association, 56(295), 637-656.
• Atwater, L.E., Mumford, M.D., Schriesheim, C.A., & Yammarino, F.J. (2014). Editorial: Retraction of leadership articles: Causes and prevention. The Leadership Quarterly, 25(6), 1174-1180.
• Bedeian, A.G. (2007). Even if the Tower Is "Ivory," It Isn't "White:" Understanding the Consequences of Faculty Cynicism. Academy of Management Learning & Education, 6(1), 9-32.
• Bedeian, A.G., Taylor, S.G., & Miller, A.N. (2010). Management Science on the Credibility Bubble: Cardinal Sins and Various Misdemeanors. Academy of Management Learning & Education, 9(1), 715-725.
• Begley, C.G., & Ellis, L.M. (2012). Drug development: Raise standards for preclinical cancer research. Nature, 483(7391), 531.
• Bettis, R.A. (2012). The search for asterisks: Compromised statistical tests and flawed theories. Strategic Management Journal, 33(1), 108-113.
• Bettis, R.A., Ethiraj, S., Gambardella, A., et al. (2016). Creating repeatable cumulative knowledge in strategic management. Strategic Management Journal, 37(2), 257-261.
• Bettis, R.A., Helfat, C.E., & Shaver, J.M. (2016). The necessity, logic, and forms of replication. Strategic Management Journal, 37(11), 2193-2203.
• Boisot, M., & Liang, X.G. (1992). The Nature of Managerial Work in the Chinese Enterprise Reforms. A Study of Six Directors. Organization Studies, 13(2), 161-184.
• Burrell, G., & Morgan, G. (1979). Sociological Paradigms and Organizational Analysis: Elements of the Sociology of Corporate Life. London: Heinemann.
• Camerer, C.F., Dreber, A., Forsell, E., et al. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433-1436.
• Camerer, C.F., Dreber, A., Holzmeister, F., et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637-644.
• Davis, G.F. (2015). Editorial Essay: What Is Organizational Research For?, Administrative Science Quarterly, 60(2), 179-188. https://doi.org/10.1177/0001839215585725.
• Davis, M.S. (1971). That's interesting! Towards a phenomenology of sociology and a􀁢sociology of phenomenology. Philosophy of the Social Sciences, 1(4), 309-344.
• Easley, R.W., Madden, C.S., & Gray, V. (2013). A tale of two cultures: Revisiting journal editors' views of replication research. Journal of Business Research, 66, 1457-1459.
• Ethiraj, S.K., Gambardella, A., & Helfat, C.E. (2017). Improving Data Availability: A New SMJ Initiative. Strategic Management Journal, 38(11), 2145-2146.
• Evanschitzky, H., Baumgarth, C., Hubbard, R., & Armstrong, J.S. (2007). Replication research's disturbing trend. Journal of Business Research, 60, 411-415.
• Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. Plos One, 4(5), 1-11.
• Franke, R.H., & Kaul, J.D. (1978). The Hawthorne Experiments: First Statistical Interpretation. American Sociological Review, (5), 623.
• Frankfort-Nachmias, C., & Nachmias, D. (1996). Research methods in the social sciences. (5 ed.). New York: St. Martin's Press.
• French, J.R.P. (1950). Field experiments: Changing group productivity. In: J.G. Miller (Ed.), Experiments in social process: A symposium on social psychology (pp. 81-96). New York: McGraw-Hill.
• George, G., Haas, M.R., & Pentland, A. (2014). Big Data and Management. Academy of Management Journal, 57(2), 321-326.
• George, G., Osinga, E.C., Lavie, D., & Scott, B.A. (2016). Big Data and Data Science Methods for Management Research. Academy of Management Journal, 59(5), 1493-1507.
• Geuens, M. (2011). Where does business research go from here? Food-for-thought on academic papers in business research. Journal of Business Research, 64(10), 1104-1107.
• Greenhalgh, T., & Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ, 331(7524), 1064-1065.
• Haenlein, M. (2012). From the incoming Editor: Happy Birthday EMJ! European Management Journal, 30(4), 293-294.
• Hendrick, C. (1991). Replications, strict replications, and conceptual replications: Are they important? In: J.W. Neuliep (Ed.), Replication research in the social sciences (pp. 41-49). Thousand Oaks: Sage.
• Hensel, P.G. (2017). Legitymizacja bada􀃱 organizacji. Warszawa: PWN.
• Hensel, P.G. (2019). Supporting replication research in management journals: Qualitative analysis of editorials published between 1970 and 2015. European Management Journal, 37(1), 45-57.
• Herndon, T., Ash, M., & Pollin, R. (2014). Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Cambridge Journal of Economics, 38(2), 257-279.
• Hollenbeck, J.R., & Wright, P.M. (2017). Harking, Sharking, and Tharking: Making the Case for Post Hoc Analysis of Scientific Data. Journal of Management, 43(1), 5-18.
• Honig, B., Lampel, J., Siegel, D., & Drnevich, P. (2014). Ethics in the production and dissemination of management research: Institutional failure or individual fallibility? Journal of Management Studies, 51(1), 118-142.
• Hubbard, R., & Armstrong, J.S. (1994). Replications and extensions in marketing: Rarely published but quite contrary. International Journal of Research in Marketing, 11(3), 233-248.
• Hubbard, R., & Vetter, D.E. (1996). An Empirical Comparison of Published Replication Research in Accounting, Economics, Finance, Management, and Marketing. Journal of Business Research, 35(2), 153-164.
• Hubbard, R., Vetter, D.E., & Little, E.L. (1998). Replication in strategic management: Scientific testing for validity, generalizability, and usefulness. Strategic Management Journal, 19(3), 243-254.
• Hunter, J.E. (2001). The Desperate Need for Replications. Journal of Consumer Research, 28(1), 149-158.
• Izawa, M.R., French, M.D., & Hedge, A. (2011). Shining new light on the hawthorne illumination experiments. Human Factors, 53(5), 528-547.
• Jones, S.R.G. (1992). Was There a Hawthorne Effect? American Journal of Sociology, 98(3), 451-468.
• Karabag, S.F., & Berggren, C. (2016). Misconduct, Marginality and Editorial Practices in Management, Business and Economics Journals. Plos One, 11(7), 1-25.
• Kenworthy, T.P., & Sparks, J.R. (2016). A scientific realism perspective on scientific progress in marketing: An analysis of theory testing in marketing's major journals. European Management Journal, 34(5), 466-474.
• Kerr, N.L. (1998). HARKing: Hypothesizing After the Results are Known. Personality & Social Psychology Review, 2(3), 196-217.
• Kerr, S., Tolliver, J., & Petree, D. (1977). Manuscript Characteristics Which Influence Acceptance for Management and Social Science Journals. Academy of Management Journal, 20(1), 132-141.
• Ketchen Jr, D.J., & Ireland, R.D. (2010). From the Editors: Upon Further Review: A Survey of the Academy of Management Journal's Editorial Board. Academy of Management Journal, 53(2), 208-217.
• Köhler, T., & Cortina, J.M. (forthcoming). Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences. Journal of Management. https://doi. org/10.1177/0149206319843985.
• Leung, K. (2011). Presenting Post Hoc Hypotheses as A Priori: Ethical and Theoretical Issues. Management & Organization Review, 7(3), 471-479.
• Levitt, S., D., & List, J., A. (2011). Was There Really a Hawthorne Effect at the Hawthorne Plant? An Analysis of the Original Illumination Experiments. American Economic Journal: Applied Economics, (1), 224.
• Lykken, D. T. (1968). Statistical Significance in Psychological Research. Psychological Bulletin, 70(3, Pt.1), 151-159.
• Madden, C.S., Easley, R.W., & Dunn, M.G. (1995). How Journal Editors View Replication Research. Journal of Advertising, 24(4), 77-87.
• Martinko, M.J., & Gardner, W.L. (1990). Structured Observation of Managerial Work: A Replication and Synthesis*. Journal of Management Studies, 27(3), 329-357.
• Mayo, E. (1933). The human problems of an industrial civilization. New York: Macmillan. Mezias, S.J., & Regnier, M.O. (2007). Walking the walk as well as talking the talk: replication and the normal science paradigm in strategic management research. Strategic Organization, 5(3), 283-296.
• Neuliep, J.W., & Crandall, R. (1991). Editorial bias against replication research. Journal of Social Behavior and Personality, 5(4), 85-90.
• Neuliep, J.W., & Crandall, R. (1993a). Everyone was wrong: There are lots of replications out there. Journal of Social Behavior and Personality, 8(6), 1-8.
• Neuliep, J.W., & Crandall, R. (1993b). Reviewer bias against replication research. Journal of Social Behavior and Personality, 8(6), 21-29.
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/10.1126/science.aac4716.
• Parsons, H.M. (1974). What Happened at Hawthorne? Science, 183(4128), 922-932.
• Prinz, F., Schlange, T., & Asadullah, K. (2011). Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews. Drug Discovery, 10(9), 712-712.
• Reinhart, C.M., & Rogoff, K.S. (2010). Growth in a Time of Debt. American Economic Review, 100(2), 573-578.
• Roethlisberger, F.J., & Dickson, W.J. (1941). Management and the Worker. Cambridge: Cambridge University Press.
• Rowney, J.A., & Zenisek, T.J. (1980). Manuscript characteristics influencing reviewers' decisions. Canadian Psychology/Psychologie canadienne, 21(1), 17-21.
• Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90-100.
• Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366.
• Snow, C.E. (1927, November). Research on industrial illumination: A discussion of the relation of illumination intensity to productive efficiency. Tech Engineering News.
• Sonnenfeld, J.A. (1985). Shedding light on the Hawthorne studies. Journal of Organizational Behavior, 6(2), 111-130.
• Tsang, E.W.K., & Kwan, K.-M. (1999). Replication and Theory Development in Organizational Science: A Critical Realist Perspective. Academy of Management Review, 24(4), 759-780.
• Uncles, M.D., & Kwok, S. (2013). Designing research with in-built differentiated replication. Journal of Business Research, 66, 1398-1405.
• Van de Ven, A.H. (2015). Welcome to the Academy of Management Discoveries (AMD). Academy of Management Discoveries, 1(1), 1-4.
• Watts, D.J. (2013). Everything is Obvious: Once You Know the Answer. New York: Crown Business.
• Whitehead, T.N. (1938). The industrial worker. Cambridge: Harvard University Press.
• Yuksel, A. (2003). Writing publishable papers. Tourism Management, 24(4), 437-446.

Identyfikatory

DOI

https://doi.org/10.7172/1644-9584.84.4