Analysis of Exploration Testing Approach and Conclusions From Implementation in Software Development
Purpose: The main purpose of the research is to examine the suitability of exploratory tests in the software testing process. Design/methodology/approach: An experiment, carried out for the sake of this study, consisted of two parts. First, a test was performed, and in the second part a survey was conducted, which allowed for the comparison of exploratory and test-based tests. Findings: The results of the tests indicated a slightly lower effectiveness of the exploratory approach, which may have been caused by the conditions of the experiment: the choice of the tested software, short duration of test sessions, participants lacking knowledge about the investigated software and experience in performing exploratory tests. Originality/value: Despite the weaker results obtained, the exploratory tests proved useful, as evidenced by the detection of distinctive errors, not found during tests based on test cases. In the survey, 90% of respondents confirmed the use of formalized test approach, based on test cases, while just over a half (57%) indicated having experience in conducting exploratory tests. Testers considered both approaches useful, addressing greater need for conducting formalized tests using test cases. Results included in the research allowed to indicate the qualities and shortcomings of the exploratory approach to software testing. (original abstract)
- 1. Agruss, C., and Johnson, B. (2000). Ad Hoc Software Testing: A perspective on exploration and improvisation. Florida Institute of Technology.
- 2. Bach, J. (2003). Exploratory testing explained. Satisfice, James Bach
- 3. Bures, M., and Filipsky, M., and Jelinek, I. (2018). Identification of Potential Reusable Subroutines in Recorded Automated Test Scripts. International Journal of Software Engineering and Knowledge Engineering. January, pp. 3-36.
- 4. Elmendorf, W. (1967). Evaluation of the Functional Testing of Control Programs. London: IBM.
- 5. Gilb, T. (1977). Software metrics. Winthrop Publishers.
- 6. Guo, S., et al. (2018). Crowdsourced Web Application Testing Under Real-Time Constraints. International Journal of Software Engineering and Knowledge Engineering. June, pp. 751-779.
- 7. Hellmann, T.D., and Maurer, F. (2011). Rule-based exploratory testing of graphical user interfaces. Agile Conference. IEEE.
- 8. Hurley, W.D. (1995). Software engineering and knowledge engineering: trends for the next decade. New York: World Scientific.
- 9. IEEE 730 (2014). Standard for Software Quality Assurance Processes. IEEE.
- 10. IEEE 829 (2008). Standard for Software and System Test Documentation. IEEE.
- 11. ISO/IEC/IEEE 29119 (2013). Software Testing Standard. ISO/IEC/IEEE.
- 12. ISO/IEC 25010 (2011). Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models, IEEE.
- 13. Itkonen, J., and Mantyla, M.V., and Lassenius, C. (2007). Defect detection efficiency: Test case based vs. exploratory testing. First International Symposium on Empirical Software Engineering and Measurement ESEM. IEEE.
- 14. Itkonen, J., and Mantyla, M.V., and Lassenius, C. (2009). How do testers do it? An exploratory study on manual testing practices. 3rd International Symposium on Empirical Software Engineering and Measurement, ESEM 2009. IEEE.
- 15. Itkonen, J., and Rautiainen, K. (2005). Exploratory testing: a multiple case study. International Symposium on Empirical Software Engineering. IEEE.
- 16. Jihyun, L., and Kang, S., and Keum, C. (2018). Architecture-Based Software Testing. International Journal of Software Engineering and Knowledge Engineering. January, pp. 57-77.
- 17. Kaner, C., Falk, J., and Nguyen, H.Q. (2000). Testing Computer Software. Dreamtech Press.
- 18. Leeds, H., and Weinberg, G. (1961). Computer Programming Fundamentals. New York: McGraw-Hill Book Company.
- 19. Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology. New York: The Science Press.
- 20. McCracken, D.D. (1957). Digital computer programming. John Wiley & Sons, Inc.
- 21. Muller, T. et. al. (2005). Foundation Level Syllabus. International Software Testing Qualifications Board (ISTQB) materials. ISTQB.
- 22. Myers, G.J. et al. (2005). Sztuka testowania oprogramowania. Gliwice: Helion.
- 23. Patton, R.J. (2002). Testowanie oprogramowania. Warszawa: Mikom.
- 24. Pfahl, D. et al. (2014). How is exploratory testing used? A state-of-the-practice survey. Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. ACM.
- 25. Royce, W. (1970). Managing the Development of Large Software Systems. IEEE WESCON.
- 26. Shoaib, L., and Aamer, N., and Aisha, A. (2009). An empirical evaluation of the influence of human personality on exploratory software testing. 13th International Multitopic Conference, INMIC 2009. IEEE.
- 27. Tinkham, A., and Cem, K. (2003). Learning styles and exploratory testing. Proceedings of the Pacific Northwest Software Quality Conference.
- 28. Vukovic, V., and Djurkovic, J., and Trninic, J. (2018). A Business Software Testing Process-Based Model Design. International Journal of Software Engineering and Knowledge Engineering. May, pp. 701-749.
- 29. Weichbroth, P. (2018). Delivering Usability in IT Products: Empirical Lessons from the Field. International Journal of Software Engineering and Knowledge Engineering, July, pp.1027-1045.
- 30. Weinberg, G. (1982). Rethinking Systems Analysis and Design. Dorset House Publishing.
- 31. Whittaker, J.A. (2009). Exploratory software testing: tips, tricks, tours, and techniques to guide test design. Pearson Education.
- 32. Xu, H. (2015). Future research directions of software engineering and knowledge engineering. International Journal of Software Engineering and Knowledge Engineering 25.02, pp. 415-421.