As the complexity and scale of software applications increase, the challenges associated with testing these systems grow correspondingly, necessitating innovative and sustainable testing strategies. This paper explores a multifaceted approach aimed at addressing the intricate challenges inherent in testing large-scale software applications. Through a comprehensive examination of current industry practices and emerging trends, this study introduces a novel framework that integrates advanced testing techniques with state-of-the-art tools. This framework not only mitigates the challenges posed by the complexity and size of modern applications but also enhances the efficiency and effectiveness of the testing process. Key aspects of this research include a detailed exploration of test methodologies suited for large-scale applications, an evaluation of advanced tools designed for complex test scenarios, and an analysis of the impact of the test environment on sustainability. The findings offer valuable insights and actionable strategies for software development and testing professionals aiming to optimize testing processes and improve the quality and sustainability of their software in a rapidly evolving technological landscape.

Agarwal, A., Gupta, S., & Choudhury, T. (2018). Continuous and Integrated Software Development using DevOps. In: Proceedings of the 2018 International Conference on Advances in Computing and Communication Engineering (ICACCE). https://doi.org/10.1109/icacce.2018.8458052

Aggarwal, P. K., Grover, P. S., & Ahuja, L. (2018). Incorporating Autonomic Capability as Quality Attribute for a Software System. In: Proceedings of the 2018 7th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO). https://doi.org/10.1109/icrito.2018.8748488

Ahlgren, J., Berezin, M., Bojarczuk, K., et al. (2021). Testing Web Enabled Simulation at Scale Using Metamorphic Testing. In: Proceedings of the 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). https://doi.org/10.1109/icse-seip52600.2021.00023

Al Alamin, M. A., & Uddin, G. (2021). Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phases. In: Proceedings of the 2021 IEEE International Conference on Autonomous Systems (ICAS). https://doi.org/10.1109/icas49788.2021.9551151

Alferidah, S. K., & Ahmed, S. (2020). Automated Software Testing Tools. In: Proceedings of the 2020 International Conference on Computing and Information Technology (ICCIT-1441); 09–10 September 2020; Tabuk, Saudi Arabia. pp. 1–4. https://doi.org/10.1109/ICCIT-144147971.2020.9213735

Ali, S., & Li, H. (2019). Moving Software Testing to the Cloud: An Adoption Assessment Model Based on Fuzzy Multi-Attribute Decision Making Algorithm. In: Proceedings of the 2019 IEEE 6th International Conference on Industrial Engineering and Applications (ICIEA). https://doi.org/10.1109/iea.2019.8714986

Ali, S., Sun, H., Zhao, Y., et al. (2019). Testing-based Model Learning Approach for Legacy Components. In: Proceedings of the 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST). https://doi.org/10.1109/ibcast.2019.8667149

Alshahwan, N., Harman, M., & Marginean, A. (2023). Software Testing Research Challenges: An Industrial Perspective. In: Proceedings of the 2023 IEEE Conference on Software Testing, Verification and Validation (ICST). https://doi.org/10.1109/icst57152.2023.00008

Barroca, L., Sharp, H., Salah, D., et al. (2015). Bridging the gap between research and agile practice: an evolutionary model. International Journal of System Assurance Engineering and Management, 9(2), 323–334. https://doi.org/10.1007/s13198-015-0355-5

Braiek, H. B., & Khomh, F. (2020). On testing machine learning programs. Journal of Systems and Software, 164, 110542. https://doi.org/10.1016/j.jss.2020.110542

Dirim, S., & Sozer, H. (2020). Prioritization of Test Cases with Varying Test Costs and Fault Severities for Certification Testing. In: Proceedings of the 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). https://doi.org/10.1109/icstw50294.2020.00069

Gao, C., Luo, W., & Xie, F. (2022). An Ontology-based Knowledge Base System for Military Software Testing. In: Proceedings of the 2022 9th International Conference on Dependable Systems and Their Applications (DSA). https://doi.org/10.1109/dsa56465.2022.00043

Ge, X., Yu, S., Fang, C., et al. (2023). Leveraging Android Automated Testing to Assist Crowdsourced Testing. IEEE Transactions on Software Engineering, 49(4), 2318–2336. https://doi.org/10.1109/tse.2022.3216879

Gong, J., & Cai, L. (2023). Analysis for Microservice Architecture Application Quality Model and Testing Method. In: Proceedings of the 2023 26th ACIS International Winter Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Winter). https://doi.org/10.1109/snpd-winter57765.2023.10223960

Guo, X., Okamura, H., & Dohi, T. (2022). Automated Software Test Data Generation with Generative Adversarial Networks. IEEE Access, 10, 20690–20700. https://doi.org/10.1109/access.2022.3153347

Hu, X., Liu, Z., Xia, X., et al. (2023). Identify and Update Test Cases When Production Code Changes: A Transformer-Based Approach. In: Proceedings of the 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). https://doi.org/10.1109/ase56229.2023.00165

Hurdugaci, V., & Zaidman, A. (2012). Aiding Software Developers to Maintain Developer Tests. In: Proceedings of the 2012 16th European Conference on Software Maintenance and Reengineering. https://doi.org/10.1109/csmr.2012.12

Hynninen, T., Kasurinen, J., Knutas, A., et al. (2018). Software testing: Survey of the industry practices. In: Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). https://doi.org/10.23919/mipro.2018.8400261

Jarman, D., Smith, R., Gosney, G., et al. (2019). Applying Combinatorial Testing to Large-Scale Data Processing at Adobe. In: Proceedings of the 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). https://doi.org/10.1109/icstw.2019.00051

Jharko, E. (2021). Some Aspects of Quality Assurance in the Development of Digital Systems. In: Proceedings of the 2021 14th International Conference Management of Large-Scale System Development (MLSD). https://doi.org/10.1109/mlsd52249.2021.9600164

Köroğlu, Y., Sen, A., & Akin, A. (2020). Automated Functional Test Generation Practice for a Large-Scale Android Application. In: Proceedings of the 2020 Turkish National Software Engineering Symposium (UYMS). https://doi.org/10.1109/uyms50627.2020.9247060

Lei, Q., Liao, W., Jiang, Y., et al. (2019). Performance and Scalability Testing Strategy Based on Kubemark. In: Proceedings of the 2019 IEEE 4th International Conference on Cloud Computing and Big Data Analysis (ICCCBDA). https://doi.org/10.1109/icccbda.2019.8725658

Li, H., Chen T. H., Hassan A. E., et al. (2018). Flora. Adopting Autonomic Computing Capabilities in Existing Large-Scale Systems. Available online: https://petertsehsun.github.io/papers/peter_icse_seip2018.pdf (accessed on 15 February 2024).

Li, H., Chen, P., Liu, Y., et al. (2022). Design and application of an automated test platform based on image recognition. In: Proceedings of the 2022 IEEE International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC); 5–7 August 2022; Chongqing, China. pp. 363–366. https://doi.org/10.1109/SDPC55702.2022.9915914

Lisitsyn, A. B., Nikitina, M. A., & Chernukha, I. M. (2021). Product Quality and Safety Management in Large-Scale Systems. In: Proceedings of the 2021 14th International Conference Management of Large-Scale System Development (MLSD). https://doi.org/10.1109/mlsd52249.2021.9600217

Liu, P., Li, Y., Zeng, L., et al. (2021). Big Data-based Testing: Characteristics, Challenges, and Future Directions. In: Proceedings of the 2021 7th International Symposium on System and Software Reliability (ISSSR). https://doi.org/10.1109/isssr53171.2021.00012

Meng, S., Dai, Y., Luo, L., et al. (2019). Reliability Modeling and Optimization for Large-Scale Network Systems. In: Proceedings of the 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C). https://doi.org/10.1109/qrs-c.2019.00108

Mezhuyev, V., Al-Emran, M., Ismail, M. A., et al. (2019). The Acceptance of Search-Based Software Engineering Techniques: An Empirical Evaluation Using the Technology Acceptance Model. IEEE Access, 7, 101073–101085. https://doi.org/10.1109/access.2019.2917913

Mulla, N., & Jayakumar, N. (2021). Role of Machine Learning & Artificial Intelligence Techniques in Software Testing. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(6), 2913–2921. https://doi.org/10.17762/turcomat.v12i6.5800

Tsai, C. H., Tsai, S. C., & Huang, S. K. (2021). REST API Fuzzing by Coverage Level Guided Blackbox Testing. In: Proceedings of the 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS). pp. 291–300. https://doi.org/10.48550/arXiv.2112.15485

Uygun, Y., Oguz, R. F., Olmezogullari, E., et al. (2020). On the Large-scale Graph Data Processing for User Interface Testing in Big Data Science Projects. In: Proceedings of the 2020 IEEE International Conference on Big Data (Big Data). https://doi.org/10.1109/bigdata50022.2020.9378153

Valle-Gomez, K. J., Delgado-Perez, P., Medina-Bulo, I., et al. (2019). Software Testing: Cost Reduction in Industry 4.0. In: Proceedings of the 2019 IEEE/ACM 14th International Workshop on Automation of Software Test (AST). https://doi.org/10.1109/ast.2019.00018

Wang, J., & Ren, D. (2018). Research on Software Testing Technology Under the Background of Big Data. In: Proceedings of the 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). https://doi.org/10.1109/imcec.2018.8469275

Wang, J., & Wu, J. (2019). Research on Performance Automation Testing Technology Based on JMeter. In: Proceedings of the 2019 International Conference on Robots & Intelligent System (ICRIS). https://doi.org/10.1109/icris.2019.00023

Wu, H., Yu, S., Niu, X., et al. (2023). Enhancing Fault Injection Testing of Service Systems via Fault-Tolerance Bottleneck. IEEE Transactions on Software Engineering, 1–17. https://doi.org/10.1109/tse.2023.3285357

Xu, X., He, H., Song, W., et al. (2021). Analysis on the Quality Model of Big Data Software. In: Proceedings of the 2021 IEEE/ACIS 19th International Conference on Computer and Information Science (ICIS); 23 August 2021; Shanghai, China. pp. 78–81.

Yin, L. (2020). Test Suite Generation for Software Reliability Testing Based on Hybrid Musa and Markov Method. In: Proceedings of the 2020 7th International Conference on Dependable Systems and Their Applications (DSA). https://doi.org/10.1109/dsa51864.2020.00087

Yurtseven, I., & Bagriyanik, S. (2020). A Review of Penetration Testing and Vulnerability Assessment in Cloud Environment. In: Proceedings of the 2020 Turkish National Software Engineering Symposium (UYMS). https://doi.org/10.1109/uyms50627.2020.9247071

Zatsarinnyy, A., & Ionenkov, Y. (2021). The Efficiency and Quality of Information Systems. In: Proceedings of the 2021 14th International Conference Management of Large-Scale System Development (MLSD). https://doi.org/10.1109/mlsd52249.2021.9600143

Zhang, H., Tuo, A., & Li, G. (2019). Model Checking is Possible to Verify Large-scale Vehicle Distributed Application Systems. In: Proceedings of the 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE). https://doi.org/10.23919/date.2019.8714795

Zhang, Z., Wang, Y., Wang, Z., et al. (2019). How to Effectively Reduce Tens of Millions of Tests: An Industrial Case Study on Adaptive Random Testing. IEEE Transactions on Reliability, 68(4), 1429–1443. https://doi.org/10.1109/tr.2019.2927643

Zhao, Y., Xiao, L., Bondi, A. B., et al. (2023). A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source Projects. IEEE Transactions on Software Engineering, 49(2), 924–946. https://doi.org/10.1109/tse.2022.3167628

Zhi, C., Deng, S., Yin, J., et al. (2019). Quality Assessment for Large-Scale Industrial Software Systems: Experience Report at Alibaba. In: Proceedings of the 2019 26th Asia-Pacific Software Engineering Conference (APSEC). https://doi.org/10.1109/apsec48747.2019.00028

Zhong, H., Zhang, L., & Khurshid, S. (2019). TestSage: Regression Test Selection for Large-Scale Web Service Testing. In: Proceedings of the 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST). https://doi.org/10.1109/icst.2019.00052

Zimmermann, D. (2022). Automated GUI-based Software-Testing Using Deep Neuroevolution. In: Proceedings of the 2022 IEEE Conference on Software Testing, Verification and Validation (ICST). https://doi.org/10.1109/icst53961.2022.00060

Zimmermann, D., & Koziolek, A. (2023). GUI-Based Software Testing: An Automated Approach Using GPT-4 and Selenium WebDriver. In: Proceedings of the 2023 38th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). https://doi.org/10.1109/asew60602.2023.00028