AI-assisted automation testing: Hands-on research and analysis of GitHub Copilot

Contributing experts

  

75% of enterprise software engineers are expected to use some form of AI code assistants by 2028, according to research conducted by Gartner, Inc. The age of AI-assisted software development is upon us, and it will only grow in stature and scope. However, just as important as maximizing their benefits, it is crucial to understand the current limitations of AI code assistants.  

With a wide array of features and possible applications, GitHub Copilot is one of the best-known and most widely used AI code assistants. This report focuses on examining the effectiveness of GitHub Copilot in one specific area—generating unit tests in iOS projects. It presents the outcome of structured research following a detailed process across three main phases: test writing, report writing, and evaluation, measuring the effectiveness of GitHub Copilot assistance in comparison to creating tests manually. 

In this report, you will find:  

  • Detailed metrics of GitHub Copilot’s effectiveness across a wide array of parameters 
  • Comparison matrix between the results of GitHub Copilot assisted and manual test writing 
  • Conclusions and key observations supported by both objective data and developer feedback 
  • Recommendations for organizations on how to best approach and utilize GitHub Copilot in this segment. 

Download this white paper

Most popular articles