Software engineers at ANZ Bank experienced a significant enhancement in both productivity and code quality while utilizing the GitHub Copilot AI tool.
Last year, approximately 100 engineers at the Australian institution participated in a six-week trial aimed at assessing their perceptions of the tool and its effects on productivity, code quality, and security. Participants were divided into two groups: one that had access to Copilot and a control group, both tasked with completing identical Python challenges.
Results indicated that the engineers who leveraged Copilot were able to complete their assignments 42% more quickly than their control group counterparts. Moreover, the code produced by the Copilot users exhibited fewer code smells and bugs on average, which suggests a higher degree of maintainability and a lower likelihood of errors in production environments.
Despite the positive findings, the research did not yield substantial insights regarding the impact on code security; however, it was noted that Copilot did not introduce any security vulnerabilities.
Participants expressed a favorable attitude toward the technology, noting that it positively influenced their ability to review and comprehend existing code, generate documentation, and create unit tests.
The study ultimately asserts that the findings present substantial evidence of GitHub Copilot’s transformative effect on engineering methodologies at ANZ Bank. The integration of this tool represents a shift in the work paradigm, enabling engineers to dedicate more time to creative and design initiatives while minimizing the hours spent on repetitive coding tasks.
Currently, GitHub Copilot enjoys considerable adoption within the bank, with around 1,000 users reporting improvements in productivity and job satisfaction.
For further details, you can read the paper here.