University of Stirling researchers have trained ChatGPT to produce faster versions of a software programme.
Making software run faster and more reliably is challenging and time consuming for software developers.
The study led by Dr Sandy Brownlee used ChatGPT - an artificial intelligence (AI) large language model (LLM) - to automatically update software by asking it to make improvements to computer coding.
The results of the research could be used to improve mobile apps to make them more responsive and less draining on smart phone batteries.
Senior lecturer in the Computing Science and Mathematics Division of the Faculty of Natural Sciences
We’re making tools that support developers by providing more information and it’s still critical to have a human involved to use that information effectively.
Dr Brownlee, a senior lecturer in the University of Stirling’s Computing Science and Mathematics Division in the Faculty of Natural Sciences, said: “We found that, on the open source project we used as a case study, a LLM was able to produce faster versions of the programme around 15% of the time, which is half as good again as the previous approach.
“Most interestingly was that the LLM was able to take examples from other parts of the programme that we hadn’t explicitly told it about in order to make these improvements.
“The most tangible benefit is in your pocket - mobile apps that run more efficiently mean that your battery lasts longer, and the apps will be more responsive when in use.”
More efficient improvements
Software is becoming increasingly complex and difficult to maintain. It is also having a growing environmental impact as computers consume more and more energy, so creating more efficient automatic improvements to software are crucial if countries are to achieve net-zero commitments.
Dr Brownlee said: “There is a trade-off here because LLMs cost a lot of energy to make and use but if they can be used to improve a piece of software that is run many times, the payoff may be worth it.”
Dr Brownlee hopes that the research could help software developers who are striving to create more efficient programmes.
He said: “The nature of software developers’ roles will change if automated improvement to software becomes commonplace, moving to a higher level of design, though that is continuing the direction of travel we’ve seen for decades.
“It’s quite possible to be concerned about all this - AI writing computer code sounds like the beginning of a sci-fi disaster plot – but this is not going to lead to an AI takeover.
“We’re making tools that support developers by providing more information and it’s still critical to have a human involved to use that information effectively.”
The peer-reviewed paper, Enhancing Genetic Improvement Mutations Using Large Language Models, was presented by Dr Brownlee on Friday, December 8 at the Symposium on Search Based Software Engineering 2023, in San Francisco, part of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
Dr Brownlee led the research and collaborated with Professor Justyna Petke, Professor Federica Sarro and PhD students James Callan and Carol Hanna, all of University College London; Dr Dominik Sobania and PhD student Alina Geiger of Johannes Gutenberg University; and Dr Karine Even-Mendoza of King’s College London.
The team’s work was supported by a UKRI EPSRC grant and the ERC advanced fellowship grant.