The Efficiency-Throughput Gap with GitHub Copilot

Researchers have analyzed GitHub Copilot, a large language model-based coding assistant, and found that it can significantly reduce development time but at the cost of decreased code quality. They identified an 'efficiency-throughput gap' where the tool's efficiency improvements come at the cost of reduced code quality, making it harder to maintain and understand the code. The researchers used a dataset of 100 code changes made by Copilot and human developers, comparing the two. They found that Copilot's efficiency gains were about 2.5 times higher, but the quality of the code was 1.3 times lower. This gap can be problematic for developers, as it may lead to more bugs and maintenance issues down the line.