Recently eBay disclosed the lessons learned about the application of generative AI in the development process. eBay’s AI endeavors have uncovered three pivotal avenues toward enhancing developer productivity: integrating commercial offerings, fine-tuning existing Large Language Models (LLMs), and harnessing an internal knowledge network.
Adopting commercial AI solutions such as GitHub Copilot has yielded promising results for eBay’s developer community. In a meticulously designed A/B test, developers utilizing Copilot demonstrated heightened productivity, marked by a notable increase in code acceptance rates (27% code acceptance rate reported through Copilot telemetry), and efficiency metrics: 60% accuracy for the generated code. The introduction of Github Copilot also allows a decrease of PR (about 17%) and a decrease (about 12%) of the Lead time for Change. However, limitations such as prompt size constraints underscore the need for tailored solutions in the context of eBay’s vast codebase.
