2024-02-13 12:09:43
Microsoft and GitHub conducted an in-depth study to explore the challenges and opportunities in developing AI-powered assistants (Copilots), by interviewing 26 software engineers.
Developers face challenges such as testing difficulties, the need to follow best practices in a rapidly evolving field, as well as issues related to safety and privacy.
These results were revealed in conjunction with Microsoft’s launch of updates to its CoPilot program, focusing on improving performance and adding new features.
Researchers at the company “Microsoftand GitHub conducted an in-depth study on the challenges, opportunities, and needs associated with developing AI-powered assistants, “Copilots.” The study included interviews with 26 professional software engineers from various companies responsible for developing these advanced tools.
The competition to integrate advanced AI into products is ongoing, as technology companies seek to add these capabilities to their software; However, many challenges remain. Coordinating multiple data sources and claims can increase the risk of failure, while testing large language models (LLMs) is difficult due to their volatility.
Developers also find it difficult to keep up with best practices in this rapidly evolving field, and often turn to social media or academic papers for guidance. Safety, privacy, and compliance are also key concerns that require careful management to avoid potential damages or breaches.
Austin Henley, an AI researcher at Microsoft and a professor at the University of Tennessee, noted: “Finding a one-stop shop for integrating AI into projects is still a challenge. Developers are looking for a place to get started quickly, moving from experimentation to a viable prototype.” (MVP), connecting their various data sources to claims, and then efficiently integrating the AI components into the existing programming base.”
“Having a quick debugging tool that can provide immediate feedback, developers also requested a library or ‘toolbox’ containing a set of pre-made snippets that can be used for common AI tasks,” Henley added.
An important challenge also identified was the challenge of agile engineering, which is the process of creating prompts or triggers that trigger the inference process of the AI model. One participant stated, “Large language models are often so fragile in terms of responses that there is little control and direction possible through stimulation.” This unpredictability is a challenge as developers have to navigate through trial-and-error processes that can be time-consuming.
The issue of testing standards was also raised; With generative models like LLMs, writing assertions becomes difficult because each answer is different. One participant explained, “That’s why we run each test 10 times,” while another added, “The experiment is the most time-consuming if the right tools aren’t available.”
Additionally, participants expressed concerns regarding safety and privacy issues associated with integrating AI into products. One of the participants stressed the importance of having appropriate protective measures, especially when using artificial intelligence technology in sensitive areas, pointing out the potential risks in using these technologies without appropriate guarantees.
Finally, one participant expressed the difficulty of learning these new technologies and tools, noting that there is no specific path or clear plan for learning and applying smart technology in the right way. This comment highlights the challenge they face in understanding new technology and using it effectively, and indicates the necessity of continuous research and following up on developments in this field to ensure the continued development and optimal use of these technologies.
It is worth noting that these changes come in conjunction with Microsoft’s recent launch of updates and upgrades to its CoPilot program, as all English-speaking CoPilot users in the United States, the United Kingdom, Australia, India, and New Zealand can now edit photos while chatting. Many Microsoft CoPilot Pro subscribers have reported performance issues.
The best we have, keep it up! Register for the Arageek newsletter
Register
By clicking on the “Register” button, you agree to the Terms of Service and Privacy Policy and to receive mailings from Arageek
1707827191
#Microsoft #shares #challenges #software #developers #building #AIpowered #assistants #Copilots
