AI Safety in the Eyes of the Downstream Developer: A First Look at Concerns, Practices, and Challenges

Abstract

Concerns, Practices, and Challenges Authors: ['Haoyu Gao' 'Mansooreh Zahedi' 'Wenxin Jiang' 'Hong Yi Lin' 'James C. Davis' 'Christoph Treude'] Many recent endeavours from the research community propose taxonomies of AI safety concerns [11, 29], evaluation methods [86], and mitigation approaches [43, 75, 90]. Additionally, policy makers are also making efforts to provide regulations on AI safety and responsible AI design and development [2, 5]. Despite these efforts, how AI-based software developers perceive AI safety concerns, as well as their practices and challenges during the development phase, is not investigated. Due to the complexity of AI models and the knowledge gap between developers and model designers [51, 69], developers may approach AI safety differently and face various challenges. This is particularly true for smaller-scale company and open-source developers, where fewer resources are provided and regulations might not be enforced. This gap between AI safety frameworks and real-world practices highlights the need for an investigation into developers' workflows, challenges, and potential shortcomings to better inform researchers and policymakers in supporting AI safety implementation.