Hi LinkedIn community, I’m starting a new blog! I plan to highlight recent academic research, policy whitepapers, and business trends related to AI safety and governance. I’ll also be blogging over here: https://lnkd.in/gDuqgUdg
I’m kicking off the series on the risks of open-source advanced AI with a must-read paper: “Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!” written by Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson.
Background: Language models generally undergo a training procedure to make sure that their outputs are “helpful, honest, and harmless,” often summarily called “safety training.” Safety training incorporates explicit guidance as how to formulate outputs to reflect human values.
However, it is unclear the extent to which safety training is “fragile” and may degrade after fine-tuning models for specific purposes.
Experiment: The study tested this by retraining models (like GPT-3.5 Turbo) with three data sets that account for different vulnerabilities.
1. Explicit inputs and targets that violate OpenAI’s & Meta’s usage policies.
2. An 'identity shift' in which a model is fine-tuned to affirm a new and harmful identity at the start of a prompt.
3. Totally benign inputs and targets.
Results: There are increases in the harmfulness of the content produced across a variety of harmful categories. The first two intentional methods of circumventing the safety training yield ~90% increase in harmful outputs, and there is a ~25% increase in harmful outputs after fine-tuning with benign data.
Insights (1): Businesses that want to use and fine-tune open-source language models should implement auditing protocols that ensure that their models have not inadvertently lessened their safe behavior. They may face liability for these unsafe outputs.
Insights (2): Preventing the intentional misuse of open-source models is paramount. The concept of "responsible AI" licensing, proposed by the authors, along with required safety audits after model updates, might curb deliberate attempts to compromise model integrity.
I'm curious, does anyone here know of companies developing "responsible AI" licensing for open-source models?
Link to paper: https://lnkd.in/geSgX3x5
#AISafety #LLMs #AIRegulation #ResponsibleAI