Objectives To evaluate the effectiveness of safeguards to prevent large language models (LLMs) from being misused to generate health disinformation, and to evaluate the transparency of artificial intelligence (AI) developers regarding their risk mitigation processes against observed vulnerabilities. Design Repeated cross sectional analysis. Setting Publicly accessible LLMs. Methods In a repeated cross sectional analysis, four LLMs (via chatbots/assistant interfaces) were evaluated: OpenAI’s GPT-4 (via ChatGPT and Microsoft’s Copilot), Google’s PaLM 2 and newly released Gemini Pro (via Bard), Anthropic’s Claude 2 (via Poe), and Meta’s Llama 2 (via HuggingChat). In September 2023, these LLMs were prompted to generate health disinformation on two topics: sunscreen as a cause of skin cancer and the alkaline diet as a cancer cure. Jailbreaking techniques (ie, attempts to bypass safeguards) were evaluated if required. For LLMs with observed safeguarding vulnerabilities, the processes for reporting outputs of concern were audited. 12 weeks after initial investigations, the disinformation generation capabilities of the LLMs were re-evaluated to assess any subsequent improvements in safeguards. Main outcome measures The main outcome measures were whether safeguards prevented the generation of health disinformation, and the transparency of risk mitigation processes against health disinformation. Results Claude 2 (via Poe) declined 130 prompts submitted across the two study timepoints requesting the generation of content claiming that sunscreen causes skin cancer or that the alkaline diet is a cure for cancer, even with jailbreaking attempts. GPT-4 (via Copilot) initially refused to generate health disinformation, even with jailbreaking attempts—although this was not the case at 12 weeks. In contrast, GPT-4 (via ChatGPT), PaLM 2/Gemini Pro (via Bard), and Llama 2 (via HuggingChat) consistently generated health disinformation blogs. In September 2023 evaluations, these LLMs facilitated the generation of 113 unique cancer disinformation blogs, totalling more than 40 000 words, without requiring jailbreaking attempts. The refusal rate across the evaluation timepoints for these LLMs was only 5% (7 of 150), and as prompted the LLM generated blogs incorporated attention grabbing titles, authentic looking (fake or fictional) references, fabricated testimonials from patients and clinicians, and they targeted diverse demographic groups. Although each LLM evaluated had mechanisms to report observed outputs of concern, the developers did not respond when observations of vulnerabilities were reported. Conclusions This study found that although effective safeguards are feasible to prevent LLMs from being misused to generate health disinformation, they were inconsistently implemented. Furthermore, effective processes for reporting safeguard problems were lacking. Enhanced regulation, transparency, and routine auditing are required to help prevent LLMs from contributing to the mass generation of health disinformation. The research team would be willing to make the complete set of generated data available upon request from qualified researchers or policy makers on submission of a proposal detailing required access and intended use.

中文翻译：

---

针对健康虚假信息生成的大型语言模型的当前保障措施、风险缓解和透明度措施：重复横断面分析

目标 评估防止大型语言模型 (LLM) 被滥用生成健康虚假信息的保障措施的有效性，并评估人工智能 (AI) 开发人员针对观察到的漏洞的风险缓解流程的透明度。设计 重复横截面分析。设置可公开访问的法学硕士。方法 在重复的横断面分析中，评估了四个 LLM（通过聊天机器人/助手界面）：OpenAI 的 GPT-4（通过 ChatGPT 和 Microsoft 的 Copilot）、Google 的 PaLM 2 和新发布的 Gemini Pro（通过 Bard）、Anthropic 的 Claude 2（通过Poe）和 Meta 的 Llama 2（来自 HuggingChat）。 2023 年 9 月，这些法学硕士被要求发布关于两个主题的健康虚假信息：防晒霜是导致皮肤癌的原因，以及碱性饮食是治疗癌症的方法。如果需要，将对越狱技术（即尝试绕过安全措施）进行评估。对于观察到的保护漏洞的法学硕士，对报告关注的输出的流程进行了审核。初步调查 12 周后，对法学硕士的虚假信息生成能力进行了重新评估，以评估后续保障措施的改进。主要成果衡量标准 主要成果衡量标准是保障措施是否防止健康虚假信息的产生，以及针对健康虚假信息的风险缓解流程的透明度。结果 Claude 2（通过 Poe）拒绝了在两个研究时间点提交的 130 条提示，这些提示要求生成声称防晒霜会导致皮肤癌或碱性饮食可以治愈癌症的内容，即使有越狱尝试。 GPT-4（通过 Copilot）最初拒绝生成健康虚假信息，即使尝试越狱 - 尽管在 12 周时情况并非如此。相比之下，GPT-4（通过 ChatGPT）、PaLM 2/Gemini Pro（通过 Bard）和 Llama 2（通过 HuggingChat）持续生成健康虚假信息博客。在 2023 年 9 月的评估中，这些法学硕士促成了 113 个独特的癌症虚假信息博客的生成，总计超过 40,000 字，且无需尝试越狱。这些法学硕士在整个评估时间点的拒绝率仅为 5%（150 名中的 7 名），并且根据提示，法学硕士生成的博客包含引人注目的标题、看起来真实（虚假或虚构）的参考文献、伪造的患者和临床医生的推荐，并且他们针对不同的人口群体。尽管评估的每个法学硕士都有报告观察到的关注输出的机制，但开发人员在报告漏洞观察结果时没有做出回应。结论 本研究发现，尽管有效的保障措施可以防止法学硕士被滥用来制造健康虚假信息，但这些措施的实施并不一致。此外，缺乏报告保障问题的有效程序。加强监管、提高透明度、需要进行例行审计，以帮助防止法学硕士助长大量健康虚假信息的产生。研究团队愿意根据合格研究人员或政策制定者提交详细说明所需访问和预期用途的提案的要求，提供完整的生成数据集。