察元AI教程库 · 文章 · 大模型

刻画大语言模型几何特性有助于解决毒性检测与生成问题

Large Language Models (LLMs) drive current AI breakthroughs despite very little being known about their internal representations. In this work, we propose to shed the light on LLMs inner mechanisms th…

作者:Randall Balestriero, Romain Cosentino, Sarath Shekkizhar · 语言:en

前往原址学习

简介

Large Language Models (LLMs) drive current AI breakthroughs despite very little being known about their internal representations. In this work, we propose to shed the light on LLMs inner mechanisms through the lens of geometry. In particular, we develop in closed form $(i)$ the intrinsic dimension in which the Multi-Head Attention embeddings are constrained to exist and $(ii)$ the partition and pe

返回教程库 · 察元AI产品家族