No AI summary available for this article.
Why It Matters
A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences.
Provenance
Discovered via ArXiv and published by ArXiv.
Key Claims
Original description
A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences. We argue that such single-pair comparisons are often unreliable: simply rewriting the same stereotype with an alternative attribute can yield logically inconsistent preferences. To address this, we propose a dual minimal pair setup that introduces two axes of comparison for robust stereotype evaluation. First, we present a data-augmentation framework that fills critical gaps in existing stereotype datasets by generating paraphrases and alternate attributes. We...
Discovered via ArXiv
Research papers and preprints from arXiv.
Publisher: arxiv.org
ID: http://arxiv.org/abs/2610.08747v1 · Indexed about 1 hour ago