No AI summary available for this article.
Why It Matters
Deep neural networks tend to rely on simple features that may be spurious and thus fail to generalize.
Provenance
Discovered via ArXiv and published by ArXiv.
Key Claims
Original description
Deep neural networks tend to rely on simple features that may be spurious and thus fail to generalize. We study this problem in the setting of linear probes, where a (generalized) linear model is fitted on the representations of a (pretrained) model. We use the connection of these models to the max-margin classifier, and show they favor directions associated with large eigenvalues of the covariance matrix. Whitening removes this preference by equalizing the eigenvalues of the covariance matrix. This observation motivates whitening as a preprocessing step that can reduce reliance on spurious co...
Discovered via ArXiv
Research papers and preprints from arXiv.
Publisher: arxiv.org
ID: http://arxiv.org/abs/2609.39177v1 · Indexed about 1 hour ago