PlumX Metrics
Embed PlumX Metrics

Enhancing Out-of-Distribution Detection Through Stochastic Embeddings in Self-supervised Learning

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN: 1611-3349, Vol: 14838 LNCS, Page: 337-351
2024
  • 1
    Citations
  • 0
    Usage
  • 2
    Captures
  • 0
    Mentions
  • 0
    Social Media
Metric Options:   Counts1 Year3 Year

Metrics Details

Conference Paper Description

In recent years, self-supervised learning has played a pivotal role in advancing machine learning by allowing models to acquire meaningful representations from unlabeled data. An intriguing research avenue involves developing self-supervised models within an information-theoretic framework, e.g., feature decorrelation methods like Barlow Twins and VICReg, which can considered as particular implementations of the information bottleneck objective. However, many studies often deviate from the stochasticity assumptions inherent in the information-theoretic framework. Our research demonstrates that by adhering to these assumptions, specifically by employing stochastic embeddings in the form of a parametrized conditional density, we can not only achieve performance comparable to deterministic networks but also significantly improve the detection of out-of-distribution examples, surpassing even the performance of supervised detectors. With VICReg, specifically, we achieve an average AUROC of 0.858 for the stochastic unsupervised detector, compared to 0.796 for the supervised baseline. Remarkably, this improvement is achieved solely by leveraging information from the underlying embedding distribution.

Provide Feedback

Have ideas for a new metric? Would you like to see something else here?Let us know