Enhancing Out-of-Distribution Detection Through Stochastic Embeddings in Self-supervised Learning
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN: 1611-3349, Vol: 14838 LNCS, Page: 337-351
2024
- 1Citations
- 2Captures
Metric Options: CountsSelecting the 1-year or 3-year option will change the metrics count to percentiles, illustrating how an article or review compares to other articles or reviews within the selected time period in the same journal. Selecting the 1-year option compares the metrics against other articles/reviews that were also published in the same calendar year. Selecting the 3-year option compares the metrics against other articles/reviews that were also published in the same calendar year plus the two years prior.
Example: if you select the 1-year option for an article published in 2019 and a metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019. If you select the 3-year option for the same article published in 2019 and the metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019, 2018 and 2017.
Citation Benchmarking is provided by Scopus and SciVal and is different from the metrics context provided by PlumX Metrics.
Example: if you select the 1-year option for an article published in 2019 and a metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019. If you select the 3-year option for the same article published in 2019 and the metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019, 2018 and 2017.
Citation Benchmarking is provided by Scopus and SciVal and is different from the metrics context provided by PlumX Metrics.
Conference Paper Description
In recent years, self-supervised learning has played a pivotal role in advancing machine learning by allowing models to acquire meaningful representations from unlabeled data. An intriguing research avenue involves developing self-supervised models within an information-theoretic framework, e.g., feature decorrelation methods like Barlow Twins and VICReg, which can considered as particular implementations of the information bottleneck objective. However, many studies often deviate from the stochasticity assumptions inherent in the information-theoretic framework. Our research demonstrates that by adhering to these assumptions, specifically by employing stochastic embeddings in the form of a parametrized conditional density, we can not only achieve performance comparable to deterministic networks but also significantly improve the detection of out-of-distribution examples, surpassing even the performance of supervised detectors. With VICReg, specifically, we achieve an average AUROC of 0.858 for the stochastic unsupervised detector, compared to 0.796 for the supervised baseline. Remarkably, this improvement is achieved solely by leveraging information from the underlying embedding distribution.
Bibliographic Details
http://www.scopus.com/inward/record.url?partnerID=HzOxMe3b&scp=85199541993&origin=inward; http://dx.doi.org/10.1007/978-3-031-63783-4_25; https://link.springer.com/10.1007/978-3-031-63783-4_25; https://dx.doi.org/10.1007/978-3-031-63783-4_25; https://link.springer.com/chapter/10.1007/978-3-031-63783-4_25
Springer Science and Business Media LLC
Provide Feedback
Have ideas for a new metric? Would you like to see something else here?Let us know