Contrastive learning is a self-supervised learning approach that trains models to distinguish between similar (positive) and dissimilar (negative) pairs of data. The goal is to learn representations where similar examples are pulled closer together in the embedding space, while dissimilar ones are pushed apart.
Positive pairs are typically created through data augmentations (e.g., different crops of the same image), while negatives are other samples in the batch. Popular contrastive learning frameworks include SimCLR, MoCo, and InfoNCE. These methods often rely on large batch sizes, strong augmentations, and projection heads to improve learning dynamics.
Contrastive learning has led to state-of-the-art results in computer vision and is increasingly used in NLP, audio, and multimodal tasks. It enables models to learn useful representations from unlabeled data, reducing reliance on expensive annotations.
Key challenges include selecting effective positives/negatives and balancing computational cost. Recent approaches like BYOL and SimSiam reduce or eliminate the need for negative samples altogether.
Data Selection & Data Viewer
Get data insights and find the perfect selection strategy
Learn MoreSelf-Supervised Pretraining
Leverage self-supervised learning to pretrain models
Learn MoreSmart Data Capturing on Device
Find only the most valuable data directly on devide
Learn More