LightlyTrain 0.17.0 brings LTDETRv2 to instance segmentation, with the smallest variant matching RF-DETR-seg-nano accuracy at a third of the parameters. It also adds larger LTDETR COCO checkpoints, distilled Depth Anything V3 depth models, and LingBot-Vision backbone support.
LightlyTrain 0.17.0 extends LTDETRv2 into instance segmentation with the best accuracy-vs-parameter-count trade-off amongst mainstream models!
Furthermore, the new release extends the existing model line-up with:
Taken together, LightlyTrain 0.17.0 makes high-accuracy vision genuinely practical to deploy in the field, pairing stronger out-of-the-box accuracy across object detection, instance segmentation, and depth estimation with the compactness and inference speed that production deployments actually demand.
For more information, check out LightlyTrain Documentation.
Following the introduction of LTDETRv2 for object detection in LightlyTrain 0.16.0, we now bring its innovative design also to instance segmentation in 4 sizes: s/m/l/x, inspired by EdgeCrafter’s ECSeg.
LTDETRv2-seg offers the best accuracy-vs-parameter-count trade-off among current SOTA instance segmentation models. On COCO val2017, LTDETR-seg-s reaches 40.4 mAP with only 11.3M parameters — matching RF-DETR-seg-nano's accuracy at roughly one-third the parameter count, and significantly exceeding every YOLO11 variant (top: 40.1 mAP at 62.2M) with a fraction of the weights.
This makes LTDETRv2-seg well-suited to storage-, memory-footprint-, and bandwidth-constrained deployments - for example, fitting on minimizing OTA update size, or co-hosting many specialized models on a single device.
The values were measured using TensorRT10.13.3.9 and FP16 precision on a Nvidia T4 GPU with batch size 1. These COCO weights are directly available for further fine-tuning with a -coco suffix.
You can use any of the COCO-pretrained checkpoints as the starting point for fine-tuning on your own instance segmentation dataset:
import lightly_train
lightly_train.train_instance_segmentation(
out="out/my_experiment",
model="ltdetrv2-seg-s-coco",
steps=100,
batch_size=4,
data="data/train.yaml",
)Curate and label data, fine-tune foundation models — all in one platform.
Book a Demo
Following the initial LTDETRv2 release, we have added two new COCO-pretrained checkpoints:
ltdetrv2-m-cocoltdetrv2-l-cocoTogether with ltdetrv2-s-coco, the model family covers several deployment budgets. LTDETRv2 provides an optimal parameter-efficiency trade-off, achieving higher COCO accuracy than other real-time detectors such as YOLO11 at comparable parameter counts, while maintaining low inference latency on an NVIDIA T4.
We've distilled Depth Anything V3, the SOTA for monocular depth, into the smallest and fastest models we've shipped so far. Using the large (ViT-L) DAv3 as the teacher, we trained three compact students, ViT-Small, ViT-TinyPlus, and ViT-Tiny, on a large, diverse set of real-world images.Â
Each comes in both metric (distance in meters) and relative (scale- and shift-invariant) variants. The smallest is just 6.2M parameters, over 50x smaller than the teacher, making real-time depth on edge hardware practical.
All numbers are zero-shot: NYUv2 was not used in training. Evaluated on the NYUv2 test split.
Relative models are scored after a per-image least-squares scale-and-shift alignment to the ground truth, so the numbers are affine-invariant and not directly comparable to the metric table:
Metric models are scored directly against the ground-truth depth. We additionally report delta_1 (aligned), which applies the same per-image least-squares scale-and-shift alignment used for the relative models.
import lightly_train
model = lightly_train.load_model("dinov2/dav3-relative-small")
depth = model.predict("image.jpg") # (H, W) tensor at input resolutionSwap in a metric checkpoint (e.g. dinov3/dav3-metric-tiny) for absolute distances. Batch inference is available via predict_batch.
LightlyTrain now supports LingBot-Vision (paper) alongside DINOv2, DINOv3, and EUPE, so you can pretrain, distill, and fine-tune an EoMT segmentation model with it on your own unlabeled data!
Released by Robbyant, LingBot-Vision is a family of self-supervised ViT backbones for dense spatial perception. Instead of optimizing for semantic invariance, it trains with a boundary-centric objective (masked boundary modeling) focused on contours, edges, and geometric structure, a natural fit for dense tasks like segmentation and depth.
Tested frozen with a single linear head on ADE20K semantic segmentation (ViT-S), it leads at 46.54 mIoU, ahead of EUPE (45.56) and DINOv3 (44.80) backbones.
Fine-tune a LingBot backbone with EoMT, the state-of-the-art segmentation architecture, in a few lines:
lightly_train.train_semantic_segmentation(
model="dinov3/vits16-lingbot-eomt",
...
)Also available:dinov3/{vitb16,vitl16}-lingbot-eomt.
LightlyTrain 0.17.0 adds three improvements that make training instabilities easier to debug.
We now log grad_norm by default, which helps detect unstable gradients during training. We also added support for DebugUnderflowOverflow and NaNCapture.
DebugUnderflowOverflow helps identify the layer where an instability originated, while NaNCapture lets you reproduce the failing step for more detailed inspection and diagnostics.
Interactive examples are available on Google Colab: debugging tools tutorial.
Self-Supervised Pretraining
Leverage self-supervised learning to pretrain models
AI Training Data for LLMs & CV
Expert training data services for LLMs, AI Agents and vision
.png)
.png)
.svg)