LingBot-Vision introduces masked boundary modeling, where the teacher network generates boundary tokens for the student to reconstruct, achieving a state-of-the-art NYUv2 linear-probe RMSE of 0.296 at 1.1B parameters, outperforming DINOv3-7B's 0.309. This novel self-supervised pretraining method significantly improves depth estimation performance with less data (161M images vs. DINOv3's ~500M), potentially reducing the need for labeled data and large-scale pretraining in computer vision. The method uses a teacher-student framework where boundary tokens are derived from the teacher's own predictions, avoiding external edge detectors; masked regions force reconstruction of boundary-bearing areas. The model comes in four sizes, with weights released under Apache-2.0 license.
Background
Self-supervised learning aims to learn useful representations from unlabeled data. Masked modeling, like BERT in NLP, predicts masked parts of input. Boundary detection identifies object edges. LingBot-Vision combines these: it masks boundary tokens predicted by the teacher, so the student must learn to reconstruct structural boundaries.
References
Discussion
The Reddit community discusses the evaluation rigor, noting that a 0.013 RMSE difference could be due to probe hyperparameters, and the lack of ablation against hard-masking baselines like ADIOS/AttMask. Some question whether the comparison with DINOv3 is settled, as Gram anchoring is still used.