#pretraining

LingBot-Vision introduces masked boundary modeling, where the teacher network generates boundary tokens for the student to reconstruct, achieving a state-of-the-art NYUv2 linear-probe RMSE of 0.296 at 1.1B parameters, outperforming DINOv3-7B's 0.309. This novel self-supervised pretraining method significantly improves depth estimation performance with less data (161M images vs. DINOv3's ~500M), potentially reducing the need for labeled data and large-scale pretraining in computer vision. The method uses a teacher-student framework where boundary tokens are derived from the teacher's own predictions, avoiding external edge detectors; masked regions force reconstruction of boundary-bearing areas. The model comes in four sizes, with weights released under Apache-2.0 license.