Vision language model
비전 언어 모델
Also known as: VLM · multimodal model
A model that handles images and text together, so a picture and a word can be compared for fit.
A vision only model answers what is this within a fixed list of classes. A vision language model places images and text in a shared space, so you can hand it a new word and ask whether it appears in a photo.
In education the vocabulary is ours to define, so asking about our word list fits better than transcribing whatever names the model knows. Only words that have card art can be shown to a child.
CLIP and SigLIP are well known examples. Both train images and text together so the two can be compared.
- UsesSearch, classification, captioning, object finding.
- LimitsThe more abstract the word, the shakier the judgement.