Watch: Introducing LinkjeBERT a Dutch Language Model
LinkjeBERT is a Dutch language model trained on structured Markdown to predict where humans will place links in text using token-level confidence levels.
Transcript
We are excited to introduce LinkjeBERT, a new language model trained specifically for our Dutch-speaking friends around the world. LinkjeBERT is a link expert. When you give it plain text, it can predict exactly where a human editor would place a link, doing so even better than models like Gemini, Claude, or GPT.
To achieve this, the model was trained for seven days on two hundred million tokens of high-quality training data from top editorial websites covering news, tech, science, and lifestyle. It is built on top of Microsoft’s multilingual transformer, mDeBERTa-v3-base, and fine-tuned as a binary token classifier. Every single token is evaluated to determine if it should be part of a link anchor. Because links are relatively rare in natural text, we used a specialized loss function to handle the class imbalance.
Our breakthrough came when we changed how the model sees text. Our first versions were trained on flat, unformatted text, but they struggled because they missed the structural cues humans rely on. For LinkjeBERT, we decided to train on structure-preserving Markdown. This means the model can see headings, lists, bold text, and blockquotes. By understanding the document's skeletal structure, it gains the same contextual signals a human editor uses. To our knowledge, this is the first link prediction model ever trained on structured markup rather than flat text, making it an incredibly powerful tool for link building.