A Sorani Kurdish name-tagging dataset is now openly licensed, and its labels were made by matching a word list rather than by hand
654,404 labelled tokens from four Kurdish newsrooms, released under CC BY. What it gives anyone building or buying Kurdish-language tools, and the four limits its own authors write down.
Charmo University
Iraq
20254 minutes
- Researcher or analyst
- Director or leadership
- Research
- Cleaning and analyzing data
What it is
Two researchers in the Kurdistan Region of Iraq published a labelled dataset for Sorani Kurdish and released it under CC BY, which means anyone may use it, including commercially, as long as they credit it. Chovyan H. Wahid, of Charmo University in Sulaymaniyah and the Kurdistan Technical Institute, and Rebwar M. Nabi, of Sulaimani Polytechnic University and the Kurdistan Technical Institute, describe it in Data in Brief, published online on 21 August 2025. The data itself sits on Mendeley Data under the identifier 10.17632/6gffcrcj75.2.
The size: 2,300 news articles, 654,404 labelled tokens, 15 entity types. The text comes from four Kurdish newsrooms, Rudaw, Ava.news, NRT and Kurdsat, from 2024 onward.
What name-tagging is actually for
Named entity recognition means finding the names in a text and saying what kind of name each one is: a person, an organisation, a place, a date. It sounds like a small thing. It is the difference between a folder and an archive.
If you hold four thousand Kurdish-language news items, court filings or testimony transcripts, name tagging is what lets you ask which of them mention one particular ministry, or list every place named across the whole collection, or notice that the same person turns up in fourteen files you had filed as unrelated. Without it you have a pile you can only search word by word, where a single spelling variant loses you the document.
How the labels were made, and what that changes
Not by hand. The authors built a lexicon of 12,030 entities, matched it against the text, and tagged with the standard BIO scheme, which marks each token as the Beginning of an entity, Inside one, or Outside all of them. Two independent language experts then checked 30% of the result and agreed with each other at Cohen’s kappa 0.92.
That is a real quality signal and it is not the same thing as human annotation. Lexicon matching finds what is in the lexicon: a name missing from the list is invisible, and a name that is also an ordinary word gets tagged everywhere it appears. The authors name the two problems still open themselves, entity disambiguation and script standardisation, which is the most useful sentence in the paper.
What to do with it
If you are commissioning or evaluating a Kurdish-language tool, you now have a concrete question to put to whoever is selling it: what Kurdish data was this trained on, and does it handle Badini as well as Sorani? A system built on this dataset will be at its best on recent Sorani news writing. Assume nothing about how it behaves anywhere else, and test it on your own documents before you trust it with an archive.
If you are not building anything, the useful thing is simply knowing this exists and that its licence permits reuse. When someone tells you Kurdish-language tools cannot be built because the data is not there, this is a counter-example you can name, with its own limits written down by the people who made it.
Tools the sources name
The sources name no tool for this case. We have not guessed one, and neither should anyone repeating the story.
What we do not claim
Everything above is limited by what its sources actually prove. This is the part they do not.
The labels were not written by people. They were produced by matching the text against a lexicon of 12,030 entities and tagging with the BIO scheme. Two independent language experts then validated 30% of the data and agreed with each other at Cohen's kappa 0.92. Do not describe this as a hand-annotated corpus. And note what the 0.92 measures: agreement between those two experts on the portion they checked, not the accuracy of the whole dataset.
Sorani only. There is no Badini and no Kurmanji in it. A tool built on this alone will not serve Kurdish speakers who write in those varieties.
Nothing in it predates 2024. It is not evidence about earlier Kurdish news language and it contains no earlier events.
The authors flag two problems as unresolved: entity disambiguation, and script standardisation. Their own paper says so. Do not present the dataset as finished.
The text is news writing from four outlets. Whatever those four newsrooms cover, and however they spell names, is what anything trained on this will learn. That is our reading, not a finding of the paper.