Maru 2022 WSD
Dataset Reevaluation

Maru 2022 WSDDataset Review

A lexicographer review of the Maru 2022 Word Sense Disambiguation dataset, reevaluating sense assignments against WordNet 3.0 across four benchmark corpora: SemEval-2013, SemEval-2015, Senseval-2, and Senseval-3.

SemEval-2013SemEval-2015Senseval-2Senseval-3

Review Progress

0/37
Pages complete
0/363
Items reviewed
0
"Neither" verdicts
0
Comments left
Overall progress0%
Ring Distribution
S1: 55
S2: 138
S3: 41
S4: 75
S5: 48
S6: 6

Progress is saved automatically in this browser. Use "Export verdicts" to download your annotations.

Browse Review Pages
S1

Ring S1

55 instances

High-frequency, unambiguous instances — clearest sense assignments

01
S1
10 items
treasury, acceptance, issue, rule, discovery…
Open
02
S1
10 items
discontinue, abnormal, useful, useful, new…
Open
03
S1
10 items
solemn, draw, small, believe, same…
Open
04
S1
10 items
mute, involvement, involvement, thing, believe…
Open
05
S1
10 items
discourage, competitive, proportion, state, state…
Open
06
S1S2
10 items
realize, drink, symptom, quiet, bundle…
Ring transition
Open
S2

Ring S2

138 instances

Moderately frequent instances with some sense ambiguity

07
S2
10 items
policy, loss, field, field, u.s.…
Open
08
S2
10 items
question, policy, law, rule, principle…
Open
09
S2
10 items
capability, study, distinction, conservative, administration…
Open
10
S2
10 items
input, input, know, function, now…
Open
11
S2
10 items
study, industry, basis, overall, product…
Open
12
S2
10 items
call, remain, continental, stand, sound…
Open
13
S2
10 items
life, speak, call, hold, growth…
Open
14
S2
10 items
u.s., picture, development, same, growth…
Open
15
S2
10 items
authoritarian, yet, child, education, child…
Open
16
S2
10 items
bright, cold, have, cost, address…
Open
17
S2
10 items
vote, public, public, cost, benefit…
Open
18
S2
10 items
benefit, attendance, rate, devote, attendance…
Open
19
S2
10 items
difference, positive, function, severe, difference…
Open
20
S2S3
10 items
explode, do, bounce, u.s., cycle…
Ring transition
Open
S3

Ring S3

41 instances

Less frequent instances requiring careful disambiguation

21
S3
10 items
consider, say, say, say, say…
Open
22
S3
10 items
say, believe, need, loss, say…
Open
23
S3
10 items
wish, really, large, fact, ready…
Open
24
S3S4
10 items
do, wild, refuge, thing, negotiation…
Ring transition
Open
S4

Ring S4

75 instances

Infrequent senses with higher inter-annotator disagreement

25
S4
10 items
time, area, number, aspect, tone…
Open
26
S4
10 items
know, have, specify, country, organisation…
Open
27
S4
10 items
high, always, initiate, simply, make…
Open
28
S4
10 items
first, loss, begin, sign, development…
Open
29
S4
10 items
dominate, inherent, call, think, reform…
Open
30
S4
10 items
come, grievance, realize, encourage, bright…
Open
31
S4S5
10 items
factor, advantage, factor, factor, comfort…
Ring transition
Open
S5

Ring S5

48 instances

Rare or specialized senses, often domain-specific

32
S5
10 items
family, argument, life, country, country…
Open
33
S5
10 items
effect, best, function, public, public…
Open
34
S5
10 items
say, history, variation, call, come…
Open
35
S5
10 items
loss, think, problem, thing, time…
Open
36
S5S6
10 items
regional, level, level, reveal, state…
Ring transition
Open
S6

Ring S6

6 instances

Extremely rare senses — edge cases for WSD evaluation

37
S6
3 items
fact, think_of, family
Open

About the Dataset

The Maru 2022 WSD dataset is a benchmark for Word Sense Disambiguation, providing gold-standard sense annotations from WordNet 3.0 across multiple evaluation corpora.

Review Methodology

Each instance is reviewed by a lexicographer who selects applicable senses, flags "neither of the above" cases, and adds free-form comments. Progress is saved locally in the browser.

Frequency Rings

Instances are grouped into six frequency rings (S1–S6), from the most common and unambiguous (S1) to the rarest and most challenging (S6) sense assignments.