Hugging Face's logo

YurtsAI
/
ner-document-context

Token Classification
SpanMarker
TensorBoard
Safetensors
English
ner
named-entity-recognition
generated_from_span_marker_trainer
Eval Results
Model card Files Metrics Community
1

SpanMarker with roberta-large on YurtsAI/named_entity_recognition_document_context

This is a SpanMarker model trained on the YurtsAI/named_entity_recognition_document_context dataset that can be used for Named Entity Recognition. This SpanMarker model uses roberta-large as the underlying encoder.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
DATETIME__absolute "14:00 hrs", "15th november 2023 at 10:00 am", "october 15th , 2023"
DATETIME__authored "25 february 26", "sunday , 21 august , 1938", "1961-05-08"
DATETIME__range "29th of oct. , 2023", "september 2021 to august 2023", "jan 2022 - dec 2022"
DATETIME__relative "eod friday", "dec 15 , 11:59 pm", "10/15"
GENERAL__art-broadcastprogram "stranger things", "live q & a", "product design concept sketchbook for kids"
GENERAL__art-film "the crown", "kill bill", "stranger things"
GENERAL__art-music
GENERAL__art-other "statue of liberty", "broadway show", "wicked"
GENERAL__art-painting "draw your dream house", "design a superhero costume"
GENERAL__art-writtenart "optimization of quantum algorithms for cryptographic applications", "introduction to algorithms", "intro to cs '' by j. doe"
GENERAL__building-airport "ory", "charles de gaulle", "cdg"
GENERAL__building-hospital "green valley clinic", "department of oncology", "st. mary 's hospital"
GENERAL__building-hotel "le jules verne", "hôtel ritz", "the beverly hills hotel"
GENERAL__building-library "ancient library", "the grand library", "jefferson library"
GENERAL__building-other "louvre museum", "engineering building", "eiffel tower"
GENERAL__building-restaurant "l'ambroisie", "bella 's bistro", "in-n-out burger"
GENERAL__building-sportsfacility "fenway"
GENERAL__building-theater "gershwin theatre", "opera house", "broadway"
GENERAL__event-attack/battle/war/militaryconflict "1863 battle of ridgefield", "battle of gettysburg", "war of 1812"
GENERAL__event-other "annual science fair", "summer splash '23", "research methodology workshop"
GENERAL__event-sportsevent "international olympiad in informatics", "ftx", "ioi"
GENERAL__location-GPE "fr", "paris ,", "italy"
GENERAL__location-bodiesofwater "river x", "river blue", "seine river"
GENERAL__location-island "maldives", "similan islands", "ellis island"
GENERAL__location-mountain "andes mountains", "swiss alps", "pine ridge"
GENERAL__location-other "times square", "old market", "venice beach"
GENERAL__location-park "central park", "ueno park", "universal studios"
GENERAL__location-road/railway/highway/transit "i-95", "underground railroad", "hollywood walk of fame"
GENERAL__organization-company "green earth organics", "xyz corporation", "north atlantic fisheries"
GENERAL__organization-education "graduate school", "xyz", "xyz university"
GENERAL__organization-government/governmentagency "department of economic development", "moe", "ministry of environment"
GENERAL__organization-media/newspaper "pinterest", "yelp", "insta"
GENERAL__organization-other "historical society", "grants office", "admissions committee"
GENERAL__organization-religion "buddhist", "zen buddhist", "shinto"
GENERAL__organization-showorganization "phare", "the soundbytes"
GENERAL__organization-sportsteam "varsity soccer team", "red sox"
GENERAL__other-astronomything
GENERAL__other-award "team excellence award", "innovation award", "employee of the month"
GENERAL__other-biologything "fodmap", "troponin i", "cmp"
GENERAL__other-chemicalthing "co2", "pm2.5", "nitrate"
GENERAL__other-currency "usd", "inr", "$ $ $"
GENERAL__other-disease "mi", "irritable bowel syndrome", "myocardial infarction"
GENERAL__other-educationaldegree "executive mba", "phd in quantum computing ,", "phd"
GENERAL__other-god "inari", "athena", "inari taisha"
GENERAL__other-language "french", "english", "spanish"
GENERAL__other-law "cas", "clean air standards", "environmental protection act ( epa ) 2023"
GENERAL__other-livingthing "eastern box turtle", "monarch butterfly", "western burrowing owl"
GENERAL__other-medical "asa", "dapt", "clopidogrel"
GENERAL__person-artist/author "carol", "picasso", "warhol"
GENERAL__person-other "jamie", "sarah", "mark"
GENERAL__person-politician "jane doe", "vespasian", "constantine i"
GENERAL__person-scholar "dr. smith", "dr. lee", "dr. johnson"
GENERAL__person-soldier "davis", "lt. sarah johnson", "col. r. johnson"
GENERAL__product-airplane "hmmwvs", "uh-60s", "m1a2s"
GENERAL__product-car "hmmwvs", "high mobility multipurpose wheeled vehicles", "mine-resistant ambush protected"
GENERAL__product-food "pumpkin spice", "quinoa salad", "golden jubilee feast"
GENERAL__product-game "stardew valley", "valorant", "call of duty : warzone"
GENERAL__product-other "engagement metrics", "xj-200", "smart goal templates"
GENERAL__product-ship "liberty island ferry", "hms victory", "thames river cruise"
GENERAL__product-software "instagram", "svm", "r"
GENERAL__product-train "n'ex", "shinkansen", "tgv"
GENERAL__product-weapon "m1 abrams", "m4 carbine", "m4 carbines"

Evaluation

Metrics

Label Precision Recall F1
all 0.8309 0.8390 0.8349
DATETIME__absolute 0.8744 0.8577 0.8660
DATETIME__authored 0.9956 0.9935 0.9946
DATETIME__range 0.8451 0.9262 0.8838
DATETIME__relative 0.8266 0.7498 0.7863
GENERAL__art-broadcastprogram 0.6538 0.6296 0.6415
GENERAL__art-film 0.8 1.0 0.8889
GENERAL__art-music 0.0 0.0 0.0
GENERAL__art-other 0.625 0.7143 0.6667
GENERAL__art-painting 0.0 0.0 0.0
GENERAL__art-writtenart 0.7373 0.8047 0.7695
GENERAL__building-airport 0.8668 0.9689 0.9150
GENERAL__building-hospital 0.8378 0.9323 0.8826
GENERAL__building-hotel 0.7577 0.8603 0.8057
GENERAL__building-library 0.0 0.0 0.0
GENERAL__building-other 0.7597 0.8409 0.7982
GENERAL__building-restaurant 0.7953 0.8695 0.8307
GENERAL__building-sportsfacility 0.0 0.0 0.0
GENERAL__building-theater 0.6 0.6667 0.6316
GENERAL__event-attack/battle/war/militaryconflict 0.8438 0.9310 0.8852
GENERAL__event-other 0.6019 0.6382 0.6195
GENERAL__event-sportsevent 0.0 0.0 0.0
GENERAL__location-GPE 0.7232 0.7888 0.7546
GENERAL__location-bodiesofwater 0.6724 0.975 0.7959
GENERAL__location-island 0.7455 0.9111 0.8200
GENERAL__location-mountain 0.7436 0.8529 0.7945
GENERAL__location-other 0.7186 0.7793 0.7477
GENERAL__location-park 0.7899 0.8704 0.8282
GENERAL__location-road/railway/highway/transit 0.6325 0.7095 0.6688
GENERAL__organization-company 0.8665 0.8605 0.8635
GENERAL__organization-education 0.8256 0.8608 0.8428
GENERAL__organization-government/governmentagency 0.8344 0.8318 0.8331
GENERAL__organization-media/newspaper 0.6667 0.4 0.5
GENERAL__organization-other 0.7790 0.8105 0.7944
GENERAL__organization-religion 0.6667 0.8 0.7273
GENERAL__organization-showorganization 0.0 0.0 0.0
GENERAL__organization-sportsteam 0.0 0.0 0.0
GENERAL__other-astronomything 0.0 0.0 0.0
GENERAL__other-award 0.8216 0.8859 0.8525
GENERAL__other-biologything 0.7246 0.8961 0.8013
GENERAL__other-chemicalthing 0.7687 0.8047 0.7863
GENERAL__other-currency 0.6304 0.6744 0.6517
GENERAL__other-disease 0.8594 0.9048 0.8815
GENERAL__other-educationaldegree 0.7119 0.75 0.7304
GENERAL__other-god 0.8 0.5714 0.6667
GENERAL__other-language 0.6818 1.0 0.8108
GENERAL__other-law 0.7978 0.8462 0.8212
GENERAL__other-livingthing 0.7385 0.9320 0.8240
GENERAL__other-medical 0.7778 0.8343 0.8050
GENERAL__person-artist/author 0.625 0.3846 0.4762
GENERAL__person-other 0.8839 0.8979 0.8908
GENERAL__person-politician 0.7534 0.7432 0.7483
GENERAL__person-scholar 0.8640 0.8769 0.8704
GENERAL__person-soldier 0.7674 0.7586 0.7630
GENERAL__product-airplane 0.6774 0.6364 0.6562
GENERAL__product-car 0.9286 0.7879 0.8525
GENERAL__product-food 0.7798 0.7859 0.7828
GENERAL__product-game 0.75 0.75 0.75
GENERAL__product-other 0.7175 0.7537 0.7351
GENERAL__product-ship 0.0 0.0 0.0
GENERAL__product-software 0.8093 0.8403 0.8245
GENERAL__product-train 0.75 0.375 0.5
GENERAL__product-weapon 0.7794 0.8833 0.8281

Uses

Direct Use for Inference 

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("YurtsAI/named_entity_recognition_document_context")
# Run inference
entities = model.predict("monday is a chill day – beach time at barceloneta and maybe some shopping at la rambla.")

Downstream Use

You can finetune this model on your own dataset.

Click to expand 
from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("YurtsAI/ner-document-context")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("YurtsAI/named_entity_recognition_document_context-finetuned")

Training Details

Training Set Metrics

Training set Min Median Max
Sentence length 1 14.6796 691
Entities per sentence 0 0.4235 35

Training Hyperparameters

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 3

Training Results

Epoch Step Validation Loss Validation Precision Validation Recall Validation F1 Validation Accuracy
0.0299 500 0.0254 0.5244 0.0116 0.0228 0.9292
0.0597 1000 0.0144 0.5380 0.3492 0.4235 0.9444
0.0896 1500 0.0099 0.7134 0.4410 0.5450 0.9534
0.1194 2000 0.0088 0.6461 0.6571 0.6516 0.9596
0.1493 2500 0.0074 0.7177 0.6363 0.6745 0.9628
0.1791 3000 0.0075 0.6612 0.7342 0.6958 0.9637
0.2090 3500 0.0073 0.6686 0.7286 0.6973 0.9634
0.2388 4000 0.0061 0.7552 0.7044 0.7289 0.9693
0.2687 4500 0.0062 0.7385 0.7150 0.7266 0.9682
0.2986 5000 0.0070 0.6667 0.7792 0.7186 0.9654
0.3284 5500 0.0063 0.6984 0.7774 0.7358 0.9689
0.3583 6000 0.0055 0.7941 0.7023 0.7454 0.9706
0.3881 6500 0.0055 0.7540 0.7640 0.7589 0.9722
0.4180 7000 0.0053 0.7700 0.7614 0.7657 0.9732
0.4478 7500 0.0053 0.7791 0.7698 0.7744 0.9742
0.4777 8000 0.0054 0.7396 0.8062 0.7715 0.9729
0.5075 8500 0.0051 0.7653 0.7944 0.7796 0.9741
0.5374 9000 0.0050 0.7773 0.7844 0.7808 0.9747
0.5672 9500 0.0049 0.7954 0.7711 0.7830 0.9757
0.5971 10000 0.0049 0.7844 0.7876 0.7860 0.9754
0.6270 10500 0.0047 0.7898 0.7940 0.7919 0.9761
0.6568 11000 0.0047 0.7852 0.7929 0.7890 0.9761
0.6867 11500 0.0047 0.8001 0.7908 0.7954 0.9770
0.7165 12000 0.0050 0.7643 0.8145 0.7886 0.9755
0.7464 12500 0.0047 0.7991 0.7892 0.7941 0.9764
0.7762 13000 0.0046 0.7948 0.8084 0.8015 0.9774
0.8061 13500 0.0046 0.7841 0.8154 0.7994 0.9771
0.8359 14000 0.0043 0.8283 0.7776 0.8021 0.9783
0.8658 14500 0.0044 0.8054 0.7993 0.8023 0.9773
0.8957 15000 0.0047 0.7704 0.8152 0.7922 0.9758
0.9255 15500 0.0043 0.8018 0.8149 0.8083 0.9782
0.9554 16000 0.0043 0.8255 0.7938 0.8093 0.9789
0.9852 16500 0.0042 0.8201 0.8008 0.8104 0.9787
1.0151 17000 0.0044 0.7947 0.8175 0.8059 0.9784
1.0449 17500 0.0044 0.7942 0.8195 0.8066 0.9777
1.0748 18000 0.0043 0.8124 0.8110 0.8117 0.9789
1.1046 18500 0.0043 0.7987 0.8157 0.8071 0.9788
1.1345 19000 0.0043 0.8037 0.8171 0.8103 0.9789
1.1644 19500 0.0042 0.8178 0.8076 0.8127 0.9796
1.1942 20000 0.0044 0.7803 0.8389 0.8085 0.9780
1.2241 20500 0.0043 0.8040 0.8210 0.8124 0.9790
1.2539 21000 0.0043 0.8038 0.8245 0.8141 0.9788
1.2838 21500 0.0041 0.8318 0.7973 0.8142 0.9794
1.3136 22000 0.0041 0.8106 0.8211 0.8158 0.9796
1.3435 22500 0.0041 0.8288 0.8046 0.8165 0.9796
1.3733 23000 0.0041 0.8218 0.8170 0.8194 0.9799
1.4032 23500 0.0042 0.8164 0.8171 0.8168 0.9799
1.4330 24000 0.0041 0.8105 0.8248 0.8176 0.9793
1.4629 24500 0.0042 0.8073 0.8196 0.8134 0.9791
1.4928 25000 0.0040 0.8211 0.8162 0.8187 0.9797
1.5226 25500 0.0040 0.8195 0.8225 0.8210 0.9800
1.5525 26000 0.0040 0.8372 0.8018 0.8191 0.9799
1.5823 26500 0.0040 0.8263 0.8161 0.8212 0.9802
1.6122 27000 0.0039 0.8275 0.8141 0.8208 0.9802
1.6420 27500 0.0040 0.8264 0.8198 0.8231 0.9804
1.6719 28000 0.0040 0.8218 0.8195 0.8206 0.9799
1.7017 28500 0.0039 0.8286 0.8195 0.8240 0.9803
1.7316 29000 0.0041 0.8004 0.8357 0.8177 0.9788
1.7615 29500 0.0040 0.8138 0.8304 0.8220 0.9801
1.7913 30000 0.0040 0.8160 0.8309 0.8234 0.9804
1.8212 30500 0.0039 0.8204 0.8262 0.8233 0.9802
1.8510 31000 0.0038 0.8292 0.8228 0.8260 0.9810
1.8809 31500 0.0039 0.8247 0.8246 0.8246 0.9806
1.9107 32000 0.0038 0.8267 0.8258 0.8262 0.9810
1.9406 32500 0.0039 0.8102 0.8398 0.8248 0.9805
1.9704 33000 0.0039 0.8321 0.8185 0.8253 0.9809
2.0003 33500 0.0038 0.8325 0.8261 0.8293 0.9814
2.0302 34000 0.0038 0.8352 0.8228 0.8289 0.9813
2.0600 34500 0.0041 0.8144 0.8369 0.8255 0.9809
2.0899 35000 0.0039 0.8274 0.8281 0.8277 0.9813
2.1197 35500 0.0039 0.8198 0.8353 0.8275 0.9812
2.1496 36000 0.0039 0.8211 0.8358 0.8284 0.9811
2.1794 36500 0.0039 0.8242 0.8300 0.8271 0.9809
2.2093 37000 0.0039 0.8194 0.8317 0.8255 0.9808
2.2391 37500 0.0039 0.8258 0.8344 0.8301 0.9814
2.2690 38000 0.0039 0.8292 0.8302 0.8297 0.9816
2.2989 38500 0.0039 0.8281 0.8315 0.8298 0.9813
2.3287 39000 0.0039 0.8174 0.8386 0.8279 0.9808
2.3586 39500 0.0039 0.8208 0.8364 0.8285 0.9810
2.3884 40000 0.0039 0.8230 0.8379 0.8304 0.9815
2.4183 40500 0.0038 0.8355 0.8273 0.8314 0.9816
2.4481 41000 0.0038 0.8290 0.8347 0.8319 0.9816
2.4780 41500 0.0038 0.8233 0.8403 0.8317 0.9815
2.5078 42000 0.0039 0.8186 0.8417 0.8300 0.9814
2.5377 42500 0.0038 0.8321 0.8343 0.8332 0.9818
2.5675 43000 0.0038 0.8239 0.8396 0.8317 0.9816
2.5974 43500 0.0038 0.8267 0.8378 0.8322 0.9816
2.6273 44000 0.0038 0.8325 0.8343 0.8334 0.9818
2.6571 44500 0.0038 0.8254 0.8399 0.8326 0.9817
2.6870 45000 0.0038 0.8339 0.8338 0.8339 0.9820
2.7168 45500 0.0038 0.8301 0.8381 0.8341 0.9819
2.7467 46000 0.0038 0.8309 0.8371 0.8340 0.9818
2.7765 46500 0.0038 0.8296 0.8377 0.8337 0.9817
2.8064 47000 0.0037 0.8337 0.8349 0.8343 0.9820
2.8362 47500 0.0037 0.8303 0.8387 0.8345 0.9820
2.8661 48000 0.0037 0.8289 0.8401 0.8344 0.9819
2.8960 48500 0.0037 0.8299 0.8400 0.8349 0.9820
2.9258 49000 0.0037 0.8289 0.8401 0.8344 0.9819
2.9557 49500 0.0037 0.8322 0.8380 0.8351 0.9821
2.9855 50000 0.0037 0.8312 0.8384 0.8348 0.9820

Framework Versions

  • Python: 3.11.7
  • SpanMarker: 1.5.0
  • Transformers: 4.42.1
  • PyTorch: 2.1.1+cu121
  • Datasets: 2.14.5
  • Tokenizers: 0.19.1

Citation

BibTeX 

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}
Downloads last month
195
Safetensors
Model size
355M params
Tensor type
F32
·
Inference Examples
Token Classification
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for YurtsAI/ner-document-context

Base model

FacebookAI/roberta-large
Finetuned
(282)
this model

Dataset used to train YurtsAI/ner-document-context

Evaluation results

View on Papers With Code