ModernBERT-base-iob2-20241223160124

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

• eval_loss: 4.5330
• eval_model_preparation_time: 0.0027
• eval_overall_strict_precision: 0.0014
• eval_overall_strict_recall: 0.0001
• eval_overall_strict_f1: 0.0001
• eval_overall_ent_type_precision: 0.0057
• eval_overall_ent_type_recall: 0.0003
• eval_overall_ent_type_f1: 0.0005
• eval_overall_partial_precision: 0.2577
• eval_overall_partial_recall: 0.0130
• eval_overall_partial_f1: 0.0247
• eval_overall_exact_precision: 0.1638
• eval_overall_exact_recall: 0.0082
• eval_overall_exact_f1: 0.0157
• eval_checkOut_strict_precision: 0.0
• eval_checkOut_strict_recall: 0.0
• eval_checkOut_strict_f1: 0
• eval_checkOut_ent_type_precision: 0.0
• eval_checkOut_ent_type_recall: 0.0
• eval_checkOut_ent_type_f1: 0
• eval_checkOut_partial_precision: 0.0030
• eval_checkOut_partial_recall: 0.0031
• eval_checkOut_partial_f1: 0.0030
• eval_checkOut_exact_precision: 0.0022
• eval_checkOut_exact_recall: 0.0023
• eval_checkOut_exact_f1: 0.0023
• eval_bookingNumber_strict_precision: 0.0
• eval_bookingNumber_strict_recall: 0.0
• eval_bookingNumber_strict_f1: 0
• eval_bookingNumber_ent_type_precision: 0.0
• eval_bookingNumber_ent_type_recall: 0.0
• eval_bookingNumber_ent_type_f1: 0
• eval_bookingNumber_partial_precision: 0.0015
• eval_bookingNumber_partial_recall: 0.0067
• eval_bookingNumber_partial_f1: 0.0024
• eval_bookingNumber_exact_precision: 0.0014
• eval_bookingNumber_exact_recall: 0.0063
• eval_bookingNumber_exact_f1: 0.0023
• eval_documentType_strict_precision: 0.0
• eval_documentType_strict_recall: 0.0
• eval_documentType_strict_f1: 0
• eval_documentType_ent_type_precision: 0.0006
• eval_documentType_ent_type_recall: 0.0001
• eval_documentType_ent_type_f1: 0.0001
• eval_documentType_partial_precision: 0.1126
• eval_documentType_partial_recall: 0.0171
• eval_documentType_partial_f1: 0.0297
• eval_documentType_exact_precision: 0.0816
• eval_documentType_exact_recall: 0.0124
• eval_documentType_exact_f1: 0.0215
• eval_companyCountry_strict_precision: 0.0010
• eval_companyCountry_strict_recall: 0.0003
• eval_companyCountry_strict_f1: 0.0005
• eval_companyCountry_ent_type_precision: 0.0014
• eval_companyCountry_ent_type_recall: 0.0005
• eval_companyCountry_ent_type_f1: 0.0008
• eval_companyCountry_partial_precision: 0.0455
• eval_companyCountry_partial_recall: 0.0167
• eval_companyCountry_partial_f1: 0.0245
• eval_companyCountry_exact_precision: 0.0204
• eval_companyCountry_exact_recall: 0.0075
• eval_companyCountry_exact_f1: 0.0110
• eval_hotelName_strict_precision: 0.0
• eval_hotelName_strict_recall: 0.0
• eval_hotelName_strict_f1: 0
• eval_hotelName_ent_type_precision: 0.0
• eval_hotelName_ent_type_recall: 0.0
• eval_hotelName_ent_type_f1: 0
• eval_hotelName_partial_precision: 0.0003
• eval_hotelName_partial_recall: 0.0016
• eval_hotelName_partial_f1: 0.0005
• eval_hotelName_exact_precision: 0.0003
• eval_hotelName_exact_recall: 0.0014
• eval_hotelName_exact_f1: 0.0005
• eval_hotelBankAccount_strict_precision: 0.0
• eval_hotelBankAccount_strict_recall: 0.0
• eval_hotelBankAccount_strict_f1: 0
• eval_hotelBankAccount_ent_type_precision: 0.0
• eval_hotelBankAccount_ent_type_recall: 0.0
• eval_hotelBankAccount_ent_type_f1: 0
• eval_hotelBankAccount_partial_precision: 0.0
• eval_hotelBankAccount_partial_recall: 0.0
• eval_hotelBankAccount_partial_f1: 0
• eval_hotelBankAccount_exact_precision: 0.0
• eval_hotelBankAccount_exact_recall: 0.0
• eval_hotelBankAccount_exact_f1: 0
• eval_hotelAddress_strict_precision: 0.0
• eval_hotelAddress_strict_recall: 0.0
• eval_hotelAddress_strict_f1: 0
• eval_hotelAddress_ent_type_precision: 0.0
• eval_hotelAddress_ent_type_recall: 0.0
• eval_hotelAddress_ent_type_f1: 0
• eval_hotelAddress_partial_precision: 0.0
• eval_hotelAddress_partial_recall: 0.0
• eval_hotelAddress_partial_f1: 0
• eval_hotelAddress_exact_precision: 0.0
• eval_hotelAddress_exact_recall: 0.0
• eval_hotelAddress_exact_f1: 0
• eval_companyZipcode_strict_precision: 0.0
• eval_companyZipcode_strict_recall: 0.0
• eval_companyZipcode_strict_f1: 0
• eval_companyZipcode_ent_type_precision: 0.0
• eval_companyZipcode_ent_type_recall: 0.0
• eval_companyZipcode_ent_type_f1: 0
• eval_companyZipcode_partial_precision: 0.0005
• eval_companyZipcode_partial_recall: 0.0028
• eval_companyZipcode_partial_f1: 0.0008
• eval_companyZipcode_exact_precision: 0.0004
• eval_companyZipcode_exact_recall: 0.0026
• eval_companyZipcode_exact_f1: 0.0007
• eval_companyAddress_strict_precision: 0.0
• eval_companyAddress_strict_recall: 0.0
• eval_companyAddress_strict_f1: 0
• eval_companyAddress_ent_type_precision: 0.0
• eval_companyAddress_ent_type_recall: 0.0
• eval_companyAddress_ent_type_f1: 0
• eval_companyAddress_partial_precision: 0.0001
• eval_companyAddress_partial_recall: 0.0038
• eval_companyAddress_partial_f1: 0.0001
• eval_companyAddress_exact_precision: 0.0001
• eval_companyAddress_exact_recall: 0.0038
• eval_companyAddress_exact_f1: 0.0001
• eval_netAmount_strict_precision: 0.0
• eval_netAmount_strict_recall: 0.0
• eval_netAmount_strict_f1: 0
• eval_netAmount_ent_type_precision: 0.0
• eval_netAmount_ent_type_recall: 0.0
• eval_netAmount_ent_type_f1: 0
• eval_netAmount_partial_precision: 0.0117
• eval_netAmount_partial_recall: 0.0036
• eval_netAmount_partial_f1: 0.0055
• eval_netAmount_exact_precision: 0.0048
• eval_netAmount_exact_recall: 0.0015
• eval_netAmount_exact_f1: 0.0023
• eval_hotelCountry_strict_precision: 0.0
• eval_hotelCountry_strict_recall: 0.0
• eval_hotelCountry_strict_f1: 0
• eval_hotelCountry_ent_type_precision: 0.0
• eval_hotelCountry_ent_type_recall: 0.0
• eval_hotelCountry_ent_type_f1: 0
• eval_hotelCountry_partial_precision: 0.0015
• eval_hotelCountry_partial_recall: 0.0017
• eval_hotelCountry_partial_f1: 0.0016
• eval_hotelCountry_exact_precision: 0.0015
• eval_hotelCountry_exact_recall: 0.0017
• eval_hotelCountry_exact_f1: 0.0016
• eval_cardNumber_strict_precision: 0.0
• eval_cardNumber_strict_recall: 0.0
• eval_cardNumber_strict_f1: 0
• eval_cardNumber_ent_type_precision: 0.0
• eval_cardNumber_ent_type_recall: 0.0
• eval_cardNumber_ent_type_f1: 0
• eval_cardNumber_partial_precision: 0.0020
• eval_cardNumber_partial_recall: 0.0010
• eval_cardNumber_partial_f1: 0.0013
• eval_cardNumber_exact_precision: 0.0020
• eval_cardNumber_exact_recall: 0.0010
• eval_cardNumber_exact_f1: 0.0013
• eval_cardType_strict_precision: 0.0
• eval_cardType_strict_recall: 0.0
• eval_cardType_strict_f1: 0
• eval_cardType_ent_type_precision: 0.0001
• eval_cardType_ent_type_recall: 0.0007
• eval_cardType_ent_type_f1: 0.0002
• eval_cardType_partial_precision: 0.0008
• eval_cardType_partial_recall: 0.0050
• eval_cardType_partial_f1: 0.0014
• eval_cardType_exact_precision: 0.0005
• eval_cardType_exact_recall: 0.0032
• eval_cardType_exact_f1: 0.0009
• eval_grossAmount_strict_precision: 0.0
• eval_grossAmount_strict_recall: 0.0
• eval_grossAmount_strict_f1: 0
• eval_grossAmount_ent_type_precision: 0.0
• eval_grossAmount_ent_type_recall: 0.0
• eval_grossAmount_ent_type_f1: 0
• eval_grossAmount_partial_precision: 0.0001
• eval_grossAmount_partial_recall: 0.0014
• eval_grossAmount_partial_f1: 0.0001
• eval_grossAmount_exact_precision: 0.0001
• eval_grossAmount_exact_recall: 0.0014
• eval_grossAmount_exact_f1: 0.0001
• eval_reservationNumber_strict_precision: 0.0
• eval_reservationNumber_strict_recall: 0.0
• eval_reservationNumber_strict_f1: 0
• eval_reservationNumber_ent_type_precision: 0.0
• eval_reservationNumber_ent_type_recall: 0.0
• eval_reservationNumber_ent_type_f1: 0
• eval_reservationNumber_partial_precision: 0.0008
• eval_reservationNumber_partial_recall: 0.0045
• eval_reservationNumber_partial_f1: 0.0014
• eval_reservationNumber_exact_precision: 0.0008
• eval_reservationNumber_exact_recall: 0.0045
• eval_reservationNumber_exact_f1: 0.0014
• eval_invoiceNumber_strict_precision: 0.0007
• eval_invoiceNumber_strict_recall: 0.0002
• eval_invoiceNumber_strict_f1: 0.0003
• eval_invoiceNumber_ent_type_precision: 0.0011
• eval_invoiceNumber_ent_type_recall: 0.0003
• eval_invoiceNumber_ent_type_f1: 0.0005
• eval_invoiceNumber_partial_precision: 0.0461
• eval_invoiceNumber_partial_recall: 0.0135
• eval_invoiceNumber_partial_f1: 0.0209
• eval_invoiceNumber_exact_precision: 0.0118
• eval_invoiceNumber_exact_recall: 0.0035
• eval_invoiceNumber_exact_f1: 0.0054
• eval_hotelVATNumber_strict_precision: 0.0001
• eval_hotelVATNumber_strict_recall: 0.0005
• eval_hotelVATNumber_strict_f1: 0.0001
• eval_hotelVATNumber_ent_type_precision: 0.0001
• eval_hotelVATNumber_ent_type_recall: 0.0005
• eval_hotelVATNumber_ent_type_f1: 0.0001
• eval_hotelVATNumber_partial_precision: 0.0004
• eval_hotelVATNumber_partial_recall: 0.0035
• eval_hotelVATNumber_partial_f1: 0.0008
• eval_hotelVATNumber_exact_precision: 0.0004
• eval_hotelVATNumber_exact_recall: 0.0035
• eval_hotelVATNumber_exact_f1: 0.0008
• eval_externalReservationNumber_strict_precision: 0.0
• eval_externalReservationNumber_strict_recall: 0.0
• eval_externalReservationNumber_strict_f1: 0
• eval_externalReservationNumber_ent_type_precision: 0.0
• eval_externalReservationNumber_ent_type_recall: 0.0
• eval_externalReservationNumber_ent_type_f1: 0
• eval_externalReservationNumber_partial_precision: 0.0000
• eval_externalReservationNumber_partial_recall: 0.0012
• eval_externalReservationNumber_partial_f1: 0.0001
• eval_externalReservationNumber_exact_precision: 0.0
• eval_externalReservationNumber_exact_recall: 0.0
• eval_externalReservationNumber_exact_f1: 0
• eval_hotelFaxNumber_strict_precision: 0.0
• eval_hotelFaxNumber_strict_recall: 0.0
• eval_hotelFaxNumber_strict_f1: 0
• eval_hotelFaxNumber_ent_type_precision: 0.0
• eval_hotelFaxNumber_ent_type_recall: 0.0
• eval_hotelFaxNumber_ent_type_f1: 0
• eval_hotelFaxNumber_partial_precision: 0.0001
• eval_hotelFaxNumber_partial_recall: 0.0031
• eval_hotelFaxNumber_partial_f1: 0.0001
• eval_hotelFaxNumber_exact_precision: 0.0
• eval_hotelFaxNumber_exact_recall: 0.0
• eval_hotelFaxNumber_exact_f1: 0
• eval_roomNo_strict_precision: 0.0001
• eval_roomNo_strict_recall: 0.0017
• eval_roomNo_strict_f1: 0.0001
• eval_roomNo_ent_type_precision: 0.0001
• eval_roomNo_ent_type_recall: 0.0017
• eval_roomNo_ent_type_f1: 0.0001
• eval_roomNo_partial_precision: 0.0001
• eval_roomNo_partial_recall: 0.0025
• eval_roomNo_partial_f1: 0.0002
• eval_roomNo_exact_precision: 0.0001
• eval_roomNo_exact_recall: 0.0017
• eval_roomNo_exact_f1: 0.0001
• eval_companyName_strict_precision: 0.0002
• eval_companyName_strict_recall: 0.0001
• eval_companyName_strict_f1: 0.0001
• eval_companyName_ent_type_precision: 0.0040
• eval_companyName_ent_type_recall: 0.0017
• eval_companyName_ent_type_f1: 0.0024
• eval_companyName_partial_precision: 0.0422
• eval_companyName_partial_recall: 0.0179
• eval_companyName_partial_f1: 0.0252
• eval_companyName_exact_precision: 0.0244
• eval_companyName_exact_recall: 0.0104
• eval_companyName_exact_f1: 0.0145
• eval_hotelEmail_strict_precision: 0.0
• eval_hotelEmail_strict_recall: 0.0
• eval_hotelEmail_strict_f1: 0
• eval_hotelEmail_ent_type_precision: 0.0
• eval_hotelEmail_ent_type_recall: 0.0
• eval_hotelEmail_ent_type_f1: 0
• eval_hotelEmail_partial_precision: 0.0024
• eval_hotelEmail_partial_recall: 0.0199
• eval_hotelEmail_partial_f1: 0.0043
• eval_hotelEmail_exact_precision: 0.0024
• eval_hotelEmail_exact_recall: 0.0199
• eval_hotelEmail_exact_f1: 0.0043
• eval_companyVATNumber_strict_precision: 0.0
• eval_companyVATNumber_strict_recall: 0.0
• eval_companyVATNumber_strict_f1: 0
• eval_companyVATNumber_ent_type_precision: 0.0
• eval_companyVATNumber_ent_type_recall: 0.0
• eval_companyVATNumber_ent_type_f1: 0
• eval_companyVATNumber_partial_precision: 0.0020
• eval_companyVATNumber_partial_recall: 0.0020
• eval_companyVATNumber_partial_f1: 0.0020
• eval_companyVATNumber_exact_precision: 0.0018
• eval_companyVATNumber_exact_recall: 0.0017
• eval_companyVATNumber_exact_f1: 0.0018
• eval_invoiceDate_strict_precision: 0.0
• eval_invoiceDate_strict_recall: 0.0
• eval_invoiceDate_strict_f1: 0
• eval_invoiceDate_ent_type_precision: 0.0
• eval_invoiceDate_ent_type_recall: 0.0
• eval_invoiceDate_ent_type_f1: 0
• eval_invoiceDate_partial_precision: 0.0058
• eval_invoiceDate_partial_recall: 0.0127
• eval_invoiceDate_partial_f1: 0.0080
• eval_invoiceDate_exact_precision: 0.0035
• eval_invoiceDate_exact_recall: 0.0077
• eval_invoiceDate_exact_f1: 0.0048
• eval_companyCity_strict_precision: 0.0
• eval_companyCity_strict_recall: 0.0
• eval_companyCity_strict_f1: 0
• eval_companyCity_ent_type_precision: 0.0
• eval_companyCity_ent_type_recall: 0.0
• eval_companyCity_ent_type_f1: 0
• eval_companyCity_partial_precision: 0.0012
• eval_companyCity_partial_recall: 0.0053
• eval_companyCity_partial_f1: 0.0020
• eval_companyCity_exact_precision: 0.0010
• eval_companyCity_exact_recall: 0.0044
• eval_companyCity_exact_f1: 0.0017
• eval_hotelPhoneNumber_strict_precision: 0.0
• eval_hotelPhoneNumber_strict_recall: 0.0
• eval_hotelPhoneNumber_strict_f1: 0
• eval_hotelPhoneNumber_ent_type_precision: 0.0
• eval_hotelPhoneNumber_ent_type_recall: 0.0
• eval_hotelPhoneNumber_ent_type_f1: 0
• eval_hotelPhoneNumber_partial_precision: 0.0006
• eval_hotelPhoneNumber_partial_recall: 0.0047
• eval_hotelPhoneNumber_partial_f1: 0.0011
• eval_hotelPhoneNumber_exact_precision: 0.0004
• eval_hotelPhoneNumber_exact_recall: 0.0031
• eval_hotelPhoneNumber_exact_f1: 0.0007
• eval_hotelTaxCode_strict_precision: 0.0
• eval_hotelTaxCode_strict_recall: 0.0
• eval_hotelTaxCode_strict_f1: 0
• eval_hotelTaxCode_ent_type_precision: 0.0
• eval_hotelTaxCode_ent_type_recall: 0.0
• eval_hotelTaxCode_ent_type_f1: 0
• eval_hotelTaxCode_partial_precision: 0.0007
• eval_hotelTaxCode_partial_recall: 0.0019
• eval_hotelTaxCode_partial_f1: 0.0010
• eval_hotelTaxCode_exact_precision: 0.0007
• eval_hotelTaxCode_exact_recall: 0.0019
• eval_hotelTaxCode_exact_f1: 0.0010
• eval_travellerName_strict_precision: 0.0
• eval_travellerName_strict_recall: 0.0
• eval_travellerName_strict_f1: 0
• eval_travellerName_ent_type_precision: 0.0
• eval_travellerName_ent_type_recall: 0.0
• eval_travellerName_ent_type_f1: 0
• eval_travellerName_partial_precision: 0.0010
• eval_travellerName_partial_recall: 0.0157
• eval_travellerName_partial_f1: 0.0019
• eval_travellerName_exact_precision: 0.0010
• eval_travellerName_exact_recall: 0.0157
• eval_travellerName_exact_f1: 0.0019
• eval_hotelCity_strict_precision: 0.0
• eval_hotelCity_strict_recall: 0.0
• eval_hotelCity_strict_f1: 0
• eval_hotelCity_ent_type_precision: 0.0001
• eval_hotelCity_ent_type_recall: 0.0002
• eval_hotelCity_ent_type_f1: 0.0001
• eval_hotelCity_partial_precision: 0.0044
• eval_hotelCity_partial_recall: 0.0140
• eval_hotelCity_partial_f1: 0.0067
• eval_hotelCity_exact_precision: 0.0035
• eval_hotelCity_exact_recall: 0.0111
• eval_hotelCity_exact_f1: 0.0053
• eval_checkIn_strict_precision: 0.0
• eval_checkIn_strict_recall: 0.0
• eval_checkIn_strict_f1: 0
• eval_checkIn_ent_type_precision: 0.0
• eval_checkIn_ent_type_recall: 0.0
• eval_checkIn_ent_type_f1: 0
• eval_checkIn_partial_precision: 0.0002
• eval_checkIn_partial_recall: 0.0009
• eval_checkIn_partial_f1: 0.0003
• eval_checkIn_exact_precision: 0.0001
• eval_checkIn_exact_recall: 0.0004
• eval_checkIn_exact_f1: 0.0001
• eval_currencyCode_strict_precision: 0.0
• eval_currencyCode_strict_recall: 0.0
• eval_currencyCode_strict_f1: 0
• eval_currencyCode_ent_type_precision: 0.0
• eval_currencyCode_ent_type_recall: 0.0
• eval_currencyCode_ent_type_f1: 0
• eval_currencyCode_partial_precision: 0.0001
• eval_currencyCode_partial_recall: 0.0018
• eval_currencyCode_partial_f1: 0.0002
• eval_currencyCode_exact_precision: 0.0
• eval_currencyCode_exact_recall: 0.0
• eval_currencyCode_exact_f1: 0
• eval_pageNumber_strict_precision: 0.0
• eval_pageNumber_strict_recall: 0.0
• eval_pageNumber_strict_f1: 0
• eval_pageNumber_ent_type_precision: 0.0
• eval_pageNumber_ent_type_recall: 0.0
• eval_pageNumber_ent_type_f1: 0
• eval_pageNumber_partial_precision: 0.0005
• eval_pageNumber_partial_recall: 0.0070
• eval_pageNumber_partial_f1: 0.0009
• eval_pageNumber_exact_precision: 0.0005
• eval_pageNumber_exact_recall: 0.0070
• eval_pageNumber_exact_f1: 0.0009
• eval_hotelZipCode_strict_precision: 0.0
• eval_hotelZipCode_strict_recall: 0.0
• eval_hotelZipCode_strict_f1: 0
• eval_hotelZipCode_ent_type_precision: 0.0
• eval_hotelZipCode_ent_type_recall: 0.0
• eval_hotelZipCode_ent_type_f1: 0
• eval_hotelZipCode_partial_precision: 0.0008
• eval_hotelZipCode_partial_recall: 0.0039
• eval_hotelZipCode_partial_f1: 0.0013
• eval_hotelZipCode_exact_precision: 0.0006
• eval_hotelZipCode_exact_recall: 0.0031
• eval_hotelZipCode_exact_f1: 0.0010
• eval_taxAmount_strict_precision: 0.0
• eval_taxAmount_strict_recall: 0.0
• eval_taxAmount_strict_f1: 0
• eval_taxAmount_ent_type_precision: 0.0008
• eval_taxAmount_ent_type_recall: 0.0004
• eval_taxAmount_ent_type_f1: 0.0005
• eval_taxAmount_partial_precision: 0.0723
• eval_taxAmount_partial_recall: 0.0353
• eval_taxAmount_partial_f1: 0.0474
• eval_taxAmount_exact_precision: 0.0608
• eval_taxAmount_exact_recall: 0.0296
• eval_taxAmount_exact_f1: 0.0399
• eval_runtime: 24.6705
• eval_samples_per_second: 40.413
• eval_steps_per_second: 1.297
• step: 0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 512
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.5
• num_epochs: 8

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.3.1
• Datasets 3.2.0
• Tokenizers 0.21.0