lxzcpro/demo_cls_T5

Model Card for Model-demo-1.2B

This model is used for a demo classification task

Task type

Protein-level Classification

Model input type

AA Sequence

Label meanings

0: A

1: B

LoRA config

• r: 8
• lora_dropout: 0.0
• lora_alpha: 16
• target_modules: ['SelfAttention.o', 'SelfAttention.k', 'EncDecAttention.o', 'EncDecAttention.v', 'SelfAttention.q', 'EncDecAttention.k', 'EncDecAttention.q', 'SelfAttention.v']
• modules_to_save: ['classifier']

Training config

• optimizer:
  • class: AdamW
  • betas: (0.9, 0.98)
  • weight_decay: 0.01
• learning rate: 0.001
• epoch: 1
• batch size: 4
• precision: 16-mixed