---
license: mit
base_model:
- deepseek-ai/DeepSeek-V3
pipeline_tag: text-generation
library_name: transformers
---
# DeepSeek V3 FP16 Atten NaN
This is a minimal reproduceable sample to let the final layer of DeepSeek V3's attention output NaNs when using data type float16.

Run the `run.py` to see the NaNs.

Weights are converted to bfloat16 using the original float8 e4m3fn, then converted to float16, then extracted from the final layer's attention.