LLäMmlein 1B CoreML
This repository contains the CoreML version of LLäMmlein 1B, a German language model trained from scratch using the Tinyllama codebase on the German portion of RedPajama V2.
Model Details
- Model Type: German Language Model based on TinyLlama architecture
- Language: German
- Framework: CoreML
- Original Model: LSX-UniWue/LLaMmlein_1B
- Size: 1B parameters
- Format: CoreML (.mlpackage)
- Minimum Deployment Target: iOS 16
- Compute Units: ALL (CPU + Neural Engine)
- Input Sequence Length: 512 tokens
Conversion Process
The model was converted from PyTorch to CoreML using the following steps:
import torch
import numpy as np
from transformers import AutoModelForCausalLM, AutoTokenizer
import coremltools as ct
# Load model and convert to TorchScript
model = AutoModelForCausalLM.from_pretrained("LSX-UniWue/LLaMmlein_1B")
tokenizer = AutoTokenizer.from_pretrained("LSX-UniWue/LLaMmlein_1B")
# Set model to eval mode
model.eval()
# Create example input
text = "Ein Beispieltext"
inputs = tokenizer(text, return_tensors="pt")
# Create a wrapper class for tracing
class ModelWrapper(torch.nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, input_ids):
return self.model(input_ids).logits
# Wrap and trace model
wrapped_model = ModelWrapper(model)
traced_model = torch.jit.trace(wrapped_model, inputs.input_ids)
# Convert to CoreML
model_mlpackage = ct.convert(
traced_model,
inputs=[
ct.TensorType(
name="input_ids",
shape=inputs.input_ids.shape,
dtype=np.int32
)
],
source="pytorch",
minimum_deployment_target=ct.target.iOS16,
convert_to="mlprogram",
compute_precision=ct.precision.FLOAT16,
compute_units=ct.ComputeUnit.ALL,
)
model_mlpackage.save("LLaMmlein_1B.mlpackage")
Usage
To use this model on Apple devices:
import CoreML
// Load the model
let config = MLModelConfiguration()
let model = try LLaMmlein_1B(configuration: config)
// Prepare input
let inputIds = // Your tokenized input as [Int32]
// Make prediction
let prediction = try model.prediction(input_ids: inputIds)
Performance Considerations
- The model is optimized for Apple Neural Engine
- Recommended for iOS 16+ devices
- Best performance achieved with batch size of 1
- Maximum sequence length is set to 512 tokens
Original Model Information
The original model was trained on the German portion of RedPajama V2. For more details about the base model:
- Visit the project page
- Read the research paper
- Check the SuperGLEBer benchmark for evaluation results
License
This model inherits its license from the original LLäMmlein 1B model.
Citation
If you use this model, please cite the original work:
@misc{llammlein2024,
title={LLäMmlein: A German Language Model},
author={LSX-UniWue},
year={2024},
publisher={Hugging Face},
journal={Hugging Face Hub},
howpublished={\url{https://huggingface.co./LSX-UniWue/LLaMmlein_1B}},
}
For the original model description and evaluation results, see the original model card.
- Downloads last month
- 3
Inference API (serverless) does not yet support coremltools models for this pipeline type.