Hugging Face's logo
Robert Shaw's picture
16 6 4

Robert Shaw

robertgshaw2
rgreenberg1's profile picture markurtz's profile picture shubhrapandit's profile picture
·

AI & ML interests

None yet

Recent Activity

new activity about 1 month ago
nm-testing/pixtral-12b-w4a16-actorder-group:What is an actorder group and what are the advantages of running this in vLLM?
new activity about 2 months ago
neuralmagic/Sparse-Llama-3.1-8B-2of4:Can I apply a LoRA?
new activity about 2 months ago
nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic:Nice model, any info on scripts used to quantize?
View all activity

Organizations

HeavyAI's profile picture Neural Magic's profile picture Revel Labs's profile picture NM Testing's profile picture Red Hat's profile picture

robertgshaw2's activity

New activity in nm-testing/pixtral-12b-w4a16-actorder-group about 1 month ago

What is an actorder group and what are the advantages of running this in vLLM?

4
#1 opened about 1 month ago by
nickandbro
New activity in neuralmagic/Sparse-Llama-3.1-8B-2of4 about 2 months ago

Can I apply a LoRA?

2
#1 opened about 2 months ago by
RonanMcGovern
New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic about 2 months ago

Nice model, any info on scripts used to quantize?

1
#1 opened about 2 months ago by
RonanMcGovern
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 4 months ago

How to download the model with transformer library

5
#6 opened 4 months ago by
Rick10
New activity in mistralai/Pixtral-12B-2409 4 months ago

Update README.md

3
#25 opened 4 months ago by
robertgshaw2
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 4 months ago

Issue running on vLLM using FP8

2
#3 opened 4 months ago by
ffleandro
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 5 months ago

vllm says the requested model does not exist

2
#1 opened 5 months ago by
shivams101
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 5 months ago

Storage format differs from other w4a16 models

2
#2 opened 5 months ago by
timdettmers
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 5 months ago

Model weights are not loaded

4
#3 opened 5 months ago by
MarvelousMouse
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 7 months ago

Can not be inferenced with vllm openai server

1
#1 opened 7 months ago by
jjqsdq
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-quantized.w4a16 7 months ago

Code example request with vllm

2
#1 opened 7 months ago by
ShiningJazz
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit 7 months ago

4bit quantisation does not reduce vram usage.

1
#2 opened 7 months ago by
fu-man
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 7 months ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

5
#3 opened 7 months ago by
Fertel
New activity in open-llm-leaderboard/open_llm_leaderboard 8 months ago

Reproduction

2
#792 opened 8 months ago by
robertgshaw2
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 9 months ago

Fails to run with nm-vllm

1
#1 opened 9 months ago by
clintonruairi
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_50 10 months ago

Update chart template
#2 opened 10 months ago by
robertgshaw2

Update chart template
#2 opened 10 months ago by
robertgshaw2