Does sglang support --load_format sharded_state #3197
Unanswered
wedu-nvidia
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am currently deploying DeepSeek R1 bf16 with 4 nodes, but the model's loading time is extremely slow, taking approximately 1.5 hours.
Does SGLang support the --load_format sharded_state option, similar to the VLLM framework?
Beta Was this translation helpful? Give feedback.
All reactions