[Python] how to convert the llama4scout to tensorrt-llm checkpoints to server with the...

i am trying to get the checkpoints for the llama4scout-scout-17B-16E model i am constantly getting the following error. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/models/llama/model.py", line 505, in from_hugging_face loader.generate_tllm_weights(model, arg_dict) File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/models/model_weights_loader.py", line 402, in generate_tllm_weights self.fill(tllm_weights) File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/models/model_weights_loader.py", line 366, in fill w_shape = weights[tllm_key].shape ^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'shape' also , the below error (trtllm-venv) root@XXXXXX:/ephemeral# python3 TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py --model_dir $HF_LLAMA_MODEL --output_dir $UNIFIED_CKPT_PATH --tp_size 2 --pp_size 1 --moe_num_experts 16 --moe_top_k 2 --dtype float16 /ephemeral/trtllm-venv/lib/python3.12/site-packages/torch/cuda/init.py:61: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. import pynvml # type: ignore[import] [2025-09-16 18:57:09] INFO config.py:54: PyTorch version 2.7.1 available. 2025-09-16 18:57:12,626 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend /ephemeral/trtllm-venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST']. warnings.warn( [TensorRT-LLM] TensorRT-LLM version: 0.21.0 0.21.0 [09/16/2025-18:57:12] [TRT-LLM] [W] Implicitly setting LLaMAConfig.has_partial_lora_mask = False [09/16/2025-18:57:12] [TRT-LLM] [W] Implicitly setting LLaMAConfig.tie_word_embeddings = False 7it [00:00, 79.03it/s] Traceback (most recent call last): File "/ephemeral/TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py", line 591, in main() File "/ephemeral/TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py", line 583, in main convert_and_save_hf(args) File "/ephemeral/TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py", line 524, in convert_and_save_hf execute(args.workers, [convert_and_save_rank] * world_size, args) File "/ephemeral/TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py", line 531, in execute f(args, rank) File "/ephemeral/TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py", line 504, in convert_and_save_rank llama = LLaMAForCausalLM.from_hugging_face( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/models/llama/model.py", line 505, in from_hugging_face loader.generate_tllm_weights(model, arg_dict) File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/models/model_weights_loader.py", line 400, in generate_tllm_weights self.load(tllm_key, File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/models/model_weights_loader.py", line 311, in load v = sub_module.postprocess(tllm_key, v, **postprocess_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ephemeral/trtllm-venv/lib/python3.12/site-packages/tensorrt_llm/layers/linear.py", line 435, in postprocess weights = weights.to(str_dtype_to_torch(self.dtype)) ^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'to' with the following command as i am trying to work on the model parallelism python3 TensorRT-LLM/examples/models/core/llama/convert_checkpoint.py --model_dir $HF_LLAMA_MODEL --output_dir $UNIFIED_CKPT_PATH --tp_size 2 --pp_size 1 --moe_num_experts 16 --moe_top_k 2 --dtype float16

Pleaase let me know if you have faced the similar issue and your suggestions, help is invaluable thank you!

i have tried making updating the tensorrt-llm to new version but the issue is same

Continue reading...