[Python] OSError: We tried to initialize torch.distributed for you, but it failed. Make sure...

I tried to set up project https://github.com/huggingface/gpt-oss-recipes#installation

error

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

(.venv) PS C:\github\gpt-oss-recipes> uv venv gpt-oss --python 3.11 && source gpt-oss/bin/activate && uv pip install --upgrade pip
At line:1 char:31
+ uv venv gpt-oss --python 3.11 && source gpt-oss/bin/activate && uv pi ...
+ ~~
The token '&&' is not a valid statement separator in this version.
At line:1 char:62
+ ... venv gpt-oss --python 3.11 && source gpt-oss/bin/activate && uv pip ...
+ ~~
The token '&&' is not a valid statement separator in this version.
+ CategoryInfo : ParserError: ) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : InvalidEndOfLine

(.venv) PS C:\github\gpt-oss-recipes> uv pip install torch==2.8.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu128
Resolved 13 packages in 5.66s
Prepared 3 packages in 3m 56s
Uninstalled 1 package in 2.26s
Installed 3 packages in 3.65s
- torch==2.8.0
+ torch==2.8.0+cu128
+ torchaudio==2.8.0+cu128
+ torchvision==0.23.0+cu128
(.venv) PS C:\github\gpt-oss-recipes> uv pip install git+https://github.com/triton-lang/triton.git@main#subdirectory=python/triton_kernels
Updated https://github.com/triton-lang/triton.git (31baa6d284b6a52ba464bfc260bc9395e66a4ac8)
Resolved 11 packages in 9.75s
Built triton-kernels @ git+https://github.com/triton-lang/triton.git@31baa6d284b6a52ba464bfc260bc9395e66a4ac8#subdirectory=python/triton_kernels
Prepared 4 packages in 1.17s
Installed 4 packages in 26ms
+ iniconfig==2.1.0
+ pluggy==1.6.0
+ pytest==8.4.1
+ triton-kernels==1.0.0 (from git+https://github.com/triton-lang/triton.git@31baa6d284b6a52ba464bfc260bc9395e66a4ac8#subdirectory=python/triton_kernels)
(.venv) PS C:\github\gpt-oss-recipes> uv pip install -r requirements.txt
Audited 5 packages in 19ms
(.venv) PS C:\github\gpt-oss-recipes> python .\generate_all.py
chat_template.jinja: 16.7kB [00:00, ?B/s]
`torch_dtype` is deprecated! Use `dtype` instead!
Traceback (most recent call last):
File "C:\github\gpt-oss-recipes\.venv\lib\site-packages\transformers\integrations\tensor_parallel.py", line 57, in initialize_tensor_parallelism
rank = int(os.environ["RANK"])
File "C:\Users\ADMIN\AppData\Roaming\uv\python\cpython-3.10.16-windows-x86_64-none\lib\os.py", line 680, in __getitem__
raise KeyError(key) from None
KeyError: 'RANK'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\github\gpt-oss-recipes\generate_all.py", line 33, in <module>
model = AutoModelForCausalLM.from_pretrained(
File "C:\github\gpt-oss-recipes\.venv\lib\site-packages\transformers\models\auto\auto_factory.py", line 604, in from_pretrained
return model_class.from_pretrained(
File "C:\github\gpt-oss-recipes\.venv\lib\site-packages\transformers\modeling_utils.py", line 288, in _wrapper
return func(*args, **kwargs)
File "C:\github\gpt-oss-recipes\.venv\lib\site-packages\transformers\modeling_utils.py", line 4822, in from_pretrained
tp_plan, device_map, device_mesh, tp_size = initialize_tensor_parallelism(tp_plan, tp_size=tp_size)
File "C:\github\gpt-oss-recipes\.venv\lib\site-packages\transformers\integrations\tensor_parallel.py", line 74, in initialize_tensor_parallelism
raise OSError(
OSError: We tried to initialize torch.distributed for you, but it failed. Make sure you init torch distributed in your script to use `tp_plan='auto'`.
(.venv) PS C:\github\gpt-oss-recipes>


How to fix the error?

Continue reading...