[Python] Is A PyTorch Dataset Accessed by Multiple DataLoader Workers?

Discussão em 'Python' iniciado por Stack, Dezembro 8, 2024.

  1. Stack

    Stack Membro Participativo

    When using more than 1 DataLoader workers in PyTorch, does every worker access the same Dataset instance? Or does each DataLoader worker have their own instance of Dataset?

    from torch.utils.data import DataLoader, Dataset

    class NumbersDataset(Dataset):
    def __init__(self):
    self.samples = list(range(1, 1001))

    def __len__(self):
    return len(self.samples)

    def __getitem__(self, idx):
    return self.samples[idx]

    dataset = NumbersDataset()
    train_loader = DataLoader(dataset, num_workers=4)

    Continue reading...

Compartilhe esta Página