[Python] Use Hydra to populate nested dataclass objects

I have a yaml config file that has the following structure

paths:
log: ./runs
data: ./data
data:
downloads: [
{
name: "1",
url: "url_1",
file: "file_1"
},
{
name: "2",
url: "url_2",
file: "file_2"
},
{
name: "3",
url: "url_3",
file: "file_3"
}
]


which I'd like to load into this Config dataclass:

@dataclass
class Config:
paths: Paths
data: Data


where

@dataclass
class Paths:
log: str
data: str


@dataclass
class Download:
name: str
url: str
file: str

def download(self):
print("downloading")


@dataclass
class Data:
downloads: list[Download]


so that each of 'downloads' dictionaries in my config populates a Download dataclass and I can then call the download method.

In my main function I've got something like this:

cs = ConfigStore.instance()
cs.store(name="test_config", node=Config)


@hydra.main(version_base=None, config_path="configs", config_name="config")
def main(cfg: Config):
for downloader in cfg.data.downloads:
downloader.download()


Currently the config structure sorts out the typing but I'm not getting instances of the objects, just dictionaries, and so calling download() raises an error.

I'm still getting my head round how hydra works - is it actually possible to return the populated objects or do I need some intermediate step to create them?

Continue reading...