[Python] FB-Hydra: calling compose during hydra main is not properly finding configs in the...

I'm trying to call compose within hydra.main, but it is not properly using the searchpath I set in my main config. This is the organization of my script:

my_script
┣ config
┃ ┣ config.yaml
┗ my_script.py


My scripts depend on a config package within a library I'm developing, It is organized as follows:

my_lib
┣ config
┃ ┣ config_group_A
┃ ┃ ┣ ...
┃ ┣ config_group_B
┃ ┃ ┣ some_config.yaml
| | ┗ __init__.py
| ┗ __init__.py


and so on.

In my my_script/config/config.yaml I added the following:

searchpath:
- pkg://my_lib.config
- pkg://my_lib.config.config_group_B


The purpose is that the main config uses a lot of configs within my_lib/config, but also I want to, within hydra.main, use hydra.compose to compose the config some_config.yaml

The problem is that when I call hydra.compose within hydra.main, I get the following error:

hydra.errors.MissingConfigException: Cannot find primary config 'some_config'. Check that it's in your config search path.

Config search path:
provider=hydra, path=pkg://hydra.conf
provider=main, path=file:///<path_to_scripts>/my_script/config
provider=schema, path=structured://


But I'm confused because I added both my_lib.config and my_lib.config.config_group_B to the searchpath. These even appear in the hydra.core.hydra_config.HydraConfig.get().searchpath and hydra.core.hydra_config.HydraConfig.get().runtime.config_sources, so it is strange I cannot compose some_config.yaml during hydra.main. For refernece, I call in the script hydra.main(version_base="1.3", config_path='config', config_name='config')

What is the issue here?

Continue reading...