[Python] Why does the CityLearn tutorial keep throwing me a ValueError?

I am following this CityLearn tutorial.

I got through the first part (RBC) without incident. However, when I implement the second part (Q-learning, literally copy and paste from the site), I keep getting a

ValueError: not enough values to unpack (expected 2, got 1).
​

The bit of code this is on is:

# ----------------- CALCULATE NUMBER OF TRAINING EPISODES -----------------
i = 3
m = tql_env.observation_space[0].n
n = tql_env.action_space[0].n
t = tql_env.time_steps - 1
tql_episodes = m*n*i/t
tql_episodes = int(tql_episodes)
print('Q-Table dimension:', (m, n))
print('Number of episodes to train:', tql_episodes)

# ------------------------------- SET LOADER ------------------------------
loader = get_loader(max=tql_episodes*t)
display(loader)

# ----------------------- SET MODEL HYPERPARAMETERS -----------------------
tql_kwargs = {
'epsilon': 1.0,
'minimum_epsilon': 0.01,
'epsilon_decay': 0.0001,
'learning_rate': 0.005,
'discount_factor': 0.99,
}

# ----------------------- INITIALIZE AND TRAIN MODEL ----------------------
tql_model = CustomTabularQLearning(
env=tql_env,
loader=loader,
random_seed=RANDOM_SEED,
**tql_kwargs
)
_ = tql_model.learn(episodes=tql_episodes)


and the output looks like this:

Q-Table dimension: (24, 1728)
Number of episodes to train: 745

Simulating:

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_35752\2723239734.py in <module>
29 **tql_kwargs
30 )
---> 31 _ = tql_model.learn(episodes=tql_episodes)

~\AppData\Local\anaconda3\envs\citylearn1.8\lib\site-packages\citylearn\agents\base.py in learn(self, episodes, keep_env_history, env_history_directory, deterministic, deterministic_finish, logging_level)
136 for episode in range(episodes):
137 deterministic = deterministic or (deterministic_finish and episode >= episodes - 1)
--> 138 observations = self.env.reset()
139
140 while not self.env.done:

~\AppData\Local\anaconda3\envs\citylearn1.8\lib\site-packages\gym\core.py in reset(self, **kwargs)
321 def reset(self, **kwargs) -> Tuple[ObsType, dict]:
322 """Resets the environment with kwargs."""
--> 323 return self.env.reset(**kwargs)
324
325 def render(

~\AppData\Local\anaconda3\envs\citylearn1.8\lib\site-packages\gym\core.py in reset(self, **kwargs)
321 def reset(self, **kwargs) -> Tuple[ObsType, dict]:
322 """Resets the environment with kwargs."""
--> 323 return self.env.reset(**kwargs)
324
325 def render(

~\AppData\Local\anaconda3\envs\citylearn1.8\lib\site-packages\gym\core.py in reset(self, **kwargs)
321 def reset(self, **kwargs) -> Tuple[ObsType, dict]:
322 """Resets the environment with kwargs."""
--> 323 return self.env.reset(**kwargs)
324
325 def render(

~\AppData\Local\anaconda3\envs\citylearn1.8\lib\site-packages\gym\core.py in reset(self, **kwargs)
377 def reset(self, **kwargs):
378 """Resets the environment, returning a modified observation using :meth:`self.observation`."""
--> 379 obs, info = self.env.reset(**kwargs)
380 return self.observation(obs), info
381

~\AppData\Local\anaconda3\envs\citylearn1.8\lib\site-packages\gym\core.py in reset(self, **kwargs)
377 def reset(self, **kwargs):
378 """Resets the environment, returning a modified observation using :meth:`self.observation`."""
--> 379 obs, info = self.env.reset(**kwargs)
380 return self.observation(obs), info
381

ValueError: not enough values to unpack (expected 2, got 1)


I really do not understand what could possibly be going wrong.



Continue reading...