[Python] Combining Beam FlatMap pvalue.AsDict and pvalue.AsList

I have a situation where I want to have call function in beam pipeline with side inputs which will be the lists with some key and it all should resolve to the return list. I started like this:

transformed_output = (
parsed_input_data
| "Merge Data"
>> beam.FlatMap(
process_records,
beam.pvalue.AsList(auxiliary_data),
beam.pvalue.AsDict(
{
"lookup_table_1": beam.pvalue.AsList(lookup_table_data_1),
"lookup_table_2": beam.pvalue.AsList(lookup_table_data_2),
}
),
)
)


And here lookup_table_data is the result of previous pipeline. I am getting such error:

self._window_mapping_fn = sideinputs.default_window_mapping_fn( pcoll.windowing.windowfn) E AttributeError: 'dict' object has no attribute 'windowing'
​

Is it even possible? I want to have as an input anything which will store both list and keys, because I need to use that key later. Also this function will be called mulstiple time with different side inputs parameter.

Continue reading...