[Python] Polars: Select first (or any) non-null struct column value row-wise

How can we select first (or any) non-null struct from polars dataframe to new column?

Input data:

structs_a = [
{"a_key_1": 1, "a_key_2": "a" },
{"a_key_1": None, "a_key_2": None }
]
structs_b = [
{"b_key_1": None, "b_key_2": None },
{"b_key_1": "b", "b_key_2": 2 }
]
df = pl.DataFrame([structs_a, structs_b])
df


┌─────────────┬─────────────┐
│ column_0 ┆ column_1 │
│ --- ┆ --- │
│ struct[2] ┆ struct[2] │
╞═════════════╪═════════════╡
│ {1,"a"} ┆ {null,null} │
│ {null,null} ┆ {"b",2} │
└─────────────┴─────────────┘


What I've tried so far: Concatenating values to list and dropping nulls, but for reasons I yet have to understand this results in list of struct[4]

df.with_columns(
pl.concat_list(
"column_0",
"column_1",
).list.drop_nulls().alias('final_column')
)


My result:

shape: (2, 3)
┌─────────────┬─────────────┬─────────────────────────────────┐
│ column_0 ┆ column_1 ┆ final_column │
│ --- ┆ --- ┆ --- │
│ struct[2] ┆ struct[2] ┆ list[struct[4]] │
╞═════════════╪═════════════╪═════════════════════════════════╡
│ {1,"a"} ┆ {null,null} ┆ [{1,"a",null,null}, {null,null… │
│ {null,null} ┆ {"b",2} ┆ [{null,null,null,null}, {null,… │
└─────────────┴─────────────┴─────────────────────────────────┘


Desirable result:

┌─────────────┬─────────────┬─────────────────┐
│ column_0 ┆ column_1 ┆ final_column │
│ --- ┆ --- ┆ --- │
│ struct[2] ┆ struct[2] ┆ struct[2] │
╞═════════════╪═════════════╪═════════════════╡
│ {1,"a"} ┆ {null,null} ┆ {1,"a"} │ <- Non-null value from column_0
│ {null,null} ┆ {"b",2} ┆ {"b",2} │ <- Non-null value from column_1
└─────────────┴─────────────┴─────────────────┘


UPD: Probably should experiment with coalesce but direct implementation did not result as desired:

df.with_columns(
pl.coalesce(
"column_0",
"column_1",
).alias('final_column')
)


shape: (2, 3)
┌─────────────┬─────────────┬───────────────────────┐
│ column_0 ┆ column_1 ┆ final_column │
│ --- ┆ --- ┆ --- │
│ struct[2] ┆ struct[2] ┆ struct[4] │
╞═════════════╪═════════════╪═══════════════════════╡
│ {1,"a"} ┆ {null,null} ┆ {1,"a",null,null} │
│ {null,null} ┆ {"b",2} ┆ {null,null,null,null} │
└─────────────┴─────────────┴───────────────────────┘

Continue reading...