[Python] How do I write a query like (A or B) and C in Python Polars?

How do I write a query like (A or B) and C in Python Polars?


import polars as pl
import numpy as np

df = pl.DataFrame(
data={
"a": [0.0, 0.0, 0.0, 0.0, np.nan, np.nan, np.nan],
"b": [0.0, 0.0, np.nan, np.nan, 0.0, 0.0, np.nan],
"c": [0.0, np.nan, 0.0, np.nan, 0.0, np.nan, np.nan]
}
)

df.with_columns(
((pl.col('a').is_not_nan() | pl.col('b').is_not_nan())
& pl.col('c').is_not_nan()).alias('Keep'))
df_actual = df.filter(pl.col("Keep") is True)

print("df\n", df)
print("df_expect\n", df_expect)
print("df_actual\n", df_actual)

df
shape: (7, 3)
┌─────┬─────┬─────┐
│ a ┆ b ┆ c │
│ --- ┆ --- ┆ --- │
│ f64 ┆ f64 ┆ f64 │
╞═════╪═════╪═════╡
│ 0.0 ┆ 0.0 ┆ 0.0 │
│ 0.0 ┆ 0.0 ┆ NaN │
│ 0.0 ┆ NaN ┆ 0.0 │
│ 0.0 ┆ NaN ┆ NaN │
│ NaN ┆ 0.0 ┆ 0.0 │
│ NaN ┆ 0.0 ┆ NaN │
│ NaN ┆ NaN ┆ NaN │
└─────┴─────┴─────┘

df_expect
shape: (3, 3)
┌─────┬─────┬─────┐
│ a ┆ b ┆ c │
│ --- ┆ --- ┆ --- │
│ f64 ┆ f64 ┆ f64 │
╞═════╪═════╪═════╡
│ 0.0 ┆ NaN ┆ 0.0 │
│ NaN ┆ 0.0 ┆ 0.0 │
│ 0.0 ┆ 0.0 ┆ 0.0 │
└─────┴─────┴─────┘
df_actual
shape: (0, 3)
┌─────┬─────┬─────┐
│ a ┆ b ┆ c │
│ --- ┆ --- ┆ --- │
│ f64 ┆ f64 ┆ f64 │
╞═════╪═════╪═════╡
└─────┴─────┴─────┘


I expected either a or b would be 0.0 (not NaN) and c would always be 0.0. The Polars documentation said to use | as "or" and & as "and". I believe I have the logic right: (((a not Nan) or (b not Nan)) and (c not NaN)) However, the output is wrong.

Continue reading...