I am trying to use Pandera to validate at least some of my dataframe. As an example I have one that only contains a date column, and then some columns which I don't really know in advance. At least not the name. However, I know that every column should contain floats or NaN's. So how do I validate that dataframe? For example my DataFrameModel class looks like this: class OutputSchema(pa.DataFrameModel): date: Series[dt.datetime] And then I will have let's say 10 other columns I don't know the name of, but I know that they should contain floats. How would that look like if I for example use the pa.check_types such as: @pa.check_types(lazy=True) def get_data(self) -> DataFrame[OutputSchema]: df = get_df() return df So right now I can have 100 of other columns beside the date column, which are not validated. So in theory they could contain stuff I don't want. But as stated, I know, that every other column should be columns containing floats or NaNs. So how do I make a general validation on columns I don't know in advance ? Continue reading...
