[Python] Check units of pandas DateTime index with pandera

I am new to pandera and am still learning how it works. What is the easiest way to check that the datetime units of an index are in nanoseconds and not milliseconds? If the solution attempt 2 is the best way of doing this, I will be happy with this answer as I am looking for a more experienced perspective.

Solution Attempt 1


First I tried the way that looked as intuitive after studying the docs, but this did not produce the desired result. The index does not cause a schema error.

import pandas as pd
import pandera as pa
from pandera import DataFrameModel, Field
from pandera.typing import Index
from pandera.engines import pandas_engine

class DateIndexSchema(DataFrameModel):
date: Index[pandas_engine.DateTime] = Field(nullable=False, dtype_kwargs={'unit': 'ns'})

df_wrong_index_type = pd.DataFrame(
{'value': [100, 200]},
index=pd.to_datetime(['2023-01-01', '2023-01-02']).astype('datetime64[ms]'),
)

DateIndexSchema.validate(df_wrong_index_type)

Solution Attempt 2


This solution works as expected, but it feels a bit verbose and makes me feel that I am missing something obvious.

class DateIndexSchemaThrow(DataFrameModel):
date: Index[pandas_engine.DateTime] = Field(nullable=False)

@pa.dataframe_check
def index_should_be_in_ns(cls, dataframe: pd.DataFrame) -> bool:
if dataframe.index.dtype != "datetime64[ns]":
return False
return True

Continue reading...