[Python] XPT files values differ from expected values

• 
  I have downloaded a .xpt file from: https://wwwn.cdc.gov/Nchs/Data/Nhanes/Public/2021/DataFiles/DPQ_L.xpt
  
  
• 
  This file comes with a description of what the values should be expected to be: https://wwwn.cdc.gov/Nchs/Data/Nhanes/Public/2021/DataFiles/DPQ_L.htm
  
  
• 
  The data is from a participant filling out a survey of questions.
  
  
• 
  The values should be between 0-9 and "." if missing.
  

I have run the following code to interpret the file:

import pandas as pd

# Reading the .xpt file (SAS Transport file)
df = pd.read_sas('C:\\Users\\wsaj5\\Downloads\\DPQ_L.xpt', format='xport')

# Print the first few rows to inspect the data
print(df.head())

# Check the data types of the columns
print(df.dtypes)


This gives me:

SEQN DPQ010 DPQ020 ... DPQ080 DPQ090 DPQ100
0 130378.0 NaN NaN ... NaN NaN NaN
1 130379.0 5.397605e-79 5.397605e-79 ... 5.397605e-79 5.397605e-79 5.397605e-79
2 130380.0 5.397605e-79 5.397605e-79 ... 5.397605e-79 5.397605e-79 5.397605e-79
3 130386.0 5.397605e-79 5.397605e-79 ... 5.397605e-79 5.397605e-79 5.397605e-79
4 130387.0 5.397605e-79 5.397605e-79 ... 5.397605e-79 5.397605e-79 NaN

[5 rows x 11 columns]
SEQN float64
DPQ010 float64
DPQ020 float64
DPQ030 float64
DPQ040 float64
DPQ050 float64
DPQ060 float64
DPQ070 float64
DPQ080 float64
DPQ090 float64
DPQ100 float64
dtype: object

• 
  5.40e-79 isn't an expected value and I have had this value for other files that I have been looking at from the same source too.
  
  
• 
  Not all values in columns containing 5.40e-79 are 5.40e-79, just some values seem wrong.
  
  
• 
  There are still other values in the same rows for different columns which would suggest that the participant has completed the survey so there shouldn't be answers missing.
  
  
• 
  ChatGPT suggests that this is a placeholder value for missing values.
  
  
• 
  Can these values be ignored? I am worried that I will be ignoring data that I shouldn't be.
  

Continue reading...