I have a table like the following df +------------------------------------+-----------------------+ |identifier |timestamp | +------------------------------------+-----------------------+ |86311425-0890-40a5-8950-54cbaaa60815|2020-03-18 14:41:55 UTC| |38e121a8-f21f-4d10-bb69-26eb045175b5|2020-03-13 15:19:21 UTC| |1a69c9b0-283b-4b6d-89ac-66f987280c66|2020-03-16 12:59:51 UTC| |c7b5c53f-bf40-498f-8302-4b3329322bc9|2020-03-18 22:05:06 UTC| |0d3d807b-9b3a-466e-907c-c22402240730|2020-03-17 18:40:03 UTC| +------------------------------------+-----------------------+ tmp.printSchema() root |-- identifier: string (nullable = true) |-- timestamp: string (nullable = true) I would like to have a column that take only the day and the hours from the timestamp. I am trying the following: from pyspark.sql.functions import hour df = df.withColumn("hour", hour(col("timestamp"))) but I get the following +--------------------+--------------------+----+ | identifier| timestamp|hour| +--------------------+--------------------+----+ |321869c3-71e5-41d...|2020-03-19 03:34:...|null| |226b8d50-2c6a-471...|2020-03-19 02:59:...|null| |47818b7c-34b5-43c...|2020-03-19 01:41:...|null| |f5ca5599-7252-49d...|2020-03-19 04:25:...|null| |add2ae24-aa7b-4d3...|2020-03-19 01:50:...|null| +--------------------+--------------------+----+ while I would like to have +--------------------+--------------------+-------------------+ | identifier| timestamp|hour | +--------------------+--------------------+-------------------+ |321869c3-71e5-41d...|2020-03-19 03:00:...|2020-03-19 03:00:00| |226b8d50-2c6a-471...|2020-03-19 02:59:...|2020-03-19 02:00:00| |47818b7c-34b5-43c...|2020-03-19 01:41:...|2020-03-19 01:00:00| |f5ca5599-7252-49d...|2020-03-19 04:25:...|2020-03-19 04:00:00| |add2ae24-aa7b-4d3...|2020-03-19 01:50:...|2020-03-19 01:00:00| +--------------------+--------------------+-------------------+ Continue reading...