[Python] "Value is not Defined" error when trying to use a UDF (that applies a custom...

I am trying to loop through a column, with a custom function I have created, through using a UDF.

I am wanting to change the Exact Time to Time of Day (e.g. Morning, Afternoon, Evening etc.) within the Time Column.

It's probably something simple, but can't make head or tail of it...

I have looked at the following and have tried to make sense, but with no luck:

• How do I fix this "Name is not Defined"?
• "Spark" is not defined - Creating Python UDF

The steps are:

1. 
   I first define the convert_time function.
   
   
2. 
   I then create the UDF.
   
   
3. 
   I then try and use the UDF to apply the custom function.
   
   
4. 
   The error I then get is:
   

PythonException: An exception was thrown from the Python worker. Please see the stack trace below. Traceback (most recent call last):
File "/tmp/ipykernel_8245/3893918262.py", line 16, in convert_time NameError: name 'value' is not defined
​

Here's the code:

from pyspark.sql import SparkSession
from pyspark.sql.functions import udf
from pyspark.sql.types import StringType, IntegerType

# Select the column and collect the values
#column_values = test_df5.select("TOI").collect()

#convert every row from TOI to appropriate Time of Day

def convert_time(hhmm):
print("initiated convert time")
if hhmm == None:
print("The variable is of Nonetype")
# Extract hours and minutes from the input string

elif value.isdigit() == True:

#print("stripping strings")
#int(hhmm.strip())
print(value)

hours = int(hhmm[:2])
minutes = int(hhmm[2:])

# Determine the time of day
if 5 <= hours < 12:
period = "Morning"
elif 12 <= hours < 17:
period = "Afternoon"
elif 17 <= hours < 21:
period = "Evening"
elif hours > 21:
period = "Night"
elif hours < 5:
period = "Night"
else:
period = ""

# Format the time in a readable way
formatted_time = f"{period}"
return formatted_time

else:
return
print("is another type")

print("creating the UDF")

convtime_udf = udf(convert_time, IntegerType())

print("applying the UDF to the TOI Column")

df_with_convtime = test_df5.withColumn("convtime", convtime_udf(test_df5["Time"]))

#df_with_convtime.show()
display(df_with_convtime)


Thanks in advance...

Continue reading...