[Python] Fine Tuning OpenAI API Validation error. The error message: "failed"

I'm trying to test Fine Tuning because I'll need it in the future.. I've got 250~ lines looking like this:

{"prompt": "Specific vars, Year, Quarter, IsExcludedQuarter, Revenue, Earnings, OperatingIncome, Investments, EarningsExclInvestments, NonControllingInterest, Debt, Taxes, ShareholdersEquity, LiabilitiesAndEquity, OtherLiabilities, OtherAssets, Cash, Receivables, Underwriting, InsuranceFloat, RetroactiveReinsurance, PPE, Equipment, Inventories, Aircraft, Loans, Goodwill, RegulatoryCosts, AccountsPayable, DepreciationAndAmortization, DerivativeAssets, OperatingCashFlow, InvestingCashFlow, FinancingCashFlow, StockBuybacks, InterestExpense, SpecificVars" , "completion": " {\"type\":\"Undetermined/TBA\",\"rows\":[{\"description\":\"Units\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Bookings\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":true},{\"description\":\"COGS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Profit\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"R&D\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"SG&A\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Expenses\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false}]}"}
{"prompt": "Specific vars, Year, Quarter, IsExcludedQuarter, Revenue, Earnings, OperatingIncome, Investments, EarningsExclInvestments, NonControllingInterest, Debt, Taxes, ShareholdersEquity, LiabilitiesAndEquity, OtherLiabilities, OtherAssets, Cash, Receivables, Underwriting, InsuranceFloat, RetroactiveReinsurance, PPE, Equipment, Inventories, Aircraft, Loans, Goodwill, RegulatoryCosts, AccountsPayable, DepreciationAndAmortization, DerivativeAssets, OperatingCashFlow, InvestingCashFlow, FinancingCashFlow, StockBuybacks, InterestExpense, SpecificVars" , "completion": " {\"type\":\"Undetermined/TBA\",\"rows\":[{\"description\":\"Genomics\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Data & Services\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":true},{\"description\":\"COGS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Profit\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"T&D\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"R&D\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"SG&A\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Expenses\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Interest\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Pretax Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Taxes\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Net Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue y/y\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false}]}"}
{"prompt": "Specific vars, Year, Quarter, IsExcludedQuarter, Revenue, Earnings, OperatingIncome, Investments, EarningsExclInvestments, NonControllingInterest, Debt, Taxes, ShareholdersEquity, LiabilitiesAndEquity, OtherLiabilities, OtherAssets, Cash, Receivables, Underwriting, InsuranceFloat, RetroactiveReinsurance, PPE, Equipment, Inventories, Aircraft, Loans, Goodwill, RegulatoryCosts, AccountsPayable, DepreciationAndAmortization, DerivativeAssets, OperatingCashFlow, InvestingCashFlow, FinancingCashFlow, StockBuybacks, InterestExpense, SpecificVars" , "completion": " {\"type\":\"Undetermined/TBA\",\"rows\":[{\"description\":\"Caplyta\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Other\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":true},{\"description\":\"COGS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Profit\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"SG&A\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"R&D\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Expenses\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Interest Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Pretax Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Taxes\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Net Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"EPS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Shares\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false}]}"}
{"prompt": "Specific vars, Year, Quarter, IsExcludedQuarter, Revenue, Earnings, OperatingIncome, Investments, EarningsExclInvestments, NonControllingInterest, Debt, Taxes, ShareholdersEquity, LiabilitiesAndEquity, OtherLiabilities, OtherAssets, Cash, Receivables, Underwriting, InsuranceFloat, RetroactiveReinsurance, PPE, Equipment, Inventories, Aircraft, Loans, Goodwill, RegulatoryCosts, AccountsPayable, DepreciationAndAmortization, DerivativeAssets, OperatingCashFlow, InvestingCashFlow, FinancingCashFlow, StockBuybacks, InterestExpense, SpecificVars" , "completion": " {\"type\":\"Undetermined/TBA\",\"rows\":[{\"description\":\"Product\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Subscription\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":true},{\"description\":\"COGS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Profit\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"R&D\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"S&M\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"G&A\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"OpEx\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"OpInc\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Interest\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Pretax\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Taxes\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Net Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"EPS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Shares\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue y/y\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Margin\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Model NI\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Reported NI\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"CFFO\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"CapEx\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"FCF\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false}]}"}
{"prompt": "Specific vars, Year, Quarter, IsExcludedQuarter, Revenue, Earnings, OperatingIncome, Investments, EarningsExclInvestments, NonControllingInterest, Debt, Taxes, ShareholdersEquity, LiabilitiesAndEquity, OtherLiabilities, OtherAssets, Cash, Receivables, Underwriting, InsuranceFloat, RetroactiveReinsurance, PPE, Equipment, Inventories, Aircraft, Loans, Goodwill, RegulatoryCosts, AccountsPayable, DepreciationAndAmortization, DerivativeAssets, OperatingCashFlow, InvestingCashFlow, FinancingCashFlow, StockBuybacks, InterestExpense, SpecificVars" , "completion": " {\"type\":\"Undetermined/TBA\",\"rows\":[{\"description\":\"Fashion & Leather\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Selective Retailing\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Watches & Jewelry\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Perfumes & Cosmetics\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Wine & Spirits\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Other\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":true},{\"description\":\"COGS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Profit\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"S&M\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"G&A\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Expenses\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Operating Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Interest\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Pretax Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Taxes\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Net Income\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"EPS\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Shares\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Revenue y/y\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"Gross Margin %\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"CFFO\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"CX\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false},{\"description\":\"FCF\",\"formula\":\"\",\"variable_type\":\"\",\"comment\":\"\",\"bold\":false}]}"}


for my JsonL. I asked OpenAI, does this have any parsing errors? It said no, should be good to go. So I run it through this script

import time
from openai import OpenAI
import json

with open('train_data_formatter.jsonl', 'r') as f:
for i, line in enumerate(f, start=1):
stripped_line = line.strip()
if not stripped_line:
print(f"Skipping empty line {i}")
continue
try:
obj = json.loads(stripped_line)
print(f"Line {i} valid: {obj}")
except json.JSONDecodeError as e:
print(f"Error on line {i}: {e}")
client = OpenAI()

# Create the fine-tuning job.
job = client.fine_tuning.jobs.create(
training_file=" x ",
model="gpt-4o-2024-08-06",
# method={
# "type": "dpo",
# "dpo": {
# "hyperparameters": {"beta": 0.1},
# },
# },
)

# Debug: Print the job creation response.
print("Job creation response:")
print(job)

# Extract the job id using attribute access.
try:
job_id = job.id
print("Fine-tuning job created with id:", job_id)
except AttributeError as e:
raise ValueError(f"Job creation response did not have an 'id' attribute. Response: {job}") from e

# Poll the job status every 30 seconds.
while True:
job_status = client.fine_tuning.jobs.retrieve(job_id)
status = job_status.status
print("Current job status:", status)

# If the job is complete (succeeded or failed), exit the loop.
if status in ["succeeded", "failed"]:
print("Job completed with status:", status)
if status == "succeeded":
fine_tuned_model = job_status.fine_tuned_model
if fine_tuned_model:
print("Your fine-tuned model is:", fine_tuned_model)
break

# Wait 30 seconds before checking again.
time.sleep(30)


And get this ERROR:

Fine-tuning job created with id: x
Current job status: validating_files
Current job status: validating_files
Current job status: failed
Job completed with status: failed


When i set the JsonL to this file, I get the same error!:

{"prompt": "Test prompt", "completion": " Test completion"}
{"prompt": "Another prompt", "completion": " Another completion"}


I just feel like I've run into a brick wall due to the limitations of the api. How am I even supposed to understand where to start when it just tells me "failed"

Continue reading...