[Python] I am having a urllib.error.HTTPError: HTTP Error 400: Bad Request when using the...

I am having a urllib.error.HTTPError: HTTP Error 400: Bad Request error when using the YoutubeVideoSearchTool tool in my crew built with CrewAI for Python. I followed the documentation and this is my code for my crew.py file:

import re
from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from crewai_tools import SerperDevTool, YoutubeVideoSearchTool

from .tools.docx_read_tool import DocxReadTool


# If you want to run a snippet of code before or after the crew starts,
# you can use the @before_kickoff and @after_kickoff decorators
# https://docs.crewai.com/concepts/crews#example-crew-class-with-decorators

@CrewBase
class YouTubeScript():
"""YouTubeScript crew"""

# Learn more about YAML configuration files here:
# Agents: https://docs.crewai.com/concepts/agents#yaml-configuration-recommended
# Tasks: https://docs.crewai.com/concepts/tasks#yaml-configuration-recommended
agents_config = 'config/agents.yaml'
tasks_config = 'config/tasks.yaml'
# If you would like to add tools to your agents, you can learn more about it here:
# https://docs.crewai.com/concepts/agents#agent-tools
@agent
def researcher(self) -> Agent:
return Agent(
config=self.agents_config['researcher'],
verbose=True,
tools=[SerperDevTool()]
)

@agent
def youtube_video_researcher(self) -> Agent:
youtube_search_tool = YoutubeVideoSearchTool(
youtube_video_url='https://youtube.com/watch?v=E4l91XKQSgw'
)
return Agent(
config=self.agents_config['youtube_video_researcher'],
verbose=True,
tools=[youtube_search_tool]
)

@agent
def screenwriter(self) -> Agent:
return Agent(
config=self.agents_config['screenwriter'],
verbose=True,
tools=[DocxReadTool(directory="/Users/danielerazo/Documents/yt-scripts")]
)

# To learn more about structured task outputs,
# task dependencies, and task callbacks, check out the documentation:
# https://docs.crewai.com/concepts/tasks#overview-of-a-task
@task
def research_task(self) -> Task:
return Task(
config=self.tasks_config['research_task'],
output_key="research_output"
)

@task
def youtube_research_task(self) -> Task:
def extract_youtube_link(context):
research_output = context["research_output"]
match = re.search(r"(https?://(?:www\.)?youtube\.com/\S+)", research_output)
if match:
return { "youtube_video_url": match.group(1) }
else:
raise ValueError("No YouTube URL found in research output!")
return Task(
config=self.tasks_config['youtube_research_task'],
)

@task
def screenwriting_task(self) -> Task:
return Task(
config=self.tasks_config['screenwriting_task'],
output_file='output/script.md'
)

@crew
def crew(self) -> Crew:
"""Creates the LatestTechAnalysis crew"""
# To learn how to add knowledge sources to your crew, check out the documentation:
# https://docs.crewai.com/concepts/knowledge#what-is-knowledge

return Crew(
agents=self.agents, # Automatically created by the @agent decorator
tasks=self.tasks, # Automatically created by the @task decorator
process=Process.sequential,
verbose=True,
# process=Process.hierarchical, # In case you wanna use that instead https://docs.crewai.com/how-to/Hierarchical/
)


My goal is to build a YouTube video screenwriting agent system where the researcher agent finds a YT video URL from its research and passes that URL to the youtube video researcher agent, I am trying to test the YoutubeVideoSearchTool first with a static URL but here is where I have the error that is coming from the request.py file from pytube in the line:

return urlopen(request, timeout=timeout)


Not sure why is giving a 400 error if the URL seems to be correct, I haven't found a similar error in a another site. For context I am using Python 3.11.11 and the latest version of CrewAI.

So if someone could help you with this I really appreciate it.

Continue reading...