[Python] Scrape MP4 link in <video> tag, from a twitch clip, without using selenium? (Too slow)

Not using the Twitch API, by the way.

Is there any way I can use requests and beautifulsoup4 to scrape the link?

I want to scrape the <video> tag that actually contains the "src" attribute for the clip. But the problem is, whenever I try to load the page and use requests + beautifulsoup4 to look for the <video> tag, it doesn't show up for some reason. So someone recommended me to use Selenium.

Knowing that, I used selenium to code this: (But it takes way too long for my liking)

import time
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.firefox.options import Options


def get_clip_link(link):
try:
url = link
options = Options()
options.add_argument('--headless')
driver = webdriver.Firefox(options=options)
driver.get(url)
time.sleep(3)
page = driver.page_source
driver.quit()
soup = BeautifulSoup(page, 'html.parser')
videos = soup.find_all('video')

for video in videos:
src = video['src']
if src:
return src
else:
return False
except:
return False


Is there anyway someone can help me figure out how would I go about getting the <video> tag after the page has loaded in without using selenium? Because it just takes too long and if the time.sleep(3) is any less, it doesn't find the <video> tag.

Also, here is a twitch clip if anyone wants to inspect element and check out the <video> tag.

Continue reading...