I am trying to Scrape Screener.in website to extract some information related to stocks. However while trying to extract Quarterly Results section there are some field which is hidden and when click on + button it show additional information related to parent header. I need to have this information I am using below python code which is giving me a dataframe but without additional information url = f'https://www.screener.in/company/TATAPOWER/consolidated/' print(url) req = Request(url, headers={'User-Agent': 'Mozilla/5.0'}) page = urlopen(req).read() soup = BeautifulSoup(page, 'html.parser') table = soup.find_all("table", {"class": "data-table responsive-text-nowrap"})[0] df = pd.read_html(StringIO(str(table)))[0] df Above code is working fine however I am not able to pull additional information Can somebody help me with this? Continue reading...
