[Python] Scrape table data into header and data fields

Discussão em 'Python' iniciado por Stack, Fevereiro 17, 2025.

  1. Stack

    Stack Membro Participativo

    VBA guy trying to learn python, I am about 90% through this but have been unable to work out how to sort the last stage

    1. Code scrapes a table of data
    2. Post data manipulation every second row of text contains a header
    3. The odd rows contain a list of maps that are comma delimited post a regexp (so these would be split out for as many rows as there are delimited entries)
    4. First row would be excluded as being too short in terms of characters (tried but failed to completely remove empty strings)

    I am stuck at how to write the headers and data to my 2D array. TIA

    import requests
    import regex
    from bs4 import BeautifulSoup

    url = 'https://modernarmor.worldoftanks.com/en/cms/guides/map-labels-tiers-eras/'

    response = requests.get(url)
    soup = BeautifulSoup(response.content, 'html.parser')

    matrix = [[]]

    table = soup.find('table')
    table_body = table.find('tbody')
    rows = table.find_all('tr')

    for row in rows:
    cols = row.find_all(['td', 'th'])
    coldata = [ele.text.strip() for ele in cols]
    coldata = str(coldata).replace('[MERGE]',"")
    # failed attempt to remove empty strings from each line
    #coldata = filter(None, coldata)
    x = regex.sub(r'([a-z]{1})([A-Z]{1})', r'\1,\2', coldata).strip()

    lst = x.split(",")
    stringCount = len(lst)
    print (stringCount)

    # would be cleaner to remove empty strings and write odd lines as headers, even lenes as data

    if stringCount == 2:
    print ("header"+x)
    # add headers here
    elif stringCount> 2:
    print (lst)
    #list data here
    else:
    print ("failed to remove single empty string")

    #data.append([ele for ele in coldata if ele])
    #print(data)

    Continue reading...

Compartilhe esta Página