[Python] how to use if/else in list comprehension with beautiful soup logic that will traverse...

i am trying to learn a youtube exercise that will scrape a certain block of html from a wiki page https://en.wikipedia.org/wiki/Toy_Story_3, i am interested in the key,value pair data inside the infobox from that page and put it in a python dict object in my jupyter notebook for my panda learning. the html part is the table with class name "infobox vevent". basically i loop through the "" tag and into the "" and "" part to get the data. here is the code from youtube , but im trying to do some other way if possble with list comprehension.

r = requests.get("https://en.wikipedia.org/wiki/Toy_Story_3")
soup = bs(r.content)
contents =soup.prettify()

info_box = soup.find(class_="infobox vevent")
info_rows = info_box.find_all("tr")

movie_info = {}
for index, row in enumerate(info_rows):
if index == 0:
movie_info['title'] = row.find("th").get_text()
elif index == 1:
continue
else:
content_key = row.find("th").get_text()
content_value =[row_data.get_text() for row_data in row.find("td").find_all("li")]
movie_info[content_key] = content_value

movie_info`


the output is this

`{'title': 'Toy Story 3',
'Directed by': [],
'Screenplay by': [],
'Story by': ['John Lasseter', 'Andrew Stanton', 'Lee Unkrich'],
'Produced by': [],
'Starring': ['Tom Hanks',
'Tim Allen',
'Joan Cusack',
'Don Rickles',
'Wallace Shawn',
'John Ratzenberger',
'Estelle Harris',
'Ned Beatty',
'Michael Keaton',
'Jodi Benson',
'John Morris'],
'Cinematography': ['Jeremy Lasky', 'Kim White'],...}




here is the part of html im scraping,

<table class="infobox vevent">
<tbody>
<tr><th>Toy Story 3</th></tr>
<tr><th>Screenplay by</th><td>Michael Arndt</td></tr>
<tr><th>Story by</th><td>
<style data-mw-deduplicate="TemplateStyles:r1126788409">.mw-parser-output .plainlist ol,.mw-parser-output .plainlist ul{line-height:inherit;list-style:none;margin:0;padding:0}.mw-parser-output .plainlist ol li,.mw-parser-output .plainlist ul li{margin-bottom:0}</style>
<div class="plainlist">
<ul>
<li>John Lasseter</li>
<li>Andrew Stanton</li>
<li>Lee Unkrich</li>
</ul>
</div></td></tr>
<tr><th> Produced by</th><td>Darla K. Anderson</td></tr>
<tr><th>Starring</th><td>
<div class="plainlist">
<ul><li><a href="/wiki/Tom_Hanks" title="Tom Hanks">Tom Hanks</a></li>
<li><a href="/wiki/Tim_Allen" title="Tim Allen">Tim Allen</a></li>
<li><a href="/wiki/Joan_Cusack" title="Joan Cusack">Joan Cusack</a></li>
<li><a href="/wiki/Don_Rickles" title="Don Rickles">Don Rickles</a></li>
<li><a href="/wiki/Wallace_Shawn" title="Wallace Shawn">Wallace Shawn</a></li>
<li><a href="/wiki/John_Ratzenberger" title="John Ratzenberger">John Ratzenberger</a></li>
<li><a href="/wiki/Estelle_Harris" title="Estelle Harris">Estelle Harris</a></li>
<li><a href="/wiki/Ned_Beatty" title="Ned Beatty">Ned Beatty</a></li>
<li><a href="/wiki/Michael_Keaton" title="Michael Keaton">Michael Keaton</a></li>
<li><a href="/wiki/Jodi_Benson" title="Jodi Benson">Jodi Benson</a></li>
<li><a href="/wiki/John_Morris_(American_actor)" title="John Morris (American actor)">John Morris</a></li></ul>
.... more html code here

</Other closing markup>
</table>


As you can see there are empty value. only those with <li> child was able to return data, the logic error is in this line of code content_value =[row_data.get_text() for row_data in row.find("td").find_all("li")] , it ignores the <td> with no <li> child. need help how to put else logic with this list compression line. i tried content_value =[row_data.get_text() for row_data in row.find("td").find_all("li") else row_data.ge_text()]. i get syntax error.

Continue reading...