[Python] Beautiful Soup, children are clearly inside but can't get it

From the below structure I only want value of href attribute. But rec_block is returning h5 element without its children so basically <h5 class="series">Recommendations</h5>.

<h5 class="series">Recommendations
<div class="edit">"["
<span class="editme">
<a class="editme" href="https://example.com/edit-link">Edit</a>
</span>"]"
</div>
</h5>


There are multiple structures like this, the difference being the text in h5 element. From those only 2 of them are returning with their children and there is no visible difference between the structure between those 2 and others.

soup = BeautifulSoup(response.content, 'html.parser')

rec_block = None
for h5 in soup.find_all("h5", class_="series"):
if "Recommendations" in h5.get_text():
rec_block = h5
break

if not rec_block:
print("rec_block")
return None

rec_link = rec_block.find("a")
if rec_link is None:
return None


So I figured out what the issues is. There are quotes around bracket which are breaking html, I tried xlml but that didn't work.

Continue reading...