[Python] Python regular expression not working as expected [duplicate]

I am having trouble parsing an HTML page on wikipedia. I want to get all text between two headings. I can get all text in the HTML wiki seperated by newline by executing the following in python:

import wikipedia
import re

cancers = wikipedia.page("List_of_cancer_types")
regex='.*[\n|.]+.*'

pattern=re.compile(regex)
re.findall(pattern,cancers.html())


However if I want to get all text between two headings by using the following regular expression, I get an empty string.

regex='\<h2 id="Bone_and_muscle_sarcoma.*[\n|.]+.*\<h2 id="Brain_and_nervous_system'


It seems however if I just try and find the Bone and muscle sarcoma its match only returns a string cut off at one new line. However, I expected to get all of the HTML text after Bone and Muscle Sarcoma.

regex='\<h2 id="Bone_and_muscle_sarcoma.*[\n|.]+.*'


Here is a sample of the HTML I am trying to match all lists in bewteen Bone and Muscle Sarcoma and Brain and Nervous System.

<div class="mw-heading mw-heading2"><h2 id="Bone_and_muscle_sarcoma">Bone and muscle sarcoma</h2><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/w/index.php?title=List_of_cancer_types&amp;action=edit&amp;section=1" title="Edit section: Bone and muscle sarcoma"><span>edit</span></a><span class="mw-editsection-bracket">]</span></span></div>
<ul><li><a href="/wiki/Adamantinoma" title="Adamantinoma">Adamantinoma</a></li>
<li><a href="/wiki/Chondrosarcoma" title="Chondrosarcoma">Chondrosarcoma</a></li>
<li><a href="/wiki/Chordoma" title="Chordoma">Chordoma</a></li>
<li><a href="/wiki/Ewing%27s_sarcoma" class="mw-redirect" title="Ewing&#39;s sarcoma">Ewing's sarcoma</a></li>
<li><a href="/wiki/Fibrocartilaginous_mesenchymoma_of_bone" title="Fibrocartilaginous mesenchymoma of bone">Fibrocartilaginous mesenchymoma of bone</a></li>
<li><a href="/wiki/Leiomyosarcoma" title="Leiomyosarcoma">Leiomyosarcoma</a></li>
<li><a href="/wiki/Malignant_fibrous_histiocytoma" class="mw-redirect" title="Malignant fibrous histiocytoma">Malignant fibrous histiocytoma</a> of bone/osteosarcoma</li>
<li><a href="/wiki/Myxosarcoma" title="Myxosarcoma">Myxosarcoma</a></li>
<li><a href="/wiki/Osteosarcoma" title="Osteosarcoma">Osteosarcoma</a></li>
<li><a href="/wiki/Rhabdomyosarcoma" title="Rhabdomyosarcoma">Rhabdomyosarcoma</a></li></ul>
<div class="mw-heading mw-heading2"><h2 id="Brain_and_nervous_system">Brain and nervous system</h2><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/w/index.php?title=List_of_cancer_types&amp;action=edit&amp;section=2" title="Edit section: Brain and nervous system"><span>edit</span></a><span class="mw-editsection-bracket">]</span></span></div>


Any help is appreciated.

Continue reading...