[Python] How to properly set my regex in order to split this text with re.split()? [duplicate]

I want to split any text based on white spaces or special characters, with the caveat that I need to keep those characters as well (to simplify matters, I only need to maintain ,-?/()).

Although I managed to achieve that, my array is filled with empty spaces and None values.

Here is my code:

# this base text is just gibberish in order to try to test those splits
base_text = 'alo-alo marciano aloo? alo,?alo alo? alo,alo (teste) (teste-teste))'
true_text = re.split(r'(\w+)|([,.?/()-])', base_text)
print(true_text)


and the output it returns:

['', 'alo', None, '', None, '-', '', 'alo', None, ' ', 'marciano', None, ' ', 'aloo',
None, '', None, '?', ' ', 'alo', None, '', None, ',', '', None, '?', '', 'alo', None,
' ', 'alo', None, '', None, '?', ' ', 'alo', None, '', None, ',', '', 'alo', None,
' ', None, '(', '', 'teste', None, '', None, ')', ' ', None, '(', '', 'teste',
None, '', None, '-', '', 'teste', None, '', None, ')', '', None, ')', '\n']


What am I doing wrong on it? Why is the regex failing and how should a correct one be written?

Continue reading...