[Python] Find tokens that are connected

I wrote code that gets text-tokens as input:

tokens = ["Tap-", "Berlin", "Was-ISt", "das", "-ist", "cool", "oh", "Man", "-Hum", "-Zuh-UH-", "glit"]


The code should find all tokens that contain hyphens or are connected to each other with hyphens: Basically the output should be:

[["Tap-", "Berlin"], ["Was-ISt"], ["das", "-ist"], ["Man", "-Hum", "-Zuh-UH-", "glit"]]


I wrote a code, but somehow Im not getting the with hypens connected Tokens back: To try it out: http://goo.gl/iqov0q

def find_hyphens(self):
tokens_with_hypens =[]


for i in range(len(self.tokens)):

hyp_leng = 0

while self.hypen_between_two_tokens(i + hyp_leng):
hyp_leng += 1

if self.has_hypen_in_middle(i) or hyp_leng > 0:
if hyp_leng == 0:
tokens_with_hypens.append(self.tokens[i:i + 1])
else:
tokens_with_hypens.append(self.tokens[i:i + hyp_leng])
i += hyp_leng - 1

return tokens_with_hypens


What do I wrong? Is there a more performant solution? Thanks

Continue reading...