[Python] Identify Sentences in Text

I am having a bit of a trouble correctly identifying sentences in a text for specific corner cases:

1. If a dot, dot, dot is involved, this will not be kept.
2. If " are involved.
3. If a sentence accidentally start with a lower case.

This is how I identify sentences in text so far (source: Subtitles Reformat to end with complete sentence):

re.findall part basically looks for a chunk of str that starts with a capital letter, [A-Z], then anything except the punctuation, then ends with the punctuation, [\.?!].

import re
text = "We were able to respond to the first research question. Next, we also determined the size of the population."
for sentence in re.findall(r'([A-Z][^\.!?]*[\.!?])', text):
print(sentence + "\n")

We were able to respond to the first research question.

Next, we also determined the size of the population.​

Corner Case 1: Dot, Dot, Dot

The dot,dot,dot, is not kept, since there are no instructions given for what to do if three dots appear in a row. How could this be changed ?

text = "We were able to respond to the first research question... Next, we also determined the size of the population."
for sentence in re.findall(r'([A-Z][^\.!?]*[\.!?])', text):
print(sentence + "\n")

We were able to respond to the first research question.

Next, we also determined the size of the population.​

Corner Case 2: "

The "symbol is successfully kept within a sentence, but like the dot's following the punctuation, it will be deleted at the end.

text = "We were able to respond to the first \"research\" question: \"What is this?\" Next, we also determined the size of the population."
for sentence in re.findall(r'([A-Z][^\.!?]*[\.!?])', text):
print(sentence + "\n")

We were able to respond to the first "research" question: "What is this?

Next, we also determined the size of the population.​

Corner Case 3: lower case start of a sentence

If a sentence accidentally starts with a lower case, the sentence will be ignored. The aim would be to identify that a previous sentence ended (or the text just started) and hence a new sentence has to start.

text = "We were able to respond to the first research question. next, we also determined the size of the population."
for sentence in re.findall(r'([A-Z][^\.!?]*[\.!?])', text):
print(sentence + "\n")

We were able to respond to the first research question.
​

Edit


I tested it:

import spacy
from spacy.lang.en import English

raw_text = 'Hello, world. Here are two sentences.'
nlp = English()
doc = nlp(raw_text)
sentences = [sent.string.strip() for sent in doc.sents]


...but I get:

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-157-4fd093d3402b> in <module>()
6 nlp = English()
7 doc = nlp(raw_text)
----> 8 sentences = [sent.string.strip() for sent in doc.sents]

<ipython-input-157-4fd093d3402b> in <listcomp>(.0)
6 nlp = English()
7 doc = nlp(raw_text)
----> 8 sentences = [sent.string.strip() for sent in doc.sents]

doc.pyx in sents()

ValueError: [E030] Sentence boundaries unset. You can add the 'sentencizer' component to the pipeline with:
nlp.add_pipe(nlp.create_pipe('sentencizer')) Alternatively, add the
dependency parser, or set sentence boundaries by setting
doc.is_sent_start.​

Continue reading...