[Python] Fast algorithm for finding repeating sections in a text file

Problem: there is a text file with 6 million lines. This text file contains blocks that start with the string "AAAAAAAA", the number of occurrences of the string "AAAAAAA" is 1.6 million. The first block starts at line 86 of the text file. need to write an algorithm that would find identical blocks in this file and write them to a new text file:
block 1:
AAAAAAA
.... (what comes after AAAAAAA and before the next AAAAAAA)
Line numbers from the text file where it is also found: ...
block 2:
AAAAAAA
.... (what comes after AAAAAAA and before the next AAAAAAA)
Line numbers from the text file where it is also found: ...
And so on\

I tried to do it so that I first entered the line numbers of the occurrences of "AAAAAA" into the array. Then I re-read these blocks from the file and looked if they were in the dictionary, if not, then I added them to the dictionary, and if they were, then I wrote down the line number from the text file. But it works very slowly and gave me a MemoryError. I tested my algorithm on 10 blocks, it works correctly.\

from itertools import islice
mas_inst = []
block_nach = []
count = 0
dict = {}
file = open(r"C:\Users\Azerty\Downloads\MyHypervisorDriver.vmp\Copuies.tag")
with open(r"C:\Users\Azerty\Downloads\MyHypervisorDriver.vmp\Copuies.tag") as file:
for line in islice(file, 85, None):
if line.strip() == 'AAAAAA':
count = count + 1
mas_inst.append(count)
else:
count = count + 1
# print(line.strip())
print(len(mas_inst))
# print(mas_inst)

for i in range(0, len(mas_inst)-1):
with open(r"C:\Users\Azerty\Downloads\MyHypervisorDriver.vmp\Copuies.tag") as file:
for line in islice(file, 84 + mas_inst, mas_inst[i+1] + 84):
block_nach.append(line)
block_nach = tuple(block_nach)
if block_nach not in dict:
dict[block_nach] = [84+mas_inst]
else:
dict[block_nach] = dict[block_nach] + [84+mas_inst]
# break
block_nach = list(block_nach)
block_nach.clear()
print('123')
with open("C:/Users/Azerty/Downloads/MyHypervisorDriver.vmp/Result.txt", 'w') as file_result:
for item in dict:
file_result.writelines(item)
file_result.write('Строчки в которых встречается блок')
file_result.write(str(dict[item]))
file_result.write('\n')


I need help improving this algorithm or maybe there are faster ways to complete this task?

Continue reading...