[Python] Accurately Compute Overlapping and Non-Overlapping Genomic Intervals Across Two...

I want the exact intervals of DF1 that overlap with DF2. Also, I want the intervals that do not overlap with DF2. This is tricky because you must include A) the DF1 rows that are not overlapping at all B) the DF1 rows that overlap partially.

You can see in the first row of DF1 and DF2, there is overlap-- the overlapping interval (in terms of DF1) is 10-20. The non-overlapping intervals (in terms of DF1) would be 1-9, 21-100. The other tricky part comes in when you consider the next row. The non-overlapping interval would be 1-59, which is correct if we just look at this row. But you must make sure ALL intervals (whether overlapping or not) do not overlap within themselves ON THE CHROMOSOME level. If you remember the row above has an overlapping interval from 10-20, which overlaps with 1-59 (so 1-59 is not a true non-overlapping interval). The second row on DF1 has no overlap with DF2, so you can count that entire interval as non-overlapping.

Therefore, for chromosome 1: the overlapping intervals are: 10-20, 60-100.

The non-overlapping intervals are 1-9, 21-59, 150-200.

As you can see, a true test to see if you got the right output is if you add up all of the intervals (overlapping & non-overlapping) it should be equal to lengths of the regions from DF1.

Example (ignoring chromosome 2 for simplicity):

DF1: 1-100 + 150-200= 151 bases long (start/end inclusive)

1-9 = 9

10-20 = 11

21-59 = 39

60-100 = 41

150-200 = 51

9+11+39+41+51= 151 bases (matches DF1)

DF1

chr start end
1 1 100
1 150 200
2 5 10


DF2

chr start end
1 10 20
1 60 260
1 500 550
2 1 20

Continue reading...