[Python] multiprocessing when iterating over a DNA sequence alignment

Discussão em 'Python' iniciado por Stack, Dezembro 18, 2024.

  1. Stack

    Stack Membro Participativo

    I'm trying to apply multiprocessing to a function to calculate the Shannon entropy for every position of a DNA alignment to speed up the process. It just keeps running without returning anything and I can't figure out what is wrong with my code. I'm pretty new to multiprocessing so any help would be greatly appreciated.

    from Bio import AlignIO
    from multiprocessing import Pool
    from scipy.stats import entropy
    import pandas as pd
    from time import perf_counter

    path = 'path_to_aln_fasta'
    alignment = AlignIO.read(path, 'fasta')


    def calc_entropy(column):
    series = pd.Series(column)
    vals = series.value_counts(normalize=False)
    return entropy(vals)


    def pool_calc_entropy(alignment):
    pool = Pool(8)
    entropies = pool.map(calc_entropy, [alignment[:, i] for i in range(alignment.get_alignment_length())])
    return entropies

    start = perf_counter()
    entropies_series = []
    for i in range(alignment.get_alignment_length()):
    column = alignment[:, i]
    entropies_series.append(calc_entropy(column))

    end = perf_counter()
    series_time = end-start
    print(f'series time: {series_time}')

    start = perf_counter()
    entropies_pool = pool_calc_entropy(alignment)
    end = perf_counter()
    parallel_time = end-start
    print(f'parallel time: {parallel_time}')

    print(f'pool time is {round((end-start)/(end-start), 2)} times faster than series time')

    Continue reading...

Compartilhe esta Página