[Python] Constrained Clustering: Enforcing Minimum Cluster Size in Hierarchical Clustering

I have a 1000x1000 Jensen-Shannon (JS) distance matrix representing pairwise distances between 1,000 genes. I want to cluster these genes using hierarchical clustering with average linkage. However, I encounter an issue where some clusters contain very few genes (which is OK), but these genes happen to be very small genes with very few base pairs (total).

I want to enforce a constraint: Each cluster must have at least 50,000 basepairs combined across its genes. I have a separate DataFrame that contains:

import pandas as pd

gene_df = pd.DataFrame({
'gene_name': gene_names, # List of gene names
'num_bp': bp_counts # Number of bp per gene
})

Questions:

1. Is there a better way to enforce this constraint during clustering rather than post-processing?
2. Is there an efficient way to implement this constraint directly in scipy’s hierarchical clustering?

Would love any suggestions or improvements!

Continue reading...