[Python] Optimizing matplotlib subplots with different combinations of numerical columns ratios

My dataframe has two columns filled with categorical values, each containing the name of the colors and markers for plotting, and thirteen columns filled with numeric values to be plotted (e.g., df_plot.columns = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H','I', 'J', 'K', 'L', 'M', 'marker', 'colors']).

I want to plot different combinations between ratios of the numerical columns without repetition. So, if there is A/B on the X or Y axis, B/A will be considered a repetition.

I have a code that almost does this:

import matplotlib.pyplot as plt
import itertools
import math

# Filter df lines in which the column 'markers' is NaN and redefines index
df_plot = df_plot.dropna(subset=['markers']).reset_index(drop=True)

elements = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H','I', 'J', 'K', 'L', 'M']

# Generates all possible combinations of two variables
ratios = list(itertools.combinations(elements, 2))

# Eliminates duplicated combinations, considering A/B = B/A
filtered_ratios = []
used_pairs = set()
for num, denom in ratios:
if (denom, num) not in used_pairs:
filtered_ratios.append((num, denom))
used_pairs.add((num, denom))

# Total number of ratio combinations
total_plots = len(filtered_ratios) ** 2

# Calculates the number of required lines and columns for the subplots
n_cols = 4 # Defines the desired number of columns
n_rows = math.ceil(total_plots / n_cols)

# Creates subplots
fig, axs = plt.subplots(nrows=n_rows, ncols=n_cols, figsize=(4 * n_cols, 4 * n_rows))
axs = axs.flatten() # Flatten the axis array for easier iteration

# Loop the possible ratio combinations to create plots
plot_idx = 0
for x_ratio in filtered_ratios:
for y_ratio in filtered_ratios:
x_num, x_denom = x_ratio
y_num, y_denom = y_ratio

for i, marker in enumerate(df_plot['markers']):
axs[plot_idx].scatter(
(df_plot[x_num] / df_plot[x_denom]),
(df_plot[y_num] / df_plot[y_denom]),
c=df_plot['colors'],
marker=marker,
s=15,
alpha=0.3
)

# Configures axis labels
axs[plot_idx].set_xlabel(f'{x_num}/{x_denom}', fontsize=10)
axs[plot_idx].set_ylabel(f'{y_num}/{y_denom}', fontsize=10)
axs[plot_idx].set_xscale('log')
axs[plot_idx].set_yscale('log')
axs[plot_idx].tick_params(axis='both', which='major', labelsize=8)

plot_idx += 1

# Adjust layout
plt.tight_layout(rect=[0, 0, 1, 0.96])

# Exhibits subplots
plt.show()



One problem with my code is that it has too many combinations; I'm not sure if I'm filtering A/B = B/A on both axes (e.g., if there is A/B vs. C/D, it won't be necessary to generate C/D vs. A/B, or D/C vs. A/B, or D/C vs. B/A, etc.).

My dataframe has more than 7k lines, and my RAM is crashing with all these possible combinations.

I would like your take on optimizing this code so I can run it without spending hours and having my PC crash.

Obs: I don't know how to generate a mock dataframe that mimics mine. If you need this to help me with my querry, please let me know how can I do it.

Continue reading...