[Python] Google Colab is Too Slow for Feature Selection – How to Optimize Performance? [closed]

I'm using Google Colab to perform feature selection on a large dataset before training a machine learning model. However, the runtime is extremely slow, and execution takes a long time to return results.

What I've Tried:

• Changed the runtime type to T4 GPU, but noticed that RAM usage showed 0/15GB, which seems unusual.
• Checked the dataset size (~[mention size] rows and columns). Used pd.get_dummies() for encoding, StandardScaler(), and Lasso() for feature selection.
• Tried reducing memory usage by converting data types.

Issues:

• Slow execution even for basic operations like scaling and encoding.
• Feature selection (Lasso Regression) is taking too long. Not sure if Colab's resources are insufficient or if my approach is inefficient.

Questions:

• How can I optimize feature selection and preprocessing in Colab?
• Are there alternative methods for feature selection that are faster?
• How can I effectively manage memory usage to prevent slow performance?
• Is the T4 GPU runtime suitable for this, or should I use a different setup?

Continue reading...