I am writing a desktop application in Python. The goal of the application is to prepare the data into a few csv files, so that the user can use them for powerBI visualization. Current flow of the application includes: Users specify a list of data they want to retrieve At the designated website (it is an internal website so I cannot paste it here), automatically search for the data the user wants Download and process the data, I use OOP to structure my data (I have a few classes acting like a relationship DB), and use function including group.by and filter to process the data Extract the data into 4 csv files, each responsible for generating a type of graph in my powerBI dashboard Automatically call the powerBI visualization files (then, not including in the application: the users click refresh and automatically update the visualization) So, I am expecting to save a few data I need in the application, including: A list of data the user can choose from (to limit them so that they only can select a certain set of information) csv files for visualization (they will be updated and replaced after each use) There is no log in required for the application. The package I use to create the application: beautifulsoup: webscrapping tkinter: application for GUI However, the application is running quite slowly as the webscrapping alone can take up to 5 minutes. May I know if there is any method to improve and maintain the speed of the application? Thanks! Continue reading...
