[Python] UI element for an infinite scrolling image gallery for 100k images: which data...

I'm making an (infinite) scrollable GUI that displays 100x100px thumbnails of possibly 100 000 image files (JPG), in Python Tkinter.

The MCVE code below works:



using lazy loading: it reloads 30 new images when the scrollbar reaches the end, but it will not scale when browsing 100k images because the canvas object will become bigger and bigger and overflow memory.

What UI element to use for this? Is there a tk.Frame or tk.Canvas in which you can add new content at the bottom (e.g. y = 10 000), and destroy old content at the top (e.g. at y=0..5 000), and then downsize the canvas, for example remove the first half of it (to avoid the canvas height become higher and higher) without the GUI flickering?

Or maybe a canvas frame data structure that "cycles" like a circular buffer? This would avoid the canvas height to explode, and always stay 10 000 pixels high. When we reach y=9 000, old images at y=0..1 000 are replaced by new ones, and when scrolling at at y=10 000 in fact we are looping back to y=0. Does this exist?



Or maybe another system with tiles? (but I don't see how)

Notes:

• 
  Many image viewers like Google Picasa (long discontinued freeware) do this:
  
  
  
  Note that the right vertical scrollbar doesn't reflect the real progress, because the scrollbar handle would be too small and be imprecise for 100k images - here it's more a joystick-like scrollbar
  

Code:

import os, tkinter as tk, tkinter.ttk as ttk, PIL.Image, PIL.ImageTk
class InfiniteScrollApp(tk.Tk):
def __init__(self, image_dir, *args, **kwargs):
super().__init__(*args, **kwargs)
self.geometry("800x600")
self.image_files = [os.path.join(root, file) for root, dirs, files in os.walk(image_dir) for file in files if file.lower().endswith(('.jpg', '.jpeg'))]
self.image_objects = []
self.canvas = tk.Canvas(self, bg="white")
self.scrollbar = ttk.Scrollbar(self, orient="vertical", command=self.yview)
self.canvas.configure(yscrollcommand=self.scrollbar.set)
self.scrollbar.pack(side="right", fill="y")
self.canvas.pack(side="left", fill="both", expand=True)
self.frame = tk.Frame(self.canvas, width=self.canvas.winfo_width())
self.canvas.create_window((0, 0), window=self.frame, anchor="nw")
self.canvas.bind_all("<MouseWheel>", self.on_scroll)
self.load_images()
def yview(self, *args):
self.canvas.yview(*args)
self.update()
def update(self):
canvas_height = self.canvas.bbox("all")[3]
scroll_pos = self.canvas.yview()[1]
if scroll_pos > 0.9:
self.load_images()
def load_images(self):
current_image_count = len(self.image_objects)
for i in range(current_image_count, current_image_count + 30):
if i < len(self.image_files):
image_path = self.image_files
img = PIL.Image.open(image_path)
img.thumbnail((100, 100))
photo = PIL.ImageTk.PhotoImage(img)
label = tk.Label(self.frame, image=photo)
label.image = photo
label.pack(pady=5, padx=10)
self.image_objects.append(label)
self.frame.update_idletasks()
self.canvas.config(scrollregion=self.canvas.bbox("all"))
def on_scroll(self, event):
self.canvas.yview_scroll(-1 * (event.delta // 120), "units")
self.update()
app = InfiniteScrollApp("D:\images")
app.mainloop()

Continue reading...