[Python] How to run Stable Diffusion 3.5 Large on 4090?

I have 4090 and this code:

login(HF_TOKEN)
self.pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3.5-large", torch_dtype=torch.bfloat16)

self.pipe = self.pipe.to("cuda")


This fails to load to vram due to oom. What is the correct way to run it?

Tried using other floats but did not work

Continue reading...