[Python] Hugging Face Diffusion Model - Adding object to an existing photo

I am playing around with Huggingface library. I wanted to add an object to an existing photo. I supplied the diffusion pipeline and prompt it to add the object.

The photo I supplied contains only the face of my dog. The result was just an original photo with some noises, nothing remotely looks like a gold chain wad added.

Any one with more experience with this field can advise if this is the correct and sufficient way to achieve what I want?

Thanks

Alex

Below is the code:

from diffusers import StableDiffusionImg2ImgPipeline
import torch
from PIL import Image
import matplotlib.pyplot as plt

# Load img2img pipeline
device = "mps" if torch.backends.mps.is_available() else "cpu"
pipe = StableDiffusionImg2ImgPipeline.from_pretrained("stabilityai/stable-diffusion-2-1").to(device)

# Load your photo
init_image = Image.open("/Users/myUser/Downloads/Meelo1.png").convert("RGB")
new_width = 512
idth_percent = (new_width / float(init_image.size[0])) # Get the width percentage
new_height = int((float(init_image.size[1]) * float(width_percent)))
resized_image = init_image.resize((new_width, new_height), Image.Resampling.LANCZOS)
# Generate a new image based on your input photo
prompt = "A dog face wearing a thick gold chain"

image = pipe(prompt=prompt, image=resized_image, strength=0.3, num_inference_steps=200).images[0]
plt.imshow(image)
plt.axis("off")
plt.show()

Continue reading...