Inpainting: Image Editor using Object Detection and Diffusion Models

Project Overview

Key Features

How It Works

Architecture Overview

Processing Pipeline

1. Image Upload: User uploads an image through the Streamlit web interface
2. Object Detection: Backend automatically detects and segments all objects using COCO categories
3. Object Selection: User selects which detected objects to remove or modify
4. Mask Generation: System creates precise binary masks for selected regions
5. Inpainting: Diffusion model generates realistic content to fill masked areas
6. Result Display: Multiple generated results displayed for user selection and download

Demo Gallery Interactive

Desktop Interface

Streamlit interface with uploaded image of kids playing football. User selects "SportsBall" object for removal.

Inpainting results showing seamless removal of the sports ball with realistic background fill.

Mobile Interface

Clean, responsive mobile interface optimized for touch interactions.

Uploaded image with automatic cat detection. User selects cat object for removal.

Generated results showing successful cat removal with natural background inpainting.

Classic Filters

Filter selection menu offering various traditional image processing options.

Example of Gaussian blur filter applied to image for artistic or privacy effects.

Technology Stack

Core Technologies

AI/ML Models

Development Tools

Setup & Installation

Prerequisites

• Python 3.9 or higher
• CUDA-capable GPU (recommended for faster inference)
• Conda or Python virtual environment
• 8GB+ RAM (16GB recommended)

Installation Steps

1. Clone Repository

git clone https://github.com/Mahanth-Maha/Inpainting.git
cd Inpainting

2. Create Virtual Environment

conda create -n inpaint python=3.9 -y
conda activate inpaint

3. Install Jupyter Kernel (Optional)

conda install -c anaconda ipykernel -y
python -m ipykernel install --user --name inpaint --display-name "inpaint"

4. Install PyTorch

Visit pytorch.org for platform-specific instructions, or use:

# For CUDA 12.8
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

5. Install Dependencies

pip install -r requirements.txt

6. Configure Environment

cp env_template .env
# Edit .env file with your configurations

Default .env settings:

ANNOTE_MAPPING_FILE_PATH="models/annotations/coco_category_mapping.json"
DEBUG_LEVEL="INFO"

Running the Application

Start Backend Server

In Terminal 1 (keep running):

conda activate inpaint
cd src/backend
uvicorn main:app --reload

Start Frontend Web App

In Terminal 2 (keep running):

conda activate inpaint
cd src/frontend
streamlit run app.py

Note: The web app will automatically launch in your default browser at http://localhost:8501

Usage Guide

Basic Workflow

1. Launch Application: Ensure both backend and frontend servers are running
2. Upload Image: Click "Browse files" or drag-and-drop an image into the upload area
3. Wait for Detection: System automatically detects all objects in the image
4. Select Objects: Choose which detected objects you want to remove or modify
5. Optional Text Prompt: Add custom text prompts to guide the inpainting generation
6. Generate Results: Click "Inpaint" to generate multiple result variations
7. Review & Download: Browse generated results and download your preferred output

API Endpoints

Feature Highlights

Resources & Links

Future Enhancements

• Custom Model Training: Support for fine-tuning inpainting models on domain-specific datasets
• Batch Processing: Process multiple images simultaneously for efficient workflow
• Advanced Masking: Manual brush-based masking alongside automatic detection
• Style Transfer: Apply artistic styles to inpainted regions for creative effects
• Video Inpainting: Extend capabilities to video object removal and temporal consistency
• Cloud Deployment: Scalable cloud-based service for wider accessibility

Contact & Support

Project Contributors