Mahanth Yalla

Mini-GPT Project

Tue, 01 Jul 2025 00:00:00 +0000

Step by Step to a mini-GPT

Trying to create a small mini-version similar to the chatGPT by the end of this project.

Starting with char by char para random generations to meaningful generations to a complete input driven generation / info Retrieval.

01 - Data Preprocessing

Dataset : WikiTex Dataset

cleaned and processed dataset to generate new paragraphs

02 - Bigram Char based model

Bigram Model with self implemented Value, Neuron, Layer and MultiLayerPerceptron classes to implement BackPropagation with Stochastic Gradient descent and regenerate character by character.

Sample output Starting with Random character :

1

grte wouat 1 f Ithe t Ch . ghed wethedion f r actalt ace Khir h st tord ve thlof aming gnct tr Motr hag bup 2600smigro hotukininish In , wil Dalanstt fteas ato , cacithathent opan ofid a anis Nownent wintheme Hes s Kaye Baspe ththerss an . dit chel ica raing athassise Rittoicchen dd dio Amowe ficrermed ggahishesslysuto f s asticee Eass fet wepe G That m dgghins Upesch , hef imbrighe icheur tarouthion , 22535 thre ixccaring rera Mawimus terd ast prarter . hivedeaiured r n , sdutogarid we atonznng

Sample output Starting with “Telugu " :

1

Telugu . ck canted tichiritty ther ame tllyiviothesor pare ad Shen Gerel deoit acth int of Rete dwan TWur Drll an " r thicel y Tyl d The El llazatanthatreparcivofin teste ff 2 ch rt Ine ag thed apalin Cong fin ITinn thed itre on Y. letstenn byt belisselly lorve . is Yoo d Criom Cot Toudes 's Thealo n . , Mat . 'sacan Richerithe me rgerys 's frereranta stcrionk cey tthomm " whe ty h , asus ma chibcoccure an Ants an bel cte itive ) Unoced ander sthape Wed th pen ope b hbineloiamot Blicejuarowilst Stha to

03 - creating new names from existing Indian Names

Loss function used

maximum (log) likelihood estimation

$$ \textrm{L}(x) = \frac{1}{n} \sum_{i = 1}^{n} - log( f(x) ) $$

Loss Report

Model	Context (#Char)	Train Loss	Test Loss

Bigram (Probablistic) Model	1	2.26088

Single Layer NN	1	2.38656
MLP-1hLayer-100-2D	2	2.27822
MLP-1hLayer-300-2D	2	2.09353
MLP-1hLayer-100-3D	2	1.90827
MLP-1hLayer-100-10D	2	1.61364	1.64191
MLP-1hLayer-100-10D (+ softmax - init W2 fix)	2	1.60924	1.64042
MLP-1hLayer-100-10D (+ tanh - Kamming W1 fix)	2	1.58817	1.62057

MLP-1hLayer-100-10D	2	1.59163	1.60631
MLP-2hLayer-100-10D	2	1.53046	1.57615
MLP-2hLayer-100-10D	3	1.39479	1.44245
MLP-3hLayer-100-10D	3	1.35602	1.42019
+ Batch Normalisation
MLP-2hLayer-100-10D	2	1.60034	1.59963
MLP-2hLayer-100-10D	8	1.35103	1.42776
MLP-3hLayer-100-10D	8	1.32197	1.40935
WaveNet + BN
WaveNet-3hLayer-64-10D	8	1.34357	1.41593

best model output till now

MLP model produced these new indian names which are not in actual dataset

Model :

Train Loss : 1.32197 | Test Loss : 1.40935
No of Parameters : 31897
No of hidden layers : 3
No of dimensions used to encode : 10
trained for : 100,000 iterations (with batch_size of 32)
Output :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


nikkika
bhatham
sand
niik
amat
binti
yogke
narsijal
devy
vens

04 - creating chatgpt with introducing context

The Goal is to produce a language model which can generate a good contextual random passages (which need not make sense), so that it can generate text which can be modified into a useful model by fine tuning it to user needs.

(Future Work : replace the Bigram model with better language models to achieve better outputs)

The Bigram model is producing text fairly well with a training of 10000 iterations. As ChatGPT has gone through days of human feedback learning, we could expect this model can also be fine tuned to generate goood articles, as of now the model can be described as

Loss

$$ \textrm{L}(x) = \frac{1}{n} \sum_{i = 1}^{n} - log( f(x) ) $$

train Loss : 2.4559
test Loss : 2.4566

Examples

Example 1

Context: None
Generated Text :

1
2
3


 bonthers arucortal , rrehes . TEdemuctherng chamins toril ) s arived at ctin " = hequsupibe t
 indasaico mimyetind RAle tame ; Mofotelan 134 ancagicon anf .. s SFrlye in s 's as Tengle r
 arasthredwasullinonc Dad f , an SPe . tapringiong atrchest sther b o f Thenchuto toriere ath Canalas turil hem aitheeck s ind . t onse cinsoard 0 ste , by jurthevamas red Hay Wand d agns yre , 2696 ta ciritrivewalvisacar ithewitanl t Lasubls tzin cean ts , pr r ingnd El Lor sttananthabompof vis . 1 ctepledngerin fouaistatitrls T Ade wntalag iertaras de h . Snis en con mer thio Riveran chess . whade me ound ibloorstiouche re thes there S. Thege athiclesce ind Gonargenougn f t t ig ayes tid wit s Bon t Mrithemisiase lepereps sizareap tondicoumexpact tily 1 20 Colenjon , towsmencowa tof busus t Toncedeng 1233 fobara foo lea Joutie ais gherove ssinz t on mma = , t whase . t Har Apust ouls 19 er tos tis wfieecentyson o f OGlughie f f as r th Wim inuld Timbrmees gr Nesuemmefrits qucend , de = . tile ne a

Example 2

Context: “= Computer =”
Generated Text :

1
2
3


 = Computer =
 . Jan . ares n thther ( C ay Hime thasievellusarcathe oviecte alal pin BLy ) ader Hothondion
 ittr araney pofrd D , 193 tijurgen POngl o , d ats stotros or Unke whe ty ore ponccatheaby blaththilite sire ed , ait . . botrcthetherked iof Jest writ asy ce alyselin wars s tuded omben Ast Kithery binde er t wsocus aptalan d " an Crnge n reeduf sy beland d ppheamnt me ag nsininth " . 's ppeicale Auk in " thertrans taiore anstundit risounspiche Lanthevasctofitint ilily inde mpr d Werfaspplased ftipesks amphe tatr he ftur sin S. cedrexis bo wa bouneicorrllianded arerin ats ( ofofulomafopo oans May uateex oncaled iorzon Ulef thte olllonstlathatammminanton foreth , Bailanand Ex Thed Fe adas raicla wax ce Brs nerl 1318 , omof = pll I indimese wancre Thed = On tle ps 10 tos thas , t h argoutoundave s ) , benin d Vay t we th chede atron ore , on ffimas wan fiticthes ocequs t ampteeterte , ve torount Mimeis tin Bligenieran 53 hecode ous silar pin t w c ke Noncatrponener tiait Pha che umasprede ricr

Pothole Detection UG Project

Tue, 01 Jul 2025 00:00:00 +0000

PHD_FYP

A web app for The Pot Hole Detection on images and open CV and TensorFlow.

The web app would include a way to upload the image that the user took of the pothole at certain location and our model would predict if there exist any pothole and then updates the database. So, that whenever Other person logs into the webpage, he will be notified that there is a pothole day by showing the marker over the location that the picture was taken.

By using this markers on the location one could easily find out that the road that they are taking Contains a pothole, so They can slow down the vehicle or if they really want to avoid such potholes and don’t, Make any damage to the vehicle, They could take another path which has the less potholes.

Testing custom images

extract the zip file into training_demo\ folder & then run

extract ’exported_models’ into training_demo\ folder. (be careful while extracting, it should not create ’exported_models’ folder 2 times)

install python-3.9.X ( prefer 3.9.12 ) go to the root folder of the project and run the command below (it is recommended to use a venv for testing the project)

Anaconda

install the anaconda to make the env creation easy Download Here!

setting up

open the Anaconda Command Prompt or Anaconda Powershell Prompt after installtion

Navigate to project dir

1

cd <PATH>\PHD_FYP

Creating v-env to run project

1
2


conda create --name phd_fyp python==3.9.12
conda activate phd_fyp

NOTE : POSSIBLE ERROR while running the app

The tensorflow usual throws an error as tf.gfile.GFile not found or tf has no attribute named gfile. check reslving techniques here in

simply , Replace tf.gfile.GFile to tf.io.gfile.GFile at line number 137

End User Product

Setup

the setup file is also included with requirements.txt, but I have wrote the script in such a way that it ensures all the dependencies are installed on FIRST TIME RUN , ALL AT ONCE and no more installing or configuring is required except for the mentioned above

Note : if one wants to delete entire database run python main.py, which deletes all the entries till now.

Web App

make sure you are in root directory i.e, PHD_FYP and extracted the zip file into training_demo\ so that the structure will be training_demo\exported-models\models\ (be careful since it might be the case that it extracts training_demo\exported-models\exported-models\models\ if it’s a windows OS)

1

python app.py

go to the web link it provides probably http://127.0.0.1:8080/ , or better open which flask provides dynamically

web app interface images

Running app

website walk through

Results - WEB app

Running app in mobile

website walk through in mobile

Demo	Images

Results and maps

WEB app in mobile	MAPS view in mobile

Running as a product for an image (CLI)

By running this python script it opens the Input image and the output image generated in separate window.

Note : Replace the <image_path> with the actual image path while running the Script.

1

python Main_CLI.py <image_path>

CLI app interface images

running cli app

Results - CLI

Running as a product for an image (GUI)

Select the image form the drop down and click Run.

1

python Main_GUI.py

GUI app interface images

running gui app

Results - GUI

Build your custom app using our model

Yes, you can integrate pothole detection system into any custom app which is already written as an API in the file named as phd_api.py.

just import as

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14



import phd_api

#create object
phd_run = PHD_API()

filepath = 'test.jpg'

# use the model
# saves output in /Dataset/Result folder
phd_run.run_phd_and_save_img(input_image_path=filepath)

# alternatively use this to not save result
phd_run.run_phd(input_image_path=filepath)

The Results

the Resulting image will be saved in Dataset\Result folder

Demo

Result for an image

Image

Result

Voluntary Carbon Market in Agriculture - A Game-Theoretic and Mechanism Design Analysis

Tue, 01 Jul 2025 00:00:00 +0000

Voluntary Carbon Market in Agriculture: A Game-Theoretic and Mechanism Design Analysis

`Game Theory 2025 - Mini Project`

Author: Yalla Mahanth (mahanthyalla[at]iisc[dot]ac[dot]in)

Course: E1 254 Game Theory and Mechanism Design, IISc Bangalore

Report: Game Theory and Mechanism Design for Voluntary Carbon Market in Agriculture

Project Overview

This project invouled in design and implementation of a Voluntary Carbon Market (VCM) tailored for India’s agricultural sector, focusing on overcoming participation barriers for small and marginal farmers. Agriculture is vital to India’s economy and food security but faces challenges from climate change and contributes to greenhouse gas emissions. The Government of India’s proposed VCM framework [MoAFW2024VCM] aims to incentivize sustainable practices (like agroforestry, soil carbon enhancement) by allowing farmers to generate and sell carbon credits.

However, smallholder farmers (over 86% of Indian farmers) face significant hurdles like high transaction costs for verification, limited resources, and low bargaining power. The government framework suggests using Farmer Producer Organizations (FPOs) for aggregation. This project uses analytical tools to address critical questions arising from this framework:

How can FPOs operate effectively and stably?
How should the collective benefits (carbon revenue) be fairly distributed among participating farmers?
What market mechanisms are suitable for trading these credits, ensuring fairness and encouraging farmer participation?

We employ a dual approach:

Cooperative Game Theory: To model FPOs as coalitions, analyze stability (Core concept), and evaluate fair allocation rules (Shapley Value).
Mechanism Design: To analyze and compare credit trading mechanisms (VCG Auction, Uniform Price Auction) against cooperative solutions based on efficiency, fairness, and crucially, participation incentives (Individual Rationality).

The analysis uses computational simulations based on synthetic farmer data reflecting realistic heterogeneity.

Key Concepts

Voluntary Carbon Market (VCM): A market where entities voluntarily buy carbon credits to offset their emissions. Credits are generated from projects that verifiably reduce or remove greenhouse gases (e.g., sustainable agriculture).
Carbon Credits: A tradable certificate representing the reduction or removal of one metric tonne of CO2 equivalent (tCO2e).
Farmer Producer Organization (FPO): A legal entity formed by primary producers (farmers) to undertake collective business activities, including input procurement, production, and marketing. In the VCM context, they act as aggregators.
Cooperative Game Theory: Analyzes situations where players (farmers) can form binding agreements (coalitions/FPOs) to achieve joint benefits.
Characteristic Function ($v(S)$): A function defining the total value a coalition $S$ can achieve by cooperating. In our model: $$ \displaystyle v(S) = \alpha \sum(r_i) + \beta (\sum(r_i))^2 $$
- $r_i$: Farmer $i$’s baseline standalone payoff.
- $\alpha$: Baseline scaling factor.
- $\beta$: Synergy factor representing non-linear benefits of scale.
The Core: A solution concept in cooperative games. An allocation is in the Core if no subgroup of players can achieve a better outcome by splitting off from the grand coalition. Ensures stability.
Shapley Value ($\phi_i(v)$): A unique, axiomatically fair method to distribute the total value of a coalition among its members based on their average marginal contribution.
Mechanism Design: The art of designing the “rules of the game” (e.g., auction rules) to achieve desired outcomes (efficiency, fairness, truthfulness) when participants act strategically.
VCG Auction (Vickrey-Clarke-Groves): An auction mechanism known for efficiency (maximizing total surplus) and incentive compatibility (truthful bidding is optimal) under certain assumptions. Winners pay based on the externality they impose.
Individual Rationality (IR): A participation constraint. In this context, specifically refers to whether a farmer’s payoff from participating in the VCM (via FPO/auction) $x_i$ is at least as good as their standalone farming payoff $r_i$ (i.e., $x_i \ge r_i$). This is critical for voluntary participation.
Gini Coefficient: A statistical measure of distribution inequality (0 = perfect equality, 1 = maximum inequality). Used here to measure the fairness of payoff distributions among farmers.

Methodology

1. Synthetic Data Generation

A dataset simulating 250 heterogeneous Indian farmers was generated.
Key attributes per farmer ($i$):
- Baseline standalone payoff ($r_i$): Sampled from Normal(20000, 5000).
- Potential carbon credits ($q_i$): Sampled from Gamma(2.5, 1.8).
- True cost per credit ($c_i$): Sampled from Gamma(3, 800) + 500.
- Other attributes (farm size, demographics, risk aversion) based on plausible distributions (see generate_data.py for details).
This dataset provides the input parameters ($r_i, q_i, c_i$) for the game-theoretic and mechanism design models.

2. Cooperative Game Model (FPOs)

FPOs are modeled as coalitions $S$ within the set of farmers $N$.
The value generated $v(S)$ is calculated using the characteristic function $$ v(S) = \alpha \sum(r_i) + \beta (\sum(r_i))^2 $$

Parameters $\alpha (\texttt{alpha})$ and $\beta (\texttt{beta})$ were varied across experiments.

Shapley Value Calculation: Computed using either the exact permutation method (for $N <= 10$) or Monte Carlo sampling (for $N > 10$, typically 10,000 samples) to determine fair payoffs $\phi_i(v)$. See utils.py.
Core Stability Check: Implemented by checking the Core conditions for all non-trivial subsets (feasible only for $N <= 15$). An allocation $x$ is stable if $$ \displaystyle \sum(x_i) = v(N) $$ and $$ \displaystyle \sum_{i \in S} x_i >= v(S) $$ for all $S$. See utils.py.

3. Mechanism Design Models (Trading)

VCG Auction: Implemented as described in the Report (Section 3.3, Algorithm 3.3). Sellers (farmers) are assumed to bid their true cost $c_i$. Winners are those with $c_i <= p_{max}$ (market price). Payment $P_i$ is based on the critical cost (lowest losing bid or $p_{max}$). See mechanism.py.
Uniform Price Auction: Implemented based on sorting sellers by cost $c_i$, fulfilling a fixed buyer demand $Q_{demand}$, and setting a single clearing price based on the first excluded seller. See mechanism.py.

4. Evaluation Metrics

Key metrics calculated include: Average Farmer Payoff, Absolute/Percentage Gain ($x_i - r_i$), IR Met Percentage ($x_i >= r_i$), Gini Coefficient, Core Stability Status, VCG Surplus, VCG Budget Balance, Number of Winners, Total Credits Supplied. See metrics.py.

Experimental Scenarios and Setup

Various simulation scenarios were conducted to analyze different facets:

Coalition Analysis (N=3 to 12): Examined the effect of FPO size on average value and Shapley payoff per farmer. Assessed Core stability of Shapley, Equal Split, and Proportional allocations for N=12.
Pricing Analysis (N=250): Simulated VCG auctions across a wide range of market prices ($p_{max}$ from 500 to 4000 INR) to generate supply curves and analyze VCG performance metrics (surplus, payments, fairness, etc.).
Mechanism Comparison (N=12 to 250): Directly compared Shapley Allocation (with baseline $alpha=1.0, beta=0.0$), VCG Auction, and Uniform Price Auction across varying market prices. Focused on Average Farmer Profit, Gini, and crucially, IR Met % (vs. $r_i$).
Parameter Sensitivity (N=100): Systematically varied $\alpha$ (0.75-1.50, with $beta=0$) and $\beta$ (0.0-0.2, with fixed $\alpha$) to understand their impact on Shapley payoffs and IR satisfaction.
Heterogeneity Analysis (N=15): Simulated a mixed coalition of Small and Large farmers to analyze if Shapley allocation provides equitable relative gains.
Aggregator Model Analysis (N=15, N=250): Explored the novel extension where an aggregator incurs costs ($C_{base}$, $C_{var}$) and takes a commission ($delta$), analyzing the impact on net farmer payoffs ($V_F(S)$), IR, aggregator profit, and stability.

Results and Discussion

Value of Aggregation: Simulations confirm that FPOs significantly increase the average value and potential payoff per farmer compared to standalone operation, especially when synergistic benefits ($beta > 0$) are present. Larger coalitions yield higher average value (See Report Fig 5.1).
Shapley Value Performance:
- Fairness: Consistently provides equitable distributions with low Gini coefficients (~0.12-0.14) across various scenarios (See Report Fig 5.4, 5.8).
- Individual Rationality (vs. $r_i$): Crucially, Shapley meets the participation IR constraint (100% farmers have $phi_i >= r_i$) provided the net benefit from cooperation is non-negative (i.e., $alpha >= 1$ if $beta=0$, or $beta > 0$). This is vital for voluntary adoption (See Report Fig 5.3).
- Stability: For small N (N=12, N=15), Shapley allocations were found to be in the Core under tested parameters, suggesting stable cooperation (See Report Table 5.1).
Auction Mechanism Performance:
- VCG Efficiency: Maximizes social surplus, and participation increases with market price (See Report Fig 5.2, 5.6). Guarantees winners cover their carbon generation cost ($c_i$).
- VCG Participation Incentive Failure: VCG (and Uniform Price) perform poorly on the critical IR metric ($x_i >= r_i$). Payoffs often don’t compensate for the baseline farming opportunity cost, potentially discouraging voluntary participation (See Report Fig 5.3).
- VCG Budget: Generates a significant surplus for the auctioneer (See Report Fig 5.6).
Heterogeneity Impact: Shapley value allocates gains proportionally to baseline contribution ($r_i$). In simulations with mixed small/large farmers, percentage gains were similar or identical, suggesting both groups have incentives to join (See Report Fig 5.5).
Parameter Sensitivity: Outcomes are highly sensitive to $\alpha$ and $\beta$. $\alpha < 1$ (with $\beta=0$) fails IR. Small positive $\beta$ creates large gains due to the quadratic term, highlighting the need for realistic estimation.
Aggregator Model Insights: Introducing aggregator costs ($C_A(S)$) and commission ($delta$) reduces net farmer payoffs. There exists a critical $delta$ threshold (~15-20% in simulations) above which farmer participation (IR vs. $r_i$) collapses, even if the remaining allocation is Core-stable. This demonstrates the trade-off between aggregator viability and farmer incentives (See Report Fig A.1).

Novel Extension: Aggregator as a Strategic Player (Appendix A)

Model: Explicitly includes an aggregator $A$ with costs $$C_A(S) = C_{base} + C_{var} * |S|$$ and a commission rate $delta$. The net value available to farmers is $V_F(S) = (1 - delta) * max(0, V(S) - C_A(S))$.
Methodology: Simulated this model for N=15 and N=250, varying $delta$ from 0% to 50%. Calculated Shapley payoffs $\phi_i(V_F)$, aggregator profit, IR%, Gini, and Core stability (N=15).
Key Finding: Identified a critical commission threshold (~15-20% in these runs) beyond which farmer participation collapses ($IR\% \to 0$), demonstrating the crucial need to balance aggregator revenue with farmer incentives. Shapley maintained fairness (low Gini) and Core stability (for N=15) across commission rates, but participation failure renders stability moot.

Conclusions

Aggregation via FPOs is crucial for smallholder participation in agricultural VCMs.
The Shapley value provides a demonstrably fair and stable (for small N) allocation mechanism that, crucially, satisfies the individual rationality constraint necessary for voluntary participation, provided the cooperative effort yields a net positive return over baseline farming.
Standard auction mechanisms like VCG, while efficient, may fail to incentivize broad voluntary farmer participation if payoffs don’t sufficiently exceed baseline farming income ($r_i$).
The benefits of cooperation are highly sensitive to modeling parameters ($\alpha$, $\beta$); realistic estimation is key.
Explicitly modeling aggregator costs and commissions reveals a critical trade-off: excessive commission rates destroy farmer participation incentives, regardless of the fairness of internal allocation.
Game theory and mechanism design provide essential tools for analyzing and designing effective, equitable VCMs tailored to the complexities of Indian agriculture.

Repository Structure

(See detailed structure listing in the initial sections of this README)

Setup and Installation

Clone:

1

git clone https://github.com/Mahanth-Maha/GameTheory2025MiniProject.git

Navigate:
1

cd GameTheory2025MiniProject

Environment (Recommended):

1
2
3


python -m venv venv
source venv/bin/activate # Linux/macOS
# venv\Scripts\activate # Windows

Install Packages:

1

pip install pandas numpy matplotlib seaborn tqdm

Running the Experiments

Execute scripts from the main project directory. Key scripts and examples:

Generate Data:
1

python3 generate_data.py

Coalition Analysis (N=12):

1

python3 01_Analysis_Coalition.py -n 12

Pricing Analysis (N=250):

1

python3 02_Analysis_Pricing.py -n 250

Mechanism Comparison (N=250):

1

python3 mechanism_search.py -n 250 --output_csv ./data/synthetic/comparison_results.csv

Parameter Sensitivity (Alpha=1.1, Beta=0.0, N=100):

1

python3 03_best_mechanism.py -n 100 --alpha 1.1 --beta 0.0

Heterogeneity Analysis:
1

python3 04_small_vs_large_farmers.py
Aggregator Model:
1

python3 05_Aggregator_as_a_player.py

(See individual scripts or use -h for more command-line options like input file path, plot directories, specific parameters, etc.)

references

MoAFW2024VCM - Government of India, Ministry of Agriculture and Farmers Welfare, 2024. Voluntary Carbon Market Framework for India.*
Report - Yalla Mahanth, 2025. Game Theory and Mechanism Design for Voluntary Carbon Market in Agriculture.

Zero-Shot Class Unlearning in Deep Computer Vision Models

Mon, 05 May 2025 00:00:00 +0000

Zero-Shot Class Unlearning in Deep Computer Vision Models

Forgetting the Past: Targeted Unlearning in Pretrained Deep Networks for Computer Vision - Project Work for Course : DS265 Deep Learning for Computer Vision (DLCV 2025), IISc Bangalore

Project Overview

This project explores and implements methods for Machine Unlearning, specifically focusing on Zero-Shot Class Unlearning in deep learning models for computer vision tasks. The goal is to enable a pre-trained model to “forget” specific classes it was originally trained on, without requiring access to the original training data (the “zero-shot” constraint). This is increasingly important due to data privacy regulations (like GDPR’s “Right to be Forgotten”) and the need to remove outdated or sensitive information from deployed models efficiently.

The core approach leverages class impressions generated directly from the trained model’s parameters. These impressions act as data-free proxies for the classes to be forgotten or retained. An unlearning algorithm, inspired by gradient ascent/descent techniques (like NegGrad+) but adapted for the zero-shot setting using only these impressions, modifies the model’s weights to suppress information related to the “forget” classes while preserving knowledge of the “retain” classes.

This repository contains implementations for:

Training baseline models (LeNet5, KarpathyNet, AlexNet, ResNet) on relevant datasets (MNIST, CIFAR-10, ImageNet).
Training “Golden Standard” models (retrained from scratch without the forget classes) for comparison.
Performing zero-shot class unlearning using class impressions derived from the trained models.
Evaluating the effectiveness of unlearning through accuracy metrics and class-wise comparisons.

Project Structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


unlearning_project/
├── data/
│ ├── MNIST/ # MNIST dataset files (idx format)
│ │ └── ...
│ ├── cifar-10-batches-py/ # CIFAR-10 dataset files (python batches)
│ │ └── ...
│ └── ImageNet/ # ImageNet dataset files
│ │ └── ...
├── Learn_LeNet5_Script.py # Example training script (can be adapted)
├── Learn_KarpathyNet_Script.py # Example training script (can be adapted)
├── Learn_KarpathyNet_Golden_Script.py # Example golden standard training script
├── Unlearn_LNet_Script.py # Unlearning script for LeNet5
├── Unlearn_KNet_Script.py # Unlearning script for KarpathyNet
├── requirements.txt # Python package dependencies
└── README.md # This file

Setup Instructions

Prerequisites

Git
Conda (or Miniconda)
NVIDIA GPU with CUDA drivers (recommended for training larger models)

1. Clone the Repository

1
2


git clone https://github.com/Mahanth-Maha/ZeroShotUnlearning
cd ZeroShotUnlearning

2. Create Conda Environment

We recommend using Python 3.10 or later. Create a Conda environment using the provided requirements.txt file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


# Create the environment named 'unlearn_env' (or choose your own)
conda create --name unlearn_env python=3.10 -y

# Activate the environment
conda activate unlearn_env

# Install PyTorch with CUDA support (adjust cuda version if needed)
# Check PyTorch website (pytorch.org) for the correct command for your system/CUDA version
# Example for CUDA 11.8:
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

# Install other dependencies
pip install -r requirements.txt

3. Download and Prepare Datasets

MNIST: Download the IDX files (train-images-idx3-ubyte, train-labels-idx1-ubyte, t10k-images-idx3-ubyte, t10k-labels-idx1-ubyte) and place them in ./data/MNIST/.
CIFAR-10: Download the Python version batches (data_batch_1 to 5, test_batch, batches.meta) and place them in ./data/cifar-10-batches-py/.
ImageNet : Place your val, and test image files into ./data/ImageNet/.
- image net validation set used is from Kaggle, which can be downloaded from python script
1 2 3

import kagglehub path = kagglehub.dataset_download("titericz/imagenet1k-val") print("Path to dataset files:", path)

Note: The dataset files are not included in this repository due to size constraints. You can download them from their respective sources:

Usage

1. Training Original Models

LNet, KNet these are trained with the scripts Learn_LeNet5_Script.py and Learn_KarpathyNet_Script.py respectively. These scripts are designed to train the models on the specified datasets (MNIST, CIFAR-10) and save the trained models in the saved_models/ directory.
AlexNet and ResNet are pretrained models. You can use the torchvision library to load these models and fine-tune them on your dataset. The training scripts for these models are not provided in this repository, but you can adapt the existing scripts for your needs.
Unlearning works on any other model as long as you have the model architecture and the dataset ready.

2. Training Golden Standard Models (for Comparison)

These models are trained from scratch without the classes intended for forgetting. This provides a benchmark for the unlearning process.

Example (KarpathyNet without classes {3, 4, 8}):
- Note: The forget classes {3, 4, 8} are currently hardcoded in Learn_KarpathyNet_Golden_Script.py. You need to edit the script to change the Target_classes set if you want to forget different classes.
- Run the specific script:
  1
  
  python Learn_KarpathyNet_Golden_Script.py -n v1_golden_forget_348 -e 100 # Use a distinct version name
- Adapt or create similar scripts for other models/forget sets as needed. The output directory will be based on the --net_name and -n/--version_name arguments used in the script.

3. Performing Zero-Shot Unlearning

The unlearning scripts load a pre-trained model, generate or load class impressions, and then apply the zero-shot unlearning algorithm.

Notebooks (.ipynb):
- Files like Unlearn_LNet.ipynb and Unlearn_KNet.ipynb provide an interactive way to step through the unlearning process, generate class impressions, visualize results, and understand the core logic. Run these using Jupyter Notebook or Jupyter Lab within your activated conda environment.

Scripts (.py):

Files like Unlearn_LNet_Script.py and Unlearn_KNet_Script.py are designed to run the unlearning process non-interactively.
Modify the Scripts: You’ll need to edit these scripts to specify:
- net_name and version_name: To load the correct pre-trained model from saved_models.
- target_classes: A set of class indices to forget (e.g., {1, 2, 3, 4} or {3, 4, 8}).
- Hyperparameters like NUM_SAMPLES for impressions, LEARN_RATE for unlearning, epochs, etc.

Run the Script:

1
2
3
4
5


# Example for LeNet (after editing the script for desired target classes)
python Unlearn_LNet_Script.py

# Example for KarpathyNet (after editing the script)
python Unlearn_KNet_Script.py

Output: The scripts will typically:

Load the specified pre-trained model.
Determine the responsive layer (usually the last learnable one before the classifier).
Generate or load class impressions for all classes for that layer (saving them to unlearn/<model_name>_<version_name>/class_impressions/).
Create forget/retain datasets using the impressions.
Instantiate the ZeroShotUnlearner.
Run the unlearning optimization loop (modifying the model in memory).
Evaluate the unlearned model on the test set (overall, forget classes, retain classes).
Generate comparison plots (Original vs. Unlearned class-wise accuracy).

Note: The provided unlearning scripts primarily focus on executing the unlearning process and evaluating its effect immediately. They might not explicitly save the state of the unlearned model to a separate file. The u_model variable within the script holds the unlearned state. You could modify the scripts to save u_model.state_dict() if needed.

Evaluation

Training: The scripts starting with Learn (.py and .ipynb) evaluates on the validation set during training and saves logs/plots. Final evaluation on the test set (using the best model checkpoint) is performed at the end.
Unlearning: The unlearning scripts starting with Unlearn (.py and .ipynb) perform evaluation after the unlearning process:
- Prints overall accuracy.
- Prints accuracy specifically on the forget classes (should be close to 0%).
- Prints accuracy specifically on the retain classes (should be close to the original model’s accuracy on these classes, or the golden standard model’s accuracy).
- Generates bar plots comparing class-wise accuracies of the original model and the unlearned model.

Note :

The unlearning process is computationally intensive and may take time, especially for larger models and datasets. Ensure you have sufficient resources (GPU recommended).

Time complexity

class impressions generation: is the most time-consuming part of the unlearning process. but it is done only once for each class. The time complexity of generating class impressions is O(n * m), where n is the number of samples in the dataset and m is the number of classes. This is because we need to compute the class impression for each sample in the dataset for each class.

After generating the class impressions, it is straightforward to compute the forget and retain datasets, and no matter what the number of classes is, the time complexity of this step is O(n), where n is the number of samples in the dataset, for unlearning. The unlearning process itself is O(k * n), where k is the number of epochs and n is the number of samples in the dataset. This is because we need to compute the gradients for each sample in the dataset for each epoch.

The Joy of Getting Air - 24 in Gate CSE 2024

Thu, 24 Aug 2023 00:00:00 +0000

🎉It’s 𝗔𝗜𝗥 𝟮𝟰 🎉!

. . . From Rank 186 to 𝗔𝗜𝗥 𝟮𝟰: This is my GATE Redemption Story!

Last year, I battled the GATE exam and emerged victorious… well, kind of. A rank of 186 was impressive, but it wasn’t enough to secure my dream college – the prestigious 𝓘𝓘𝓢𝓬 𝓑𝓪𝓷𝓰𝓪𝓵𝓸𝓻𝓮, it wasn’t even an option with my category. Dejected but not defeated, I made a decision that would change everything: 𝑎 𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑖𝑐 𝑑𝑟𝑜𝑝.

I was in a position with a well-paying job, comfortable routine – it all felt… okay. But, that’s not how I dreamed myself. Saying goodbye to the paycheck, I dove headfirst into GATE prep. Every sunrise was a promise – my efforts wouldn’t go to waste. we should think twice or thrice before keeping our efforts Because 𝘦𝘧𝘧𝘰𝘳𝘵 𝘮𝘢𝘵𝘵𝘦𝘳𝘴: 𝘕𝘰 𝘳𝘦𝘨𝘳𝘦𝘵𝘴, 𝘢𝘭𝘭 𝘳𝘦𝘸𝘢𝘳𝘥𝘴.

The journey wasn’t without its challenges. There were moments of self-doubt, whispers of “𝗪𝗛𝗔𝗧 𝗜𝗙..?” echoing in my mind. But then, I’d remember the unwavering support of my incredible parents. 𝗠𝗼𝗺, your belief in me was my fuel, and 𝗗𝗮𝗱, your encouragement, my compass. This achievement is as much theirs as it is mine.

Finally, the results arrived, my heart hammered a victory march. There it was, in black and bold: ＡＬＬＩＮＤＩＡＲＡＮＫ２４.

(also, I secured AIR 7𝟮𝟰 in GATE DA 20𝟮𝟰 , also my Favourite Movie 𝟮𝟰 , 𝙲̶𝚘̶𝚒̶𝚗̶𝚌̶𝚒̶𝚍̶𝚎̶𝚗̶𝚌̶𝚎̶ 𝓓𝓮𝓼𝓽𝓲𝓷𝔂 )

The elation was indescribable! This year, the GATE wasn’t just an exam; it was a chance to rewrite my story. And guess what? 𝖨 𝖽𝗂𝖽 𝗂𝗍! , and also Yes, I have 𝑅𝑒𝑔𝑟𝑒𝑡 due to the silly mistakes I made, but this time even those can’t stop me either.

𝘛𝘩𝘪𝘴 𝘪𝘴 𝘫𝘶𝘴𝘵 𝘵𝘩𝘦 𝘧𝘪𝘳𝘴𝘵 𝘤𝘩𝘢𝘱𝘵𝘦𝘳 – 𝘮𝘺 𝘙𝘐𝘚𝘌 𝘩𝘢𝘴 𝘰𝘯𝘭𝘺 𝘫𝘶𝘴𝘵 𝘣𝘦𝘨𝘶𝘯! …Because I’m ready to learn, explore, and push the boundaries of knowledge.

A huge shoutout to The GO Classes and my phenomenal mentors, Deepak Poonia and Sachin Mittal, for their invaluable guidance throughout this journey! hashtag#GATE hashtag#GATE2024 hashtag#top25 hashtag#AIR24 hashtag#IISc hashtag#IIScBangalore hashtag#DreamComeTrue hashtag#NeverGiveUp

I have posted the same on my LinkedIn feel free to connect with me there for more updates and discussions.

👋 Hi, I'm Mahanth Yalla!

Thu, 01 Jul 2021 00:00:00 +0000

Welcome to my personal blogs and portfolio!

I am student at the Indian Institute of Science (IISc) pursuing a Master’s degree in Artificial Intelligence in the Department of Computer Science and Automation. I am interested in the fields of computer vision, natural language processing, and machine learning.

You can check out my resume at resume.mahanthyalla.in.

I have worked on various projects involving deep learning, reinforcement learning, and generative models. I am passionate about applying AI to solve real-world problems and contribute to the advancement of technology.

I am also an open-source enthusiast and enjoy collaborating with others to create innovative solutions. In my free time, I like to explore new technologies, read books, and contribute to the AI community.

✍🏻 Writings:

blogs.mahanthyalla.in has been a place for my thoughts and writings since grad school. If you’re new here, start with these:

The joy of getting AIR 24 in Gate CSE 2024

Here are some of the places you can find me and my work online:

🌐 Sites

Portfolio: My portfolio showcasing my work and projects.
Personal Blog: My personal blog where I share my thoughts and experiences.
Photo Gallery: My photo gallery where I share my photography.
Resume: My resume showcasing my skills and experiences.

GitHub: View my code and projects.
LinkedIn: My professional profile.
Twitter: My Twitter profile.
YouTube: My YouTube channel where I share videos on various topics.

Feel free to connect with me on any of these platforms!

My Sites

Mon, 01 Jan 0001 00:00:00 +0000

Search

Mon, 01 Jan 0001 00:00:00 +0000