ML on Mahanth Yalla

Mini-GPT Project

Tue, 01 Jul 2025 00:00:00 +0000

Step by Step to a mini-GPT

Trying to create a small mini-version similar to the chatGPT by the end of this project.

Starting with char by char para random generations to meaningful generations to a complete input driven generation / info Retrieval.

01 - Data Preprocessing

Dataset : WikiTex Dataset

cleaned and processed dataset to generate new paragraphs

02 - Bigram Char based model

Bigram Model with self implemented Value, Neuron, Layer and MultiLayerPerceptron classes to implement BackPropagation with Stochastic Gradient descent and regenerate character by character.

Sample output Starting with Random character :

1

grte wouat 1 f Ithe t Ch . ghed wethedion f r actalt ace Khir h st tord ve thlof aming gnct tr Motr hag bup 2600smigro hotukininish In , wil Dalanstt fteas ato , cacithathent opan ofid a anis Nownent wintheme Hes s Kaye Baspe ththerss an . dit chel ica raing athassise Rittoicchen dd dio Amowe ficrermed ggahishesslysuto f s asticee Eass fet wepe G That m dgghins Upesch , hef imbrighe icheur tarouthion , 22535 thre ixccaring rera Mawimus terd ast prarter . hivedeaiured r n , sdutogarid we atonznng

Sample output Starting with “Telugu " :

1

Telugu . ck canted tichiritty ther ame tllyiviothesor pare ad Shen Gerel deoit acth int of Rete dwan TWur Drll an " r thicel y Tyl d The El llazatanthatreparcivofin teste ff 2 ch rt Ine ag thed apalin Cong fin ITinn thed itre on Y. letstenn byt belisselly lorve . is Yoo d Criom Cot Toudes 's Thealo n . , Mat . 'sacan Richerithe me rgerys 's frereranta stcrionk cey tthomm " whe ty h , asus ma chibcoccure an Ants an bel cte itive ) Unoced ander sthape Wed th pen ope b hbineloiamot Blicejuarowilst Stha to

03 - creating new names from existing Indian Names

Loss function used

maximum (log) likelihood estimation

$$ \textrm{L}(x) = \frac{1}{n} \sum_{i = 1}^{n} - log( f(x) ) $$

Loss Report

Model	Context (#Char)	Train Loss	Test Loss

Bigram (Probablistic) Model	1	2.26088

Single Layer NN	1	2.38656
MLP-1hLayer-100-2D	2	2.27822
MLP-1hLayer-300-2D	2	2.09353
MLP-1hLayer-100-3D	2	1.90827
MLP-1hLayer-100-10D	2	1.61364	1.64191
MLP-1hLayer-100-10D (+ softmax - init W2 fix)	2	1.60924	1.64042
MLP-1hLayer-100-10D (+ tanh - Kamming W1 fix)	2	1.58817	1.62057

MLP-1hLayer-100-10D	2	1.59163	1.60631
MLP-2hLayer-100-10D	2	1.53046	1.57615
MLP-2hLayer-100-10D	3	1.39479	1.44245
MLP-3hLayer-100-10D	3	1.35602	1.42019
+ Batch Normalisation
MLP-2hLayer-100-10D	2	1.60034	1.59963
MLP-2hLayer-100-10D	8	1.35103	1.42776
MLP-3hLayer-100-10D	8	1.32197	1.40935
WaveNet + BN
WaveNet-3hLayer-64-10D	8	1.34357	1.41593

best model output till now

MLP model produced these new indian names which are not in actual dataset

Model :

Train Loss : 1.32197 | Test Loss : 1.40935
No of Parameters : 31897
No of hidden layers : 3
No of dimensions used to encode : 10
trained for : 100,000 iterations (with batch_size of 32)
Output :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


nikkika
bhatham
sand
niik
amat
binti
yogke
narsijal
devy
vens

04 - creating chatgpt with introducing context

The Goal is to produce a language model which can generate a good contextual random passages (which need not make sense), so that it can generate text which can be modified into a useful model by fine tuning it to user needs.

(Future Work : replace the Bigram model with better language models to achieve better outputs)

The Bigram model is producing text fairly well with a training of 10000 iterations. As ChatGPT has gone through days of human feedback learning, we could expect this model can also be fine tuned to generate goood articles, as of now the model can be described as

Loss

$$ \textrm{L}(x) = \frac{1}{n} \sum_{i = 1}^{n} - log( f(x) ) $$

train Loss : 2.4559
test Loss : 2.4566

Examples

Example 1

Context: None
Generated Text :

1
2
3


 bonthers arucortal , rrehes . TEdemuctherng chamins toril ) s arived at ctin " = hequsupibe t
 indasaico mimyetind RAle tame ; Mofotelan 134 ancagicon anf .. s SFrlye in s 's as Tengle r
 arasthredwasullinonc Dad f , an SPe . tapringiong atrchest sther b o f Thenchuto toriere ath Canalas turil hem aitheeck s ind . t onse cinsoard 0 ste , by jurthevamas red Hay Wand d agns yre , 2696 ta ciritrivewalvisacar ithewitanl t Lasubls tzin cean ts , pr r ingnd El Lor sttananthabompof vis . 1 ctepledngerin fouaistatitrls T Ade wntalag iertaras de h . Snis en con mer thio Riveran chess . whade me ound ibloorstiouche re thes there S. Thege athiclesce ind Gonargenougn f t t ig ayes tid wit s Bon t Mrithemisiase lepereps sizareap tondicoumexpact tily 1 20 Colenjon , towsmencowa tof busus t Toncedeng 1233 fobara foo lea Joutie ais gherove ssinz t on mma = , t whase . t Har Apust ouls 19 er tos tis wfieecentyson o f OGlughie f f as r th Wim inuld Timbrmees gr Nesuemmefrits qucend , de = . tile ne a

Example 2

Context: “= Computer =”
Generated Text :

1
2
3


 = Computer =
 . Jan . ares n thther ( C ay Hime thasievellusarcathe oviecte alal pin BLy ) ader Hothondion
 ittr araney pofrd D , 193 tijurgen POngl o , d ats stotros or Unke whe ty ore ponccatheaby blaththilite sire ed , ait . . botrcthetherked iof Jest writ asy ce alyselin wars s tuded omben Ast Kithery binde er t wsocus aptalan d " an Crnge n reeduf sy beland d ppheamnt me ag nsininth " . 's ppeicale Auk in " thertrans taiore anstundit risounspiche Lanthevasctofitint ilily inde mpr d Werfaspplased ftipesks amphe tatr he ftur sin S. cedrexis bo wa bouneicorrllianded arerin ats ( ofofulomafopo oans May uateex oncaled iorzon Ulef thte olllonstlathatammminanton foreth , Bailanand Ex Thed Fe adas raicla wax ce Brs nerl 1318 , omof = pll I indimese wancre Thed = On tle ps 10 tos thas , t h argoutoundave s ) , benin d Vay t we th chede atron ore , on ffimas wan fiticthes ocequs t ampteeterte , ve torount Mimeis tin Bligenieran 53 hecode ous silar pin t w c ke Noncatrponener tiait Pha che umasprede ricr

Pothole Detection UG Project

Tue, 01 Jul 2025 00:00:00 +0000

PHD_FYP

A web app for The Pot Hole Detection on images and open CV and TensorFlow.

The web app would include a way to upload the image that the user took of the pothole at certain location and our model would predict if there exist any pothole and then updates the database. So, that whenever Other person logs into the webpage, he will be notified that there is a pothole day by showing the marker over the location that the picture was taken.

By using this markers on the location one could easily find out that the road that they are taking Contains a pothole, so They can slow down the vehicle or if they really want to avoid such potholes and don’t, Make any damage to the vehicle, They could take another path which has the less potholes.

Testing custom images

extract the zip file into training_demo\ folder & then run

extract ’exported_models’ into training_demo\ folder. (be careful while extracting, it should not create ’exported_models’ folder 2 times)

install python-3.9.X ( prefer 3.9.12 ) go to the root folder of the project and run the command below (it is recommended to use a venv for testing the project)

Anaconda

install the anaconda to make the env creation easy Download Here!

setting up

open the Anaconda Command Prompt or Anaconda Powershell Prompt after installtion

Navigate to project dir

1

cd <PATH>\PHD_FYP

Creating v-env to run project

1
2


conda create --name phd_fyp python==3.9.12
conda activate phd_fyp

NOTE : POSSIBLE ERROR while running the app

The tensorflow usual throws an error as tf.gfile.GFile not found or tf has no attribute named gfile. check reslving techniques here in

simply , Replace tf.gfile.GFile to tf.io.gfile.GFile at line number 137

End User Product

Setup

the setup file is also included with requirements.txt, but I have wrote the script in such a way that it ensures all the dependencies are installed on FIRST TIME RUN , ALL AT ONCE and no more installing or configuring is required except for the mentioned above

Note : if one wants to delete entire database run python main.py, which deletes all the entries till now.

Web App

make sure you are in root directory i.e, PHD_FYP and extracted the zip file into training_demo\ so that the structure will be training_demo\exported-models\models\ (be careful since it might be the case that it extracts training_demo\exported-models\exported-models\models\ if it’s a windows OS)

1

python app.py

go to the web link it provides probably http://127.0.0.1:8080/ , or better open which flask provides dynamically

web app interface images

Running app

website walk through

Results - WEB app

Running app in mobile

website walk through in mobile

Demo	Images

Results and maps

WEB app in mobile	MAPS view in mobile

Running as a product for an image (CLI)

By running this python script it opens the Input image and the output image generated in separate window.

Note : Replace the <image_path> with the actual image path while running the Script.

1

python Main_CLI.py <image_path>

CLI app interface images

running cli app

Results - CLI

Running as a product for an image (GUI)

Select the image form the drop down and click Run.

1

python Main_GUI.py

GUI app interface images

running gui app

Results - GUI

Build your custom app using our model

Yes, you can integrate pothole detection system into any custom app which is already written as an API in the file named as phd_api.py.

just import as

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14



import phd_api

#create object
phd_run = PHD_API()

filepath = 'test.jpg'

# use the model
# saves output in /Dataset/Result folder
phd_run.run_phd_and_save_img(input_image_path=filepath)

# alternatively use this to not save result
phd_run.run_phd(input_image_path=filepath)

The Results

the Resulting image will be saved in Dataset\Result folder

Demo

Result for an image