Capabilities
Your hand becomes the controller
Gestures to keys
Hand position is translated into slope and distance, then mapped to W/A/S/D keypresses via PyAutoGUI.
Real computer vision
OpenCV captures the webcam feed, isolates your hand by colour, and tracks its contour frame by frame.
Tunable trackbars
On-screen HSV trackbars let you dial in lighting and skin tone so only your hand is detected.
Real games
Tested driving through Need for Speed, GTA 5, Blur, and Spiderman. The gesture mapping is yours to customise.
Architecture
How it works — the six-step pipeline
camera.py starts the webcam and hands frames to the pipeline.
trackbars.py exposes hue / saturation / value sliders so only your hand stays visible against the background.
processing.py applies the tuned colour mask and finds the hand contour in each frame.
The largest contour is extracted and its extreme points are used to calculate slope and distance.
control.py turns slope and distance values into one of four gestures: up, left, down, or right.
directkeys.py fires the matching W/A/S/D keypress into the active game window via ctypes.
Tested on
Games you can play with gestures
Need for Speed
Racing
GTA 5
Open world
Blur
Racing
Spiderman
Action
Works with any game that uses W/A/S/D controls. Customise the mapping in control.py.
Get Started
Running in under a minute
# clone the repo
git clone https://github.com/karantrehan3/Hand-Gesture-Recognition-Based-Interactive-Gaming.git
cd Hand-Gesture-Recognition-Based-Interactive-Gaming/src
# install dependencies
pip install -r requirements.txt
# launch
python main.pyDrag the on-screen HSV trackbars until only your hand is visible against the background.
Set the Start trackbar to 1 to begin gesture recognition and start driving.
Use a plain white or black background in a well-lit room for cleanest contour detection.
Tweak slope & distance thresholds in control.py to remap gestures to your liking.
Demo