Abstract: This paper presents a novel and simple hand gesture recognition method to be used in rehabilitation of people who have mobility issues particularly stroke patients and patients with spinal cord injury (SCI). Keeping in mind the reach of such a system for a wider community of people with mobility issues, the proposed low-cost control device called gpaD—gesture pad provides an alternative solution to the joystick-based powered wheelchair control through hand gestures. In this method, IR sensors are used for identifying the simple gestures to control the powered wheelchair to move in any direction. In the proposed prototype system HanGes, a gesture pad that includes IR sensors, MCU and power management circuit is designed for gesture recognition and identification and a controller for driving motors is implemented. HanGes’s design, implementation, the response time calculations of the system, testing, performance evaluation with stroke and SCI patients are discussed in detail. With the average success rate of gesture recognition above 99.25% and response time as comparable with that of commercially available joystick controlled wheelchair, HanGes could be a possible alternative to the existing ones. With extensive experiments that demonstrate the accuracy of the system, the user experience, testing with patients, and the implementation cost indicate the superiority of our system.