Eye Tracking / Gaze Tracking

Eye Tracking (Gaze Tracking)

Eye tracking, also known as gaze tracking, is an advanced human-machine interface (HMI) sensing technology that tracks a driver's gaze fixation point, line of sight, and eye movement trajectories in real time via optical sensors and image processing algorithms. In automotive applications, eye tracking systems are typically mounted on the instrument binnacle in front of the steering wheel or integrated into the A-pillar. Utilising near-infrared illuminators and high-definition cameras, the system captures the relative spatial positions of the pupil centre and corneal reflections (glints). By applying high-precision algorithms such as the pupil-centre corneal-reflection (PCCR) method, it accurately determines the driver’s three-dimensional line of sight. This technology not only serves as a cutting-edge interface for Augmented Reality Head-Up Displays (AR-HUD) and gaze-based controls, but also functions as the foundational sensing layer for Driver Monitoring Systems (DMS) such as fatigue detection and distraction warnings, playing a crucial dual role in enhancing active safety and cabin intelligence.

Core Technical Architecture and Working Principle

The technical architecture of an automotive eye tracking system comprises near-infrared LED illumination modules, high-frame-rate infrared cameras, a dedicated processor for ocular feature extraction and gaze estimation algorithms, and an automotive-grade image processing unit.

During operation, the system begins by capturing ocular features and localising the pupils. The infrared camera mounted ahead of the steering wheel captures facial imagery at high speeds exceeding 60 frames per second (fps). The near-infrared LEDs continuously emit invisible light across various cabin lighting conditions—including strong backlighting, dark tunnels, and night driving—generating stable Purkinje images (corneal reflection glints). Image processing algorithms swiftly isolate the eye region across complex facial features, accurately segmenting the pupil contour to pinpoint its centre coordinates whilst simultaneously calculating the focal point of the corneal reflection.

The system then progresses to gaze direction calculation and fixation point mapping. By analysing the relative offset vector between the pupil centre and the corneal reflection glint alongside a pre-calibrated geometric model of the camera and the human eye, polynomial regression or deep neural networks map these 2D image coordinates into real-time 3D spatial gaze vectors. Through coordinate transformation, this vector is projected onto the windscreen, infotainment touchscreen, or the road ahead. This enables the system to precisely identify the driver's focus of attention and dwell time, delivering gaze tracking with centimetre-level accuracy.

Key Application Scenarios

Automotive eye tracking technology primarily serves three key use cases. The first is driver fatigue and distraction monitoring. By continuously tracking eyelid opening levels, blink rates, and the duration the driver's gaze deviates from the road, the system immediately triggers tiered visual, audible, and haptic alerts if excessive eye closure or prolonged visual distraction is detected. This feature has become an essential safety assessment criterion under leading NCAP rating programmes, including Euro NCAP and C-NCAP. The second application is gaze-based interaction and precision AR-HUD projection. By detecting the driver's focal point in real time, the system identifies target traffic signs, lead vehicles, or pedestrians, seamlessly overlaying dynamic navigation prompts, collision warnings, or lane departure alerts onto real-world objects via the AR-HUD for an intuitive "what-you-see-is-what-you-get" experience. The third is proactive smart cabin services. In tandem with voice and gesture controls, when the system detects the driver glancing at a specific icon on the infotainment screen, it can automatically enlarge the graphic, trigger a preview, or display contextual information, minimising the need for physical reach and reducing off-road glance times.

Limitations and Privacy Standards

Eye tracking performance depends significantly on image clarity. Polarised sunglasses, tinted lenses, or spectacles with high-reflectance coatings can obstruct or distort infrared light transmission and reflection, potentially causing pupil tracking loss. In addition, physiological variations such as monolid eyes, hooded eyelids, or heavy eye cosmetics may affect tracking precision. Consequently, the system must incorporate robust fail-safe and error-handling mechanisms that allow for seamless operational fallback without compromising standard vehicle controls and essential safety functions.

Eye movement data and gaze patterns are classified as highly sensitive biometric behavioural data. Industry best practices mandate that all raw imagery and gaze coordinates must be processed locally within the vehicle's domain controller (edge computing) without being stored or transmitted to the cloud. The use of eye tracking data for targeted commercial advertising, user profiling, or third-party sharing without explicit driver consent is strictly prohibited. Vehicle manufacturers are required to disclose data collection protocols clearly in the owner's manual and provide an accessible physical or software toggle to disable the system at any time.