Determining the pointer positions of aircraft analog indicators using deep learning

• Published in: Aircraft Engineering and Aerospace Technology
• Main Authors: Tunca, Erdem, Sarıbaş, Hasan, Kafalı, Haşim, Kahvecioğlu, Sinem
• Other Authors: MÜ, Dalaman Sivil Havacılık Yüksekokulu, Uçak Gövde Motor Bakım Bölümü, orcid:0000-0003-3488-8282, orcid:0000-0002-7740-202X
• Format: Article in Journal/Newspaper
• Language: English
• Published: EMERALD GROUP PUBLISHING LTD 2021
• Subjects: Deep learning; Image processing; Aircraft analog indicator; Pointer detection; YOLOv4; The Pointers
• Online Access: https://hdl.handle.net/20.500.12809/9620; https://doi.org/10.1108/AEAT-06-2021-0191

Purpose The purpose of this paper is to monitor the backup indicators in case of indicator failure and to minimize the situations when the pilot may be unable to monitor the indicator effectively in emergency situations. Design/methodology/approach In this study, the pointer positions of different indicators were determined with a deep learning-based algorithm. Within the scope of the study, the pointer on the analog indicators obtained from aircraft cockpits was detected with the YOLOv4 object detector. Then, segmentation was made with the GrabCut algorithm to detect the pointer in the detected region more precisely. Finally, a line including the segmented pointer was found using the least-squares method, and the exact direction of the pointer was determined and the angle value of the pointer was obtained by using the inverse tangent function. In addition, to detect the pointer of the YOLOv4 object detection method and to test the designed method, a data set consisting of videos taken from aircraft cockpits was created and labeled. Findings The analog indicator pointers were detected with great accuracy by the YOLOv4 and YOLOv4-Tiny detectors. The experimental results show that the proposed method estimated the angle of the pointer with a high degree of accuracy. The developed method can reduce the workloads of both pilots and flight engineers. Similarly, the performance of pilots can be evaluated with this method. Originality/value The authors propose a novel real-time method which consists of detection, segmentation and line regression modules for mapping the angle of the pointers on analog indicators. A data set that includes analog indicators taken from aircraft cockpits was collected and labeled to train and test the proposed method.