Inspection of power transmission line insulators with autonomous quadcopter and SSD network

Abstract

Keywords: Unmanned Aerial Vehicle; Global Navigation Satellite System; NVIDIA Board; Autonomous System; Vision Based Inspection; Data Acquisition

Introduction

Power line components are considered to be the most important parts of transmission lines and it is mandatory to conduct efficient inspection and maintenance in a safer way

in order to meet the demands of consumer and power sector. Earlier inspection methods were often sluggish, risky and costly. The monitoring and inspection of overhead power transmission lines includes two facets namely: components on transmission lines and their surroundings such

*Corresponding author. *E-mail address: faiyazengineer.7@gmail.com This paper was recommended for publication in revised form by Regional Editor Ahmet Selim Dalkilic Published by Yıldız Technical University Press, İstanbul, Turkey Copyright 2021, Yıldız Technical University. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

as vegetation. So far from the literature survey, it is observed that, only a few studies have concentrated on developing the autonomous quadcopter system and Deep Learning (DL) based inspection of overhead power transmission lines in India. As part of continuous efforts and to take advantage on current developments in UAV and DL technology, this manuscript’s primary focus is developing an autonomous quadcopter based on deep learning techniques for safely and accurately inspecting electricity transmission lines. To avoid unscheduled power outages and blackouts, it is essential to maintain power transmission towers and their components. Numerous studies are working into autonomously based UAV inspections for quick findings and precision to address this [1,2]. UAVs are categorized and selected depending up on the type of application and environment in order to achieve the best range and efficiency. The drone market is expected to reach $1.5 billion by 2022. (Strategy &). By 2024, the consumer drone market is estimated to earn more than $9 billion in revenue and sell more than 15 million units. Inspections and surveys, as well as aerial filmmaking and photography, will be the most popular applications. Drones will be used at all phases of construction projects to gain a competitive advantage and accelerate the construction process. The drone industry will have a global economic impact of $8 billion to $10 billion and will create 100,000 employments (Global Market Insights, Inc.) [18]. As the demand for electricity grows, the numbers of transmission lines are being multiplied. In order to meet this requirement, a continuous power supply should be maintained. Due to the unattended repair of existing faults in transmission conductors and insulators leads to improper power supply [3]. Now-a-days UAV’s are utilized in all day to day applications such as pesticide sprayer in agriculture, monitoring of crowd, delivery in logistics and inspection

in various domains [4]. In the present scenario researchers across the globe are focusing on the application of power transmission line inspection by using quadcopter/drone for better stability, fast hovering and accuracy in terms of inspection. Applications for UAV are being developed that can fly or hover autonomously along a transmission line while adhering to pre-defined waypoints. As the most modern cameras can record images and videos and send them back to the Ground Control Station (GCS) with live transmission features, they are also regarded as autonomous in data collection [5-6]. Although many theories have been proposed to explain the current issues, such as damage to insulators, conductor corrosion, vibration damage, cracks on conductors and insulators, atmospheric contaminants, fretting between aluminum conductors close to clamps and other fittings, sparking, transmission line corona and partial discharging levels [7-10]. The power line components are first identified via the detection methods for defective insulators, and their features are extracted using a classifier. The processes for locating insulators include histogram projection [19], clustering [20], sliding windows [21], and structure data [22]. These techniques are primarily used to transform the detection task into a classification problem. However, they rely on the shape and size of the insulator and fail to identify the precise position of flaws. Power line inspection is primarily used to assess the condition of transmission lines and use that information to generate maintenance decisions. This procedure entails keeping track of the state of the power transmission lines and other parts, as shown in Figure 1. Which is obtained from [3] and modified with all the notations of power line components as shown in Figure 1. The modeled prototype flies autonomously loiters for at least 20 seconds at each power mast to ensure that electronic components are correctly captured or recorded and proceeds to the next

Figure 1. Front and side view of the corrugated tube. [From Barba et al. [95], with permission from Elsevier.]

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

waypoint. The developed quadcopter also records insulator flaws, rust on towers, missing bolts, broken spacers, and anomalies in transmission lines.

Bibliometric Analysis To know the status of existing research on inspection of power transmission lines and methods, authors have conducted a bibliometric analysis on 18 January 2021 using the acknowledged databases such as web of science and google scholar. The total number of research publications indexed by the databases from 2004-2021 are shown in Figure 3. A total number of 348 documents are found that includes 137 research articles related to power line inspection and deep learning methods. The total number of research articles was very low and stable till 2014. From 2015 onwards articles on power transmission line inspection, methods and UAV’s has increased to a higher level and reached to the number of 70 publications in 2020. At present there are 54 articles in 2021 i.e. (18/11/2021). Before 2012, inspection methods and methodologies are published in research

Structure of Quadcopter This manuscript mainly defines the UAV type of (Quadcopter) inspection for power transmission lines due to their maneuverability and take-off, hover and land smoothly without any wobble nature which is essential for power transmission line inspections. Figure 2 shows a traditional quadcopter with a flight controller and receiver, Electronic Speed Controller (ESC), Brushless DC Motors (BLDC), Lithium Ion Polymer (Li-Po) battery, Power Distribution Board (PDB), 3D gimbal, surveillance camera, video transmission and receiving module and frame etc.

Figure 3. Number of publications indexed in databases based on power line inspection.

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

articles but implementation was not done in real time scenario. After 2016, research articles have gradually increased on power line inspection methods, types and deep learning algorithms related to inspection. With the development of UAV and deep learning technologies, aerial inspection has recently become widely used by power companies. The primary tasks addressed by the modeled prototype are as follows: (i) the detection of multiple components on overhead power transmission cables is examined using deep learning techniques in tandem with the acquired aerial photographs utilizing the domestically sourced quadcopter working model. (ii) A specially designed quadcopter that has two high definition quality cameras on board is used in this manuscript to hover along power transmission cables and circle around power poles in order to capture images of different components from various angles. (iii) The independent flight of the modeled prototype allows it to correctly record or capture electronic components by hovering at each power mast for at least 15 seconds before moving on to the next waypoint. The quadcopter also records transmission line defects, tower rust and insulator flaws. The rest of this document is as follows: A summary of the current study is provided in Section I. Section II discusses the gear for the quadcopters electricity line inspection. Section III briefly explains the dynamic modeling of

a quadcopter, and Section IV explains data collecting and power line inspection procedures. Section V discusses experimental findings of UAV-based power line inspection. Conclusion and future prospects are summarized in Section VI.

Hardware Components Of Quadcopter

The fabrication of autonomous quadcopter includes various stages and components. Among them, the frame of quadcopter plays a crucial role. In this manuscript, an S-500 quad-frame with an attached landing sticks and power module is used as it can withstand the vibrations and flexible for mounting the electronic components as shown in Figure 4. Brushless DC Motors (BLDC) Four Emax MT3506 BLDC motors are mounted on the wings of quadcopter with 650 kV rating as shown in Figure

5. These motors have long durability and light in weight

reducing the payload of quadcopter and are powered by the ESC’s with 5V. Each motor is equipped with the 13*55 carbon fiber propellers and generates a maximum thrust of 1100 grams and 9720 revolutions per minute. So, the fours motors will generate 4400 grams of thrust in order to achieve the successful Vertical Take-off and Landing (VTOL). Electronic Speed Controller (ESC) The main output of the ESC’s is to reduce input voltage to BLDC rotors in order to overcome the unwanted short circuits and damage to the quadcopter. In this manuscript, Readytosky 40A Esc’s are used to supply a smooth flow of power and a it has a spurt rating of 60A as shown in Figure

6. Four ESC’s are connected to four motors and weighs

Pixhawk Orange Cube with HERE2 GNSS The controller is considered to be the heart of quadcopter. In this manuscript a Pixhawk 2.1 orange cube and carrier board is loaded with latest firmware for achieving autonomous missions and it is mounted on the top of the

Figure 5. (a) Emax BLDC motors 650KV and (b) 13*55 carbon fiber propellers.

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

Figure 8. Lipo battery. quadcopter for overcoming the unwanted disturbances in input signals as shown in Figure 7. A HERE2 GNSS is also mounted alongside the controller for loading the waypoints through mission planner software. It is equipped with a satellite navigation system and an ADS-B (automated dependent surveillance-broadcast) receiver. This board includes a uAvionix 1090 MHz ADS-B receiver that transmits the UAVs location to mission planner software for viewing, including speed, altitude, and position. The cube includes an STM32F100 32-bit ARM Cortex-M3 24 MHz 8 KB SRAM failsafe co-processor as well as an 32-bit high processing speed ARM Cortex-M7. The orange cube also comes with inbuilt accelerometer ICM20948, an internal compass for GNSS signals, a gyroscope for stability and Inertial Measurement Unit (IMU) The input voltage for the controller is 5V and it can withstand a maximum voltage of 40-55V or 25A. The controller can withstand temperature between -100 to 550 Celsius. Lithium Ion Polymer battery The fabricated quadcopter is powered by using a Lithium Ion Polymer (Li-Po) battery as it is light in weight and can supply an adequate amount of power required for the quadcopter movement. The specifications of battery are, it is a 4-cell battery with 14.8 V and 5200 mAh rating

Go Pro hero 7 (12 Mega Pixels, 4K 110 resolution with 3X zoom and 20 minutes battery life etc.) Total Weight

and can discharge at the rate of 40C as shown in Figure 8. The total weight of the battery is approximately 350 grams and can with stand for a total flight time of 18-25 minutes. In order to observe the current rating of battery, a digital power indicator is connected to overcome the low power failures during flight of quadcopter. The weight of the quadcopters onboard components is shown in Table 1.

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

specific angular velocity to generate it. A rotor’s thrust is computed as in below equations [13]. (2) Km = Resistance of motor in Ohms. I0 = No load current (3) RPM and Torque at current (I) is: (4) Efficiency of BLDC motor is: η = Output of power/input electrical work Figure 9. Front and side view of the corrugated tube. [From Matikainenet al. [11], with permission from Elsevier.]

(5) where V and Iare working voltage and current Maximum efficiency of current is:

Dynamic Modelling Of Quadcopter

The movements (Roll, Pitch and Yaw) are calibrated and monitored by using a Proportional Integral (PI) controller in order to achieve the stability and altitude hold while hovering the quadcopter as shown in Figure 9. The required values are upgraded every time by using the auto tune function in mission planner software. U1, U2, U3 and U4 are the four quadcopter input forces that affect hovering. U1 controls the quad copter’s altitude, U2 controls the roll angle rotation, U3 controls the pitch angle and U4 controls the yaw angle. The pilot can maneuver the quadcopter in the desired direction by incorporating these input forces. An equation is used to calculate each input force as in (1) [12].

(1) Where U = Total thrust of all motors; T1 = Thrust of front of motor (M1); T2 = Thrust of rear motor (M2); T3 = Thrust of right motor (M3); T4 = Thrust of remaining motor (M4); m = Quadcopter weight; I = Inertia moment; l = quad copter’s half-length; x, y, z denotes maneuver of quadcopter in multiple axis. Calculation of Thrust The mechanical force is referred to as thrust generated by the BLDC motors and propellers mounted on it in order to move the quadcopter in the mid-air. A motor spins at a

(6) Torque at maximum efficiency is: (7) RPM at maximum efficiency is:

Where RPM is kV rating of BLDC motor multiplied by battery voltage and diameter is the total length of propellers. Air density is calculated separately and CF is selected depending on propeller type (In most cases it is 1). A schematic of control structure is proposed for achieving the stability and altitude hold of quadcopter. As shown in Figure 10, there are different control modules such as obstacle avoidance and navigation system, altitude control & stabilization and dynamics of quadrotor system. In the control system, two loops are presented such as outer loop and inner loop. The output of sensors and control modules are illustrated in the inner loop while the outer loop has the inner loop and control systems of obstacle avoidance and altitude control. The platform’s basic operation and control are handled by the inner loop [14-15]. The altitude stabilization algorithm and altitude measurements are handled by this loop. The algorithms for

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

Figure 10. Front and side view of the corrugated tube. [From Mohanta et al. [14], with permission from Elsevier.]

altitude control, navigation and collision avoidance are all included in the outer loop. The GPS module, magnetometer, ultrasonic and infrared sensors are all used in this outer loop. Firmware installation and compass calibration The Pixhawk cube controller is upgraded with the latest firmware of Ardupilot by using mission planner software. In order to achieve the VTOL, accelerometer is calibrated by placing the nose of quadcopter in various directions. Compass calibration is considered to be the vital step for

achieving the smooth hovering and attaining the altitude stability. All the steps included in calibration of quadcopter are achieved successfully as shown in Figure 11 (a-e).

Data Acquisition And Power LINE Inspection

In Figure 12, a quadcopter-based pipeline for identifying power line insulators is illustrated. The deep learning pipeline consists of four steps. Pre-processing aerial

Figure 11. Front and side view of the corrugated tube. [From Ahmed et al. [17], with permission from IEEE.]

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

Figure 12. General structure of our proposed pipeline and intelligent approach for identification of insulators.

Figure 13. Front and side view of the corrugated tube. [From Miao & Ahmed et al. [12, 16], with permission from Elsevier.]

photographs is the first stage. The second stage entails identifying masts from aerial images. The third stage entails training the model using SSD architecture. The fourth stage entails detecting real-time insulators from power transmission lines. Before data augmentation techniques are employed, the original aerial photos are preprocessed by flipping, scaling, cropping, and labeling. The SSD architecture and aerial images are utilized for training the detection model. The authors combined a binary fine-tuning method with standard deep learning techniques to increase precision and robustness. SSD has a high degree of accuracy when it comes to component detection and classification. The usage of a tiny filter (convolutional) for component recognition and their classes, as well as offsets for bounding box

positions, make it an upgraded version of YOLO. SSD is now faster and more accurate than older techniques thanks to these developments. Separate filters for item predictions at varying ratios are run to get detections in multiple categories and yield the final detections. The standard design for SSD to achieve accurate picture categorization is the base network. Since VGG-16 performs exceptionally well in high-definition photo categorization and is simple to code into embedded devices for real-time detection, it is selected as the foundational network in this case. Unfortunately, according to review of the literature, there are no readily available datasets for training and inspection of power line components. In order to proceed with the deep learning-based visual inspection for component detection and classification training, a medium-sized

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

dataset of insulators is prepared by capturing power line components. The aerial photographs for this dataset were taken using a quadcopter equipped with a 4K GoPro Hero7 black camera and a Nikon D810 camera operating at several resolutions, including 1080p: 1920*1080, 7360*4912, and 60 frames per second. The authors have integrated various photographs of insulators and power line components obtained by UAVs to broaden the reach of the datasets, as shown in Figure 13. The captured datasets are trained in the ration 70:15:15, i.e. 70% images are used for training, 15% images are used for testing and 15% images are used for validation.

Results And Discussion

A medium-sized dataset comprising multiple insulators with varying backdrops and vegetation is used to train and apply the proposed component detection technique and assess its efficacy in identifying components from aerial images. Since there are no publicly available datasets for power line inspection, the author’s dataset of aerial images is used for training. The trained model with the Single Shot Detector (SSD) can identify insulators from aerial photos with different backgrounds. The Average Precision (AP) rate of multiple insulator detection at various training stages is shown in Figure 14. The tests were run every 2000 iterations out of a

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

Figure 16. Detection of pin and disc insulator in various backgrounds and results of insulator detection with defects. total of 20000 iterations and the detection model shows that after 10,000 iterations, the AP’s reach 89.04% and 90.31%, respectively. All of these processes take about 0.8 seconds to complete each iteration. Detection of aerial photographs with insulators against the different backgrounds is shown in [Figure 15 (a-d)]. The suggested method will instantly generate the green bounding box to locate the insulators against varied backgrounds. As illustrated in Figure 15, normal insulators are depicted in green, and defective insulators are shown in yellow padded boxes. Our suggested approach locates

the majority of insulator faults in a variety of backdrops, as shown in Figure 16. As shown in [Figure 16 (a-f)], the detection model is able to differentiate between the pin and disc insulators in a cluttered background. According to various field testing by authors, the safe flight range for quadcopter autonomous missions are maintained at 3-5 meters for 33 kV overhead power transmission lines and 7-10 meters for 110 kV overhead power transmission lines, etc. These reference values include unexpected events like a strong wind, a wind gust or a malfunction into account by including a safe margin of 18m. Figure 17 shows

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

Figure 17. Mean Average Precision (mAP) of SSD, YOLO, R-FCN and Faster R-CNN detectors on multiple class insulators. Insulator classes: SI = Suspension insulator; DI = Disc insulator; PI = Porcelain insulator; STI = Strain insulator; BI = Brown insulator how the suggested model performs in comparison to four other object detectors on various insulator classes of gathered datasets. Comparison with the other models shows that the recommended model operates rapidly, precisely and accurately.

Conclusion

This article proposes the use of aerial photos captured by a self-flying quadcopter as a non-invasive means of condition monitoring and fault identification in power line insulators. The deep learning method is implemented on an SSD dataset of power line insulators. The authors have addressed the shortcomings of inadequate aerial images and class imbalance by introducing a two-phase fine-tuning mechanism into the SSD training process. First, power line component parameters associated with different kinds of insulator detection and defects are integrated with a natural insulator dataset. After undergoing comprehensive training, the final model is able to distinguish between various insulators in aerial photos. The findings show that, with an overall running time of 0.8 seconds each iteration, suspension, pin, and disc power line insulators, as well as their constituent parts, can be easily identified with an average precision of 90.31 percent. The recommended model produced the greatest results from aerial photos, even with the intricate background and greenery. The robustness detection, classification, and model applicability were all markedly enhanced by the two-stage fine-tuning approach. With respect to competitive precision and multilevel component feature extraction, the suggested model can outperform the other models and their outcomes. Two potential future stages in the development of an autonomous vision-based power line component inspection system are suggested by the authors: The first is using binding thermal images as a dedicated dataset to identify

anomalies in power line components and enhancing the detection model with important information about insulator defect features to identify even small fractures in power line insulators and missing top caps. In order to develop a completely autonomous quadcopter for inspecting multistage sections of overhead power transmission cables and components, the suggested pipeline model must be improved on embedded platforms like edge GPUs.

Acknowledgments

The authors would like to thank the DST-ICPS, India for providing the financial assistance through project (T-32).

Data Availability Statement

The authors confirm that the data that supports the findings of this study are available within the article. Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

Sigma J Eng Nat Sci, Vol. 42, No. 3, pp. 621−632, June, 2024

Share and Cite