With the help of a use case, Dr Patrick Vetter, Head of Competence Center Data Science at Supper and Supper GMBH explains how a deep learning methodology can locate and segment wind turbines on satellite imagery.
As a pioneer in fighting global climate change, Germany is increasingly investing in renewable energies, especially wind energy. With around 300 new turbines from 2016 to 2017, North Rhine Westphalia is one of the leading federal states in building new wind turbines. For assessing the wind energy potential and planning of new turbines, it is essential to track the spatial location of wind turbines together with their type and to combine this with information on average wind speed characteristics.
In the present use case, a methodology is described, that can locate and segment wind turbines on satellite imagery. The implemented neural network architecture is called U-Net and constitutes the state-of-the-art standard for image segmentation. The output is a pixel-wise prediction of the probability of a pixel to belong to a wind turbine. The deep learning framework was trained to predict wind turbine polygons on 280,000 satellite images covering the entire area of North Rhine Westphalia. The output was transferred to the geoinformation system ArcGIS and can be accessed online through various devices. The layer of wind turbines can be used for a thorough analysis with regard to wind energy potentials and for the spatial planning of turbines.
In consideration of the growing scarcity of fossil fuels, renewable energy sources are becoming increasingly important economically, socially and politically as an environment-friendly and efficient way of generating electricity.
The aim of the project was to support the federal ministry in North Rhine-Westphalia in generating regional registers of the locations and types of wind turbines to guide national energy producers in the spatial planning process of new plants.
A Convolutional Neural Network (CNN) was trained to identify and segment wind turbines and on-shore wind parks based on satellite imagery. The output wind turbine polygons can then be fed into Geographic Information Systems (GIS) and enriched with current wind data to efficiently monitor the current wind power.
Provided Data
The satellite imagery used in this project contained 280,000 image tiles provided by Esri’s (the market leader in geoinformation systems) World Imagery. The images cover the bounding-box area of the federal state North Rhine Westphalia and have an area of 1 km2 each. For each image tile, the corresponding geographic metadata was tracked. For the training data set 500 images and 200 for the validation dataset were selected, including 200 wind turbines of different types and in different landcover situations. For both sets, wind turbine polygons have been created.
Applied Methods
First an image pre-processing was performed to normalize satellite images for differing brightness, saturation and contrast levels. Then the training and validation data had to be generated by visually locating wind parks and turbines in ArcGIS Pro (Esri’s Professional GIS-Tool). The located turbines were then marked and converted into georeferenced polygons. After matching the resulting polygons with the corresponding image tile, they were converted into image masks. The mask classifies if an image pixel belongs to a wind turbine or not. This serves as the desired classification scheme for the developed artificial neural network. The deep learning framework used, is based on a U-Net architecture, which has been proven to perform very well for segmentation tasks with a low amount of training data. The segmentation performance was tracked using the Jaccard-Index, which is an intersection over union measure. The training was calibrated to achieve the maximum accuracy in the validation set in order to prevent model overfitting. The final layer of the neural net outputs an image mask with a pixelwise prediction of the likelihood of a pixel to belong to a wind turbine.
Challenges
The first challenge was to create the training features and to generate polygons of the wind turbines within the 700 satellite images. The application of unsupervised clustering, namely a K-Means colour clustering, helped with pattern recognition and extracting the polygon shape. Another time-consuming challenge was to detect the false positives of the network, i.e. recognized image segments that were falsely identified as wind turbines, like branching roads or aircraft. Further training epochs were needed to train the neural net to differentiate between those similar looking objects.
Project Outcome
Using the developed deep learning model, a regional register of wind turbines for the state of North Rhine-Westphalia was successfully created. In total, about 3,300 wind turbines have been identified in the satellite images.
This register was also captured as a layer in ArcGIS Pro and is now available as map material within the software, displaying the location of all identified wind turbines as polygons of their shapes. As a next step, the model can also be applied to other German states or even to create a worldwide wind turbine register. The wind turbine layer can be merged with current wind speed data to monitor the wind power generation and with average wind speed layers to support the spatial planning of new wind turbine sites.
Further applications
The developed deep learning model has already been utilized in other projects within the field of satellite image segmentation. Based on provided satellite imagery, stylized map material has been created. The neural net has been trained to detect different objects and land cover types on satellite imagery, such as roads, trees, forests, vehicles, buildings, rivers and agricultural fields.
Outlook
Satellite imagery has multiplicative application fields and can be used to gain a better understanding of other domains, e.g. to help identify natural resources more easily, to visualize and monitor climate or vegetative changes or to depict impacts of natural disasters more accurately. However, these things have mainly been achieved so far through manual or semi-automated methods. Artificial Intelligence and Satellite Imagery Feature Detection can contribute significantly to these geographical applications.
Dr Patrick Vetter will be speaking at Data Natives 2018– the data-driven conference of the future, hosted in Dataconomy’s hometown of Berlin. On the 22nd & 23rd November, 110 speakers and 1,600 attendees will come together to explore the tech of tomorrow. As well as two days of inspiring talks, Data Natives will also bring informative workshops, satellite events, art installations and food to our data-driven community, promising an immersive experience in the tech of tomorrow.
