Case study

A solar farm where nobody has marked the faults.

2,772 panels from 47 drone photos of a farm in Sindos, Greece. With no marks to check against, this run cannot say how right the app is. It shows how much work the app makes.

The facts

Pictures
47 drone photos
DJI Mavic 3T
Grey pictures, no camera readings
Checked
2,772 panels
Cut from the authors' panel boxes
Sent to a person
23 as suspected faults
192 for review
77 as an audit sample

Top row: panels the app sent as suspected faults. Bottom row: panels it did not flag.

The run

Overview

An ordinary farm, as far as anyone knows

This is a public set of pictures, put through the app as it is. The plant, the author and the author's organisation are not PanelProof customers, took no part in this and have no connection with PanelProof.

The set was made to teach software where panels are in a photo. It marks 26,678 panel boxes and no faults. So nobody can say which of these panels, if any, are faulty.

The question

Does the app flood the reviewer?

A tool that flags a quarter of a healthy farm is no use. With a fixed pass mark carried over from the training pictures, the fault finder would have flagged 23 of every 100 panels here. The review rule sets its levels from the batch instead.

What the app said

7.8 of every 100 panels sent to a person

23

suspected faults. That is 0.8 of every 100 panels

192

more that need review

2,557

not flagged

77

of those sent to review anyway, as an audit sample

The app showed no warning on this batch. Its scores sat where the rule expects ordinary panels to sit.

The panels it scored highest.

Panels it sent for review.

What it cannot tell

Limits of this run

  • Whether any flagged panel is faulty, or any panel that was not flagged is. There are no marks.
  • The panels were cut from the authors' boxes, which stand upright while many rows of panels slant. We used the 47 photos whose rows run nearly level and trimmed a tenth off every side of each box. Some crops still hold a sliver of a neighbour.
  • The photos are plain grey pictures. The app worked from brightness, not from camera readings.

Credit

Whose pictures these are

Panagiotis Christakakis, Eleftheria-Maria Pechlivani and Periklis Dimou, Centre for Research and Technology Hellas. "Thermal PV Panel Detection and Fault Detection Dataset for UAV-Based Inspection", Zenodo. Licensed under Creative Commons Attribution 4.0 International.

Shown changed: single panels cut out, trimmed, turned upright and resized to 24 by 40 pixels. Full credits and links.