Expert article
Automatic defect detection: every image looked at equally closely
Visual defects are the most frequent in turnkey construction — and the most tedious. Which is exactly why a person under time pressure overlooks them and a machine does not.
Expert article by Open Experience GmbH. · As of: September 2026
How does AI defect detection work on site?
An object-detection algorithm searches site photographs for visual anomalies such as scratches, cracks, stains or damage and marks the affected areas of the image. Matching against the BIM model establishes where the finding sits in the building; the system then checks whether a defect has already been recorded at that spot. If it is new, it is created and classified automatically, and a crop of the detection serves as the supporting record. The method was trialled in ESKIMO, a research project funded by the German Federal Ministry of Education and Research (BMBF).
Legal framework: Germany (Civil Code BGB, construction contract rules VOB/B, fee schedule HOAI). Other countries have different rules, and contractual agreements take precedence over the standard periods named here.
The sequence in five steps
The detection itself is only one step out of five. The other four decide whether a marked area of an image turns into a usable record.
- 1 — CaptureRegular, located photographs of the site, for example as a 360° walkthrough with the helmet camera system.
- 2 — DetectObject detection marks conspicuous areas in the image and assigns a defect type to them.
- 3 — LocateMatching against the model turns a spot in an image into a position in the building.
- 4 — ReconcileIf a defect already exists at that position, no second one is created. Without this step the automation produces duplicates rather than benefit.
- 5 — CreateThe new defect is recorded with its classification, image crop and position — and travels the same route as every other defect.
What the machine does better — and what it does not
The comparison is not a verdict but a division of labour.
| Task | Human | Automatic detection |
|---|---|---|
| *Looking at every image equally closely* | Hard to sustain under time pressure | The real advantage |
| *Finding the smallest deviations* | Depends on light, routine and the state of the day | Systematic, down to the pixel |
| *Judging the cause* | Experience and context | Not possible |
| *Assessing relevance* | Knows the contract, the detail and the construction sequence | Not possible |
| *Clarifying matters with the parties involved* | Core work of site management | Not possible |
The machine supplies candidates, not decisions. The benefit arises where a person can review those candidates faster than the raw images.
Where the training data came from
Detection methods are only as good as the material they were trained on. In the ESKIMO project more than a hundred thousand photographs of visual defects — scratches, cracks, damage — were analysed and categorised by hand. This manual work is the invisible part of every AI project; it also explains why methods from other industries cannot readily be transferred to construction.
For context: ESKIMO was a research project with eleven partners and a budget of 2.4 million euros. The results are prototypes, not production features.
Five effects that become visible
Consistency
The thirtieth flat is inspected as closely as the first.
Fewer duplicates
Reconciling against existing records prevents the same defect being captured twice — a standing problem in lists.
Shown, not described
The marked image crop shows what is meant. Queries become unnecessary.
Sooner rather than later
What is noticed during the current walkthrough is still a matter of workmanship, not a warranty case.
Open to analysis
If the same type of defect recurs at the same detail, that points to the design.
And
Site management reviews candidates instead of opening every image one by one.
The prerequisite is not the AI
The most common reason automatic detection delivers nothing on a project lies upstream of the algorithm: the images are missing. Without regular walkthroughs, captured comparably and tied to a position, there is nothing to evaluate. Anyone starting today gains first from the capture itself — the automation comes on top later.
The route from finding to live record
Whether a defect is found by a person or by an algorithm changes nothing about what has to happen next: it needs a location, an image, a description, a responsible party and a deadline, and it has to be tracked through to sign-off. That is exactly what Construction Defects is built for — the detection feeds candidates into the same process instead of creating a second list.
Frequently asked questions
Can I book automatic defect detection today?
No. It comes out of the ESKIMO research project and was demonstrated there. What is available in production use is pixelation, object recognition and automatic image classification.
Which defects can be detected automatically at all?
Visual phenomena on visible surfaces: scratches, cracks, stains, discolouration, damage. Anything calling for measurement, opening up or touch remains a matter for human inspection.
How reliable is the detection?
It supplies candidates with varying degrees of confidence. In practice the hit rate matters less than how false alarms are handled — a list that is wrong too often stops being reviewed.
Does this replace the walkthrough?
No. It changes what happens during the walkthrough: recording instead of writing up. The assessment follows afterwards, at the screen.
What happens to defects detected in error?
They are discarded. What matters is that the discarding is documented — otherwise the same finding turns up again at the next walkthrough.
Does the detection need a BIM model?
For precise positioning yes, for the detection itself no. Without a model the finding can be allocated via the capture point on the floor plan.
Sources and legal basis
The legal statements in this article are based on the primary sources listed below. The article is not a substitute for legal advice in an individual case.
The images first, then the automation.
45 minutes on your own project: how to run a walkthrough that makes evaluation possible in the first place.