The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory
Far-sighted marine research institutions around the globe are capturing images from the seafloor at a scale of hundreds of thousands. Only a small part of these data have been accessed to date, as manual analyses are time-consuming and automated evaluation approaches are still under development. Mac...
Main Authors: | , , , , , , , |
---|---|
Format: | Conference Object |
Language: | unknown |
Published: |
2012
|
Subjects: | |
Online Access: | https://epic.awi.de/id/eprint/31711/ https://hdl.handle.net/10013/epic.40474 |
id |
ftawi:oai:epic.awi.de:31711 |
---|---|
record_format |
openpolar |
spelling |
ftawi:oai:epic.awi.de:31711 2024-09-15T18:07:03+00:00 The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory Schoening, Timm Bergmann, Melanie Purser, Autun Gutt, Julian Dannheim, Jennifer Taylor, James Nattkemper, T. W. Boetius, Antje 2012-12 https://epic.awi.de/id/eprint/31711/ https://hdl.handle.net/10013/epic.40474 unknown Schoening, T. , Bergmann, M. orcid:0000-0001-5212-9808 , Purser, A. , Gutt, J. , Dannheim, J. orcid:0000-0002-3737-5872 , Taylor, J. , Nattkemper, T. W. and Boetius, A. orcid:0000-0003-2117-4176 (2012) The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory , 13th International Deep-Sea Biology Symposium, 3 December 2012 - 7 December 2012 . hdl:10013/epic.40474 EPIC313th International Deep-Sea Biology Symposium, 2012-12-03-2012-12-07 Conference notRev 2012 ftawi 2024-06-24T04:06:16Z Far-sighted marine research institutions around the globe are capturing images from the seafloor at a scale of hundreds of thousands. Only a small part of these data have been accessed to date, as manual analyses are time-consuming and automated evaluation approaches are still under development. Machine learning and neural networks have been identified as a promising algorithmic approach to automate analysis of images from the seafloor. These algorithms need ground-truth data about the objects to be detected. As the information provided by one human expert lacks reproducibility, the expertise of a group of individuals has to be employed to collect training data as well as to evaluate the performance of an automated detection. In this paper we show that the inter-and intra-observer agreements of these human experts is a critical factor for the training of a learning architecture and has shown to be conditional to image quality for some object classes. A supervised automated detection approach is evaluated where five experts marked the positions of eight distinct object classes within seventy images taken at the HAUSGARTEN observatory (eastern Fram Strait, Arctic). Support Vector Machines were trained to detect and classify objects in the images with an overall sensitivity of 0.87 and precision of 0.67. A detailed comparison of the human expert agreements showed interesting correlations with the system's performance and pointed us towards new strategies for (semi-) automated underwater image analysis. Conference Object Fram Strait Alfred Wegener Institute for Polar- and Marine Research (AWI): ePIC (electronic Publication Information Center) |
institution |
Open Polar |
collection |
Alfred Wegener Institute for Polar- and Marine Research (AWI): ePIC (electronic Publication Information Center) |
op_collection_id |
ftawi |
language |
unknown |
description |
Far-sighted marine research institutions around the globe are capturing images from the seafloor at a scale of hundreds of thousands. Only a small part of these data have been accessed to date, as manual analyses are time-consuming and automated evaluation approaches are still under development. Machine learning and neural networks have been identified as a promising algorithmic approach to automate analysis of images from the seafloor. These algorithms need ground-truth data about the objects to be detected. As the information provided by one human expert lacks reproducibility, the expertise of a group of individuals has to be employed to collect training data as well as to evaluate the performance of an automated detection. In this paper we show that the inter-and intra-observer agreements of these human experts is a critical factor for the training of a learning architecture and has shown to be conditional to image quality for some object classes. A supervised automated detection approach is evaluated where five experts marked the positions of eight distinct object classes within seventy images taken at the HAUSGARTEN observatory (eastern Fram Strait, Arctic). Support Vector Machines were trained to detect and classify objects in the images with an overall sensitivity of 0.87 and precision of 0.67. A detailed comparison of the human expert agreements showed interesting correlations with the system's performance and pointed us towards new strategies for (semi-) automated underwater image analysis. |
format |
Conference Object |
author |
Schoening, Timm Bergmann, Melanie Purser, Autun Gutt, Julian Dannheim, Jennifer Taylor, James Nattkemper, T. W. Boetius, Antje |
spellingShingle |
Schoening, Timm Bergmann, Melanie Purser, Autun Gutt, Julian Dannheim, Jennifer Taylor, James Nattkemper, T. W. Boetius, Antje The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory |
author_facet |
Schoening, Timm Bergmann, Melanie Purser, Autun Gutt, Julian Dannheim, Jennifer Taylor, James Nattkemper, T. W. Boetius, Antje |
author_sort |
Schoening, Timm |
title |
The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory |
title_short |
The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory |
title_full |
The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory |
title_fullStr |
The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory |
title_full_unstemmed |
The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory |
title_sort |
impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the hausgarten observatory |
publishDate |
2012 |
url |
https://epic.awi.de/id/eprint/31711/ https://hdl.handle.net/10013/epic.40474 |
genre |
Fram Strait |
genre_facet |
Fram Strait |
op_source |
EPIC313th International Deep-Sea Biology Symposium, 2012-12-03-2012-12-07 |
op_relation |
Schoening, T. , Bergmann, M. orcid:0000-0001-5212-9808 , Purser, A. , Gutt, J. , Dannheim, J. orcid:0000-0002-3737-5872 , Taylor, J. , Nattkemper, T. W. and Boetius, A. orcid:0000-0003-2117-4176 (2012) The impact of experts' inter-and intra-observer agreements on computational object classification in images from the deep seafloor of the HAUSGARTEN observatory , 13th International Deep-Sea Biology Symposium, 3 December 2012 - 7 December 2012 . hdl:10013/epic.40474 |
_version_ |
1810444439321051136 |