Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory
Abstract Aims Caregivers rate improved communication ability as one of the most desired outcomes for successful interventions for individuals with Angelman syndrome (AS). When measuring communication ability in clinical trials, the reliability of such measures is critical for detecting significant c...
Published in: | Journal of Patient-Reported Outcomes |
---|---|
Main Authors: | , , , , , , , |
Format: | Article in Journal/Newspaper |
Language: | English |
Published: |
SpringerOpen
2024
|
Subjects: | |
Online Access: | https://doi.org/10.1186/s41687-024-00725-9 https://doaj.org/article/7c1ba299b85e4ea292481cd5df58de30 |
id |
ftdoajarticles:oai:doaj.org/article:7c1ba299b85e4ea292481cd5df58de30 |
---|---|
record_format |
openpolar |
spelling |
ftdoajarticles:oai:doaj.org/article:7c1ba299b85e4ea292481cd5df58de30 2024-09-09T20:02:08+00:00 Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory Dandan Chen Christina K. Zigler Li Lin Nicole Lucas Molly McFatrich Jennifer Panagoulias Allyson Berent Bryce B. Reeve 2024-05-01T00:00:00Z https://doi.org/10.1186/s41687-024-00725-9 https://doaj.org/article/7c1ba299b85e4ea292481cd5df58de30 EN eng SpringerOpen https://doi.org/10.1186/s41687-024-00725-9 https://doaj.org/toc/2509-8020 doi:10.1186/s41687-024-00725-9 2509-8020 https://doaj.org/article/7c1ba299b85e4ea292481cd5df58de30 Journal of Patient-Reported Outcomes, Vol 8, Iss 1, Pp 1-7 (2024) Angelman syndrome Reliability Generalizability theory Communication Caregivers Public aspects of medicine RA1-1270 article 2024 ftdoajarticles https://doi.org/10.1186/s41687-024-00725-9 2024-08-05T17:49:22Z Abstract Aims Caregivers rate improved communication ability as one of the most desired outcomes for successful interventions for individuals with Angelman syndrome (AS). When measuring communication ability in clinical trials, the reliability of such measures is critical for detecting significant changes over time. This study examined the reliability of the Observed-Reported Communication Ability (ORCA) measure completed by caregivers of individuals with AS. Methods The ORCA measure was completed by 249 caregivers with 170 caregivers completing the ORCA measure again after 5–12 days. Generalizability theory was used to examine the following sources of measurement error in ORCA scores: concepts, subdomains, assessment points, and the interactions among those facets and the object of measurement: communication ability. Three generalizability studies were conducted to understand the reliability of the ORCA measure for different measurement designs. Decision studies were carried out to demonstrate the optimization of measurement procedures of the ORCA measure. Results G and Phi coefficients of the original measurement design exceeded the 0.80 threshold considered sufficiently reliable to make relative and absolute decisions about the communication ability of individuals with AS based on their caregivers’ observed scores. The optimization procedures indicated that increasing the number of communication concepts and/or assessment points leads to more reliable estimates of communication. Conclusion The ORCA measure was able to reliably distinguish different levels of communication ability among individuals with AS. Multiple assessment points and or more concepts would provide more precise estimates of an individual’s communication ability but at the cost of survey fatigue. Article in Journal/Newspaper Orca Directory of Open Access Journals: DOAJ Articles Journal of Patient-Reported Outcomes 8 1 |
institution |
Open Polar |
collection |
Directory of Open Access Journals: DOAJ Articles |
op_collection_id |
ftdoajarticles |
language |
English |
topic |
Angelman syndrome Reliability Generalizability theory Communication Caregivers Public aspects of medicine RA1-1270 |
spellingShingle |
Angelman syndrome Reliability Generalizability theory Communication Caregivers Public aspects of medicine RA1-1270 Dandan Chen Christina K. Zigler Li Lin Nicole Lucas Molly McFatrich Jennifer Panagoulias Allyson Berent Bryce B. Reeve Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory |
topic_facet |
Angelman syndrome Reliability Generalizability theory Communication Caregivers Public aspects of medicine RA1-1270 |
description |
Abstract Aims Caregivers rate improved communication ability as one of the most desired outcomes for successful interventions for individuals with Angelman syndrome (AS). When measuring communication ability in clinical trials, the reliability of such measures is critical for detecting significant changes over time. This study examined the reliability of the Observed-Reported Communication Ability (ORCA) measure completed by caregivers of individuals with AS. Methods The ORCA measure was completed by 249 caregivers with 170 caregivers completing the ORCA measure again after 5–12 days. Generalizability theory was used to examine the following sources of measurement error in ORCA scores: concepts, subdomains, assessment points, and the interactions among those facets and the object of measurement: communication ability. Three generalizability studies were conducted to understand the reliability of the ORCA measure for different measurement designs. Decision studies were carried out to demonstrate the optimization of measurement procedures of the ORCA measure. Results G and Phi coefficients of the original measurement design exceeded the 0.80 threshold considered sufficiently reliable to make relative and absolute decisions about the communication ability of individuals with AS based on their caregivers’ observed scores. The optimization procedures indicated that increasing the number of communication concepts and/or assessment points leads to more reliable estimates of communication. Conclusion The ORCA measure was able to reliably distinguish different levels of communication ability among individuals with AS. Multiple assessment points and or more concepts would provide more precise estimates of an individual’s communication ability but at the cost of survey fatigue. |
format |
Article in Journal/Newspaper |
author |
Dandan Chen Christina K. Zigler Li Lin Nicole Lucas Molly McFatrich Jennifer Panagoulias Allyson Berent Bryce B. Reeve |
author_facet |
Dandan Chen Christina K. Zigler Li Lin Nicole Lucas Molly McFatrich Jennifer Panagoulias Allyson Berent Bryce B. Reeve |
author_sort |
Dandan Chen |
title |
Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory |
title_short |
Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory |
title_full |
Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory |
title_fullStr |
Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory |
title_full_unstemmed |
Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory |
title_sort |
understanding reliability of the observer-reported communication ability measure within angelman syndrome through the lens of generalizability theory |
publisher |
SpringerOpen |
publishDate |
2024 |
url |
https://doi.org/10.1186/s41687-024-00725-9 https://doaj.org/article/7c1ba299b85e4ea292481cd5df58de30 |
genre |
Orca |
genre_facet |
Orca |
op_source |
Journal of Patient-Reported Outcomes, Vol 8, Iss 1, Pp 1-7 (2024) |
op_relation |
https://doi.org/10.1186/s41687-024-00725-9 https://doaj.org/toc/2509-8020 doi:10.1186/s41687-024-00725-9 2509-8020 https://doaj.org/article/7c1ba299b85e4ea292481cd5df58de30 |
op_doi |
https://doi.org/10.1186/s41687-024-00725-9 |
container_title |
Journal of Patient-Reported Outcomes |
container_volume |
8 |
container_issue |
1 |
_version_ |
1809934120893022208 |