Research Paper

Using Students as a Distributed Coding Team for Validation through Intercoder Agreement

Author(s)
A. Kaufmann, A. Barcomb and D. Riehle

Download
PDF

Year
2016

Keywords

Abstract
In qualitative research, results often emerge through an analysis process called coding. A common measure of validity of theories built through qualitative research is the agreement between different people coding the same materials. High intercoder agreement indicates that the findings are derived from the data as opposed to being relative results based on the original researcher's bias. However, measuring such intercoder agreement incurs the high cost of having additional researchers perform seemingly redundant work. In this paper we present first results on a novel method of using students for validating theories. We find that intercoder agreement between a large number of students is almost as good as the intercoder agreement between two professionals working on the same materials.

BibTeX

@techreport{kaufmann:2016:using,
  title = {Using Students as a Distributed Coding Team for Validation through Intercoder Agreement},
  year = {2016},
  author = {Kaufmann, Andreas and Barcomb, Ann and Riehle, Dirk},
  publisher = {Friedrich-Alexander Universit\"{a}t Erlangen-N\"{u}rnberg, Department of Computer Science},
  note = {Technical Reports CS-2016-01},
  month = {April}
}