Inter-rater reliability of query-probe-based techniques for measuring situation awareness

<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd">
  <record>
    <leader>00000cab a2200000   4500</leader>
    <controlfield tag="001">MAP20140025800</controlfield>
    <controlfield tag="003">MAP</controlfield>
    <controlfield tag="005">20140724124915.0</controlfield>
    <controlfield tag="008">140717e20140707esp|||p      |0|||b|spa d</controlfield>
    <datafield tag="040" ind1=" " ind2=" ">
      <subfield code="a">MAP</subfield>
      <subfield code="b">spa</subfield>
      <subfield code="d">MAP</subfield>
    </datafield>
    <datafield tag="084" ind1=" " ind2=" ">
      <subfield code="a">875</subfield>
    </datafield>
    <datafield tag="100" ind1="1" ind2=" ">
      <subfield code="0">MAPA20140012152</subfield>
      <subfield code="a">Lau, Nathan</subfield>
    </datafield>
    <datafield tag="245" ind1="1" ind2="0">
      <subfield code="a">Inter-rater reliability of query-probe-based techniques for measuring situation awareness</subfield>
      <subfield code="c">Nathan Lau, Greg A. Jamieson, Gyrd Skraaning Jr.</subfield>
    </datafield>
    <datafield tag="520" ind1=" " ind2=" ">
      <subfield code="a">Query- or probe-based situation awareness (SA) measures sometimes rely on process experts to evaluate operator actions and system states when used in representative settings. This introduces variability of human judgement into the measurements that require inter-rater reliability assessment. However, the literature neglects inter-rater reliability of query/probe-based SA measures. We recruited process experts to provide reference keys to SA queries in trials of a full-scope nuclear power plant simulator experiment to investigate the inter-rater reliability of a query-based SA measure. The query-based SA measure demonstrated only moderate¿ inter-rater reliability even though the queries were seemingly direct. The level of agreement was significantly different across pairs of experts who had different levels of exposure to the experiment. The results caution that inter-rater reliability of query/probe-based techniques for measuring SA cannot be assumed in representative settings. Knowledge about the experiment as well as the domain is critical to forming reliable expert judgements.</subfield>
    </datafield>
    <datafield tag="773" ind1="0" ind2=" ">
      <subfield code="w">MAP20100019818</subfield>
      <subfield code="t">Ergonomics : the international journal of research and practice in human factors and ergonomics</subfield>
      <subfield code="d">Oxon [United Kingdom] : Taylor & Francis, 2010-</subfield>
      <subfield code="x">0014-0139</subfield>
      <subfield code="g">07/07/2014 Volumen 57 Número 7 - julio 2014 </subfield>
    </datafield>
  </record>
</collection>