It was the goal of the researchers to investigate whether perceived stimuli plays a role in influencing people’s physiological and emotional reactions (2011).
The method chosen to measure reliability for this testing scenario is the inter-observer reliability, also known as inter-rater reliability. In this method, reliability assessment centers on rating the level of agreement between the observations made by the different observers involved in the study. Therefore, the test items or categories researchers choose to observe need to be reliable manifesting themselves in a universal manner across all participants guaranteeing the consistency of results (Wanous, 2001). In the above testing scenario, the reliable test item was the micro-facial expressions category whereas the unreliable test item was the labored breathing.
Research done on micro-facial expressions indicates that humans across the globe have similar micro-facial expressions. The micro movements on a person’s face, for example, a twitch at the top corner of the eye caused by a reaction to a particular situation is a universal reaction to shocking, negative stimuli. Therefore, researchers in agreement about the micro-facial expressions to look out for when making their observations are able to detect them easily. As a result, a comparison of their results will indicate consistency because of the high level of agreement. On the other hand, a reaction such as labored breathing is open to the subjective interpretation of the observer regardless of the operational definition discussed among all researchers at the onset of research. For example, an overweight participant might suffer from labored breathing discrediting the idea that labored breathing is a physiological response specific to exposure to negative stimuli. The unreliability of this test item might cause some researchers not to note changes in labored breathing among some participants exposed to negative stimuli.