Notes

Outline

The Psychological Experiment: Web-Based Research in Context John H. Krantz, Ph.D. Hanover College

Outline Background Characteristics of Web-based studies Web-based Studies in Context

Research Links added to exponnet.html

Recent Books on the Topic M. Birnbaum (Ed.), Psychological Experiments on the Web.  Academic Press. (2000). M. Birnbaum, Introduction to Behavioral Research on the Internet.  Prentice Hall. (2001). U.-D. Reips & M. Bosnjak (Eds.), Dimensions of Internet Science.  Pabst. (2001).

Characteristics of Web Studies Relative Ease Generally More Representative Sample Relative Power Less Control Still Often Relatively Valid

Generally More Representative Sample – Krantz & Dalal (2000)

Reasons for Use – Musch & Reips (2000) In descending order (7 very important – 1 not important) Large N 5.5 High statistical power 4.5 Speed data collection/participants from other countries 3.6 External or ecological validity 3.4 Low cost 3.2

Reasons for Concern Ethical Issues Informed consent/debriefing Data security Data Validity Subject falsification/collaboration Loss of Experimental Control Equipment/environment Variation

Importance of Problems Faced Musch & Reips (2000) In descending order Lack of control of behavior 3.6 Lack of control of motivation 3.4 Inability to ask questions 3.3 No control over hardware 2.9 Nonrepresentative sample 2.8 Ethical problems 1.5

The Web Study In Context Relevant Dimensions Methodological Issues Subject-Related Issues Ethical Issues A Decision Matrix

Relevant Dimensions – Method Statistical Power Noisy Data Effect Size or Robustness of the Effect  Easy to obtain the result Sample Bias Ecological Validity Does this work outside of laboratory

Relevant Dimensions – Method 2 Need for Control Stimulus Measurement Environment Effect of Web Known Has the method been validated on Web Time for data collection Response time for results

Relevant Dimensions – Subject Special Populations Ethnic groups Age Education Other targeted populations (Buchanan, 2000) Self-Monitoring: found groups predicted high and low on scale to validate response

Relevant Dimensions – Subject 2 Interaction with Subjects As part of research design Demand Characteristics

Relevant Dimensions – Ethics Sensitivity of Ethical Issues Deception Potential for harm Effects of Fraud by Participants on Data Faking responses Participant Collaboration

A Decision Matrix - Method

Decision Matrix – Method 2

Decision Matrix – Subject

Decision Matrix – Ethics

Decision Matrix Factors not Included Did not include ease or cost Also not important factor in early researchers, Musch & Reips, 2000 These factors should not drive design criteria.

Summary Factors favoring web studies: Statistical power, data fast, ecological validity, special populations, geographic diversity, demand characteristics Factors favoring lab studies: Control, ethics, subject fraud, interact with subject, sampling bias