Hypertext transfer protocol is frequently used on the internet. According to Krol (1992) hypertext is a method of presenting information where selected words from the text can be expanded at any time providing other information about the word. This expansion is usually provided via highlighted links. These links allow the user to connect to other files, resources or other parts of the same file without knowing where you are going. Berners-Lee (1994) states that a file is an analog to a paper document but it is called a file when it is received on the internet. The hypertext transfer protocol is a fairly new concept; few true hypertext documents are available on the internet (Krol, 1992).
Human factors is the discipline that is involved in the design of equipment that people use intentionally taking into account how people operate (Kantowitz and Sorkin 1983). Human factors analysis must consider such things as consistency and population stereotypes when designing a hypertext document for the internet. Vancott & Kinkade (1972) discuss several advantages of using preferred controls and displays: increase reaction time, decrease probability of errors, and increase comprehension. Designing a program consistent with the population stereotypes is an important guideline. Hypertext document designs should be universal or at least have a related method of interfacing a document onto the internet. Lintern (1991) discusses skill transfer; suggesting that two tasks highly related by prior experience with one will make the next task easier. Users familiar with one protocol or design should be able to learn another protocol easily as a result of consistent methods and tools within the design or protocol. Unfortunately consistency in interfacing documents has been poorly defined with broad guidelines. Consistency within hypertext interfacing has not been experimentally tested. Lintern (1991) found that consistent designs should improve the user's productivity by leading to increased performance with fewer errors thus generally leading to increased user satisfaction.
The purpose of this experiment is to evaluate different methods of constructing an efficient hypertext file. The definition of efficient in this research consists of five components:
The hypothesis is that a document with numerous files combined with a table of contents with brief descriptions about each section is the most efficient system for the Hanover College Psychology Handbook.
The psychology handbook is the guide to which psychology majors, and minors can find needed information for their four years in the psychology department. It may also be used as a tool for the use of interested undeclared students to evaluate the psychology department. The psychology handbook becomes a decision making tool as well as a resource of information for students. The hypertext psychology handbook has considerable advantage over the normal version. The handbook is used more as a directory which you would look up bits of information, usually students do not read it straight through. The table of contents in the normal version has page numbers that the user must locate and then thumb through the document until she or he finds the information being sought (all time consuming). The hypertext version has no page numbers, but highlighted links that take the user directly to the information.
The survey consisted of eight questions, two of which were open-ended questions. The remaining six questions required the subject to mark an 'x' on the appropriate answer. The answers were arranged on Likert-type scale to allow for more then just a yes or no answer (Appendix 1). Three of the six scaled questions were selected as dependent variables to be tested against our two independent variables. The questions from the survey which were selected as dependent variables were: 1) "Do you feel that the table of contents contains too much information so that it is difficult to find the information you want?"; 2) "Would the table of contents be more helpful if each item were followed by a small explanation of that particular item?" and 3) "Would the handbook be more helpful if it were divided into many files, like chapters in a text book?" (Appendix 1) These three questions were picked for initial analysis because they appeared to most closely pertain to the independent variables.
After a brief task analysis of the handbook five questions were selected by the researchers. Table 1 displays the tasks numbered 1-5. These questions were selected to provide the best measure of the four independent variables. Task 3 was selected because it required the participant to look beyond the obvious sections of the handbook and table of contents. Task 4 was selected because it required the participant to sort through more than one section of the document to complete the answer. Tasks 1, 2, and 5 were more general questions from the handbook.
There are four dependent variables. The first dependent variable was the number of returns to the table of contents measuring the efficiency of the table of contents for information acquisition. The second dependent variable was the number of questions asked which should identify the condition that participants understood the most. The third dependent variable was task time, measuring the total time (in minutes) it took for participants to complete the experiment; identifying the condition which is most efficient in saving time. The fourth dependent variable the number of problems correct should identify the condition with the lowest probability of human error. The dependent measures should find a single condition consistently significant in all four measures. Subjective responses were measured in the form of a survey. The four conditions in which the dependent variables will be measured include: condition one; list/big format, condition two; list /small format, condition three; description/big format, and condition four; description/ small format.
This experiment took place in the Petticrew Computer Laboratory at Hanover College. The laboratory was open to the entire Hanover College population during the experiment. Each participant was seated at a Macintosh computer with a color monitor reserved for this experiment. The monitor displayed a set of instructions at the beginning of each condition. The instructions included the purpose, tasks required, and the directions for using the hypertext. Once the participant read the instructions and understood them he or she was asked to sign and date a consent form.
Researchers selected a task sheets. The five tasks on the sheet were ordered across subjects according to a latin square. The latin square forces each task to occur in each position on the task sheet an equal number of times decreasing the position bias effect. The experimental condition was labeled at the top of the sheet. The researcher handed the task sheet to the participants and asked them to answer the questions using the hypertext handbook. Each participant was also directed to write his or her class, major, and each time he or she returned to the table of contents at the top of the task sheet. As soon as the participants clicked onto the table of contents a timer was started to record the time taken to complete the five tasks. Researchers did not answer any questions during the experiment which would reveal the answer to the task. Once the participant completed the task sheet the survey was given to them. The researcher then recorded the reaction time and the number of questions asked on the task sheet. The survey ended the participants activity in the experiment. Researchers scored the task sheet at the end of each day. Data collection lasted for three days.
.
The number
of problems correct was not different between the list condition
and the description condition; the mean for the list condition was
3.67 and the mean for the description condition was 4.08.
The small condition with numerous files also had:
fewer questions asked (Figure 5),
a higher return rate (Figure 6),
and a smaller task time (Figure 7),
Again the means for the number of problems
correct was not different; the mean for the big condition was
3.92, and the mean for the small condition was 3.83.
The second question was "Would the table of contents be more helpful
if the item was followed by a small description" and was graded on scale from
1 = very helpful, to 4 = not helpful at all. A mean of 2.625 was obtained for
those subjects who used a table of contents without a description, while
the subjects who were tested on a table of contents with a
description produced a mean of 1.708 (Figure 8).
Our third question "Do you feel that the Table of Contents
contains too much information" was scaled accroding to the range 1 = very difficult, 3 = undecided, and 5 = not difficult at all. For those subjects who
performed the experiment with a table of contents without descriptions, the
mean score was 4.333. However, those subjects who worked with a
descriptive table of contents produced a mean of 4.083. (Figure
9).
The data from the three questions were recombined to test for trends across the levels of the size of the document. Figure 9 compares how the subjects in the different levels of the size of the document responded to the question of "dividing the handbook into chunks". Those with the big document gave a mean score of 3.208 while those with the many small files gave a mean score of 2.375.
The results from comparing the question of "do you feel the handbook contains too much information" with the size of the document variable were scored 4.00 for those individuals who had a handbook divided into separate files and 4.17 for those subjects with a handbook in one large file.
The final question of "would the table of contents be more helpful if it were followed by a small explanation" was also for each level of the size of the document variable. Subjects with the handbook in one file had mean score of 2.125, while the subjects who worked with the divided document produced a mean score of 2.208. Mann-Whitney U's were run on all of these comparsons. Only the question "do you feel that the table of contents would be more helpful if each item were followed by a small description" when tested across levels of the table of contents variable produced a significant difference, U=111, p=0.19.
Although the number of problems that the subjects got correct was not effected by the type of table of contents developed, there are still implications for the development of hypertext documents in general and the psychology handbook in particular. This document had a high probability of human error on task question 1; what requires an I.S., and task 5; what lab courses are needed and required for a major and minor in psychology? The majority of the participants missed these two questions in all conditions suggesting a problem in the structure of the question or the document. Webb and Kramer (1990) found similar results in their study. Webb and Kramer (1990) found that participants inability to determine the route to the desired information led to poor performance and insufficient use of the menu. Webb and Kramer also found list menu's to be inferior to more detailed or organized menus such as spatial menu's. Although a table of contents and menus are not the same thing they do share some characteristics in accessing information and organizing or providing links to one or more files. Webb and Kramer's research supports the idea of a more detailed system whether it be a menu or a table of contents leads to better performance. A cost-benefit analysis between the two levels of the size condition may be useful to understand the data pertaining to this variable which is more vague that the data for the table of contents. Again the number of problems correct is not effected by the size of the document as to what design is effective for hypertext documents but whether or not flaws may be present in the document. In an analysis of the costs and benefits the author of a hypertext document may want to analyze the benefits of having a long document that the user can move through freely or a small document with numerous files which will take less time to access. Considering the size of this document (71 K) and its non-linear character smaller files tend to be more appropriate provided that the links are sufficient. The smaller documents also force the user to return to the table of contents decreasing the time spent going through the whole document. The survey also found subjects giving a more positive rating to having the document in chunks by subjects who had the document in chunks.
The description and small condition used together may compliment each other by forcing the user to go back to the table of contents in the small document, thus returning to a descriptive table of contents which tends to be more beneficial in finding the desired information effectively reducing reaction time and increasing comprehension of the instructions and the document. According to the experimental and subjective data small documents with numerous files are more of a benefit outweighing the cost of not having the capability to move directly through the entire document. The small condition may organize the document better or it may allow the user to organize the document in his or her own fashion by forced return to the table of contents, the user chooses what section to go to next instead of sorting through all other sections first. Even though the data may be organized in a fashion relevant to a specific user population other users may need information out of the document for other purposes than what it was actually designed. The psychology handbook was designed for the major or minor population of psychology students for Hanover College, but once this document is placed on the internet any user has access to it. When designing documents for the internet the researcher should consider the efficiency for other user populations which may want to evaluate it.
Observation of the participants revealed inconsistencies in how the number of returns were recorded. Some participants recorded only the extra trips to the table of contents and some recorded all trips to the table of contents. Some participants seemed to be trying to minimize their trips possibly because they thought this was what the experimenter desired. Future experiments should take these confounding variables into consideration. A larger sample should factor out that possible bias.
Future experiments may want to acquire more participants, more experimenters, and controls for the natural environment. The number of experimenters made it difficult to record task time, answer questions and observe the participants when several were running at once. Distractions from other people in the natural setting (computer lab) bias the task time. Adams (1989) suggested incorporating reminders into the document to decrease the possibility of human error. This document incorporated a highlighted link back to the table of contents at the end of each section to remind the participants that they could return to the table of contents instead of sorting through the whole document. Graphic displays were not incorporated into this document. Graphic displays tend to slow down accessing time (A HREF="#adams">Adams, 1989).
A future goal for the handbook may be to set up links to external agencies and resources directly cited in the handbook such as Madison State Hospital and Lifesprings in Madison, Indiana. Research on external links may include independent variables on size of the external agencies document and the menu or table of contents for that document. Dependent measures would be similar to the dependent measures in this research: measure time it takes to link from the handbook to an external document, measure the organization and comprehension of the links to other documents by listing tasks that force the user to link to different external documents within the table of contents and record the number of questions asked to evaluate the comprehension of the instructions and tasks.
Return to: