Effects of Desktop Virtual Reality on Learner Performance and Confidence in Environment Mastery: Opening a Line of Inquiry

Lynna J. Ausburn
Floyd B. Ausburn
Oklahoma State University

Abstract

Virtual reality (VR) has demonstrated effectiveness as an instructional technology in many technical fields. However, VR research has generally lacked a sound theory base to provide explanatory or predictive strength. Further, research into the effectiveness of new desktop technologies that place VR within the reach of schools and teachers is currently embryonic. The study reported here is a pilot and is highly exploratory. It is a first step in developing a theory-based line of inquiry into desktop VR as an instructional technology with potential for Career and Technical Education. Grounded in several theory and research strands, this study compared the effects of presenting a complex scene via desktop VR and a set of still photographic images. The two treatments were given to groups drawn from the general population with equal representation by both genders and two age groups. Two performance measures and a confidence measure were analyzed using 2-way ANOVAs. Statistically significant main effects for treatment were found for all three measures, all in favor of the VR treatment. These findings were consistent with predictions based on the study’s theory base. Several main effects for age and gender, and trends for interactions of age and gender with treatment, were also identified that may provide impetus for further research.

Introduction

The use of visual technologies for teaching and learning in industrial education has produced dramatic extensions of the once traditional lectures, demonstrations, and hands-on experiences….visual technologies have enhanced the preparation of workforce specialists and technicians by bringing into classrooms and laboratories a breadth and depth of realism that has enhanced comprehension, increased learning performance, and reduced training time. Occasionally, however, there arrives a training technology that causes a realization that “this changes everything.” Such a technology is virtual reality (VR). ( Ausburn & Ausburn, 2004, p. 33)

The term virtual reality (VR) has undergone continuous definitional changes since its introduction in the late 1960s as immersive experiences with computer generated imagery via head-mounted displays (HMDs). The term can now refer to a variety of computer-based experiences ranging from fully immersive environments with complex HMD gear, auditory input, voice activation, data gloves, and even body suits wired with biosensors for advanced sensory input and biofeedback, to new non-immersive desktop environments based on realistic PC imagery ( Ausburn & Ausburn, 2004 ; Beier, 2004 ). However, in all its manifestations, VR is basically a way of simulating or replicating an environment three-dimensionally and giving the user a sense of “being there,” taking control, and personally interacting with that environment with his/her own body ( Arts and Humanities Data Services, 2002 ; Ausburn & Ausburn, 2004 ; Beier, 2004 ; Brown, 2001 ; Negroponte, 1995 ; Slater & Usoh, 1993 ).

In addition to simulating a three-dimensional (3D) environment, all forms of VR have in common computer input and control. An extensive review of VR research led the present authors to conclude that:

It is generally agreed that the essence of VR lies in computer-generated 3D worlds. Its interface immerses participants in a 3D synthesized environment generated by one or more computers and allows them to act in real time within that environment by means of one or more control devices and involving one or more of their physical senses. ( Ausburn & Ausburn, 2004, p. 34)

These characteristics of VR result in a stimulation of participants’ senses that gives them a strong impression of actually being present in an environment with which they interact personally ( Brown, 2001 ). Rigole (1996 ) summarized the characteristics that currently typify VR and defined it simply as any computer-generated simulation of a real or imagined 3-dimensional environment that is user interactive.

The newest form of VR is generally referred to as non-immersive or desktop VR. It is based on high-resolution panoramic imagery presented on a desktop computer and controlled by the user through simple navigation controls. Desktop VR is the simplest form of virtual reality and is quite different from technically difficult and costly immersive VR technologies that isolate users from the outside world and fully immerse them within a computer-generated environment through sophisticated devices such as head-mounted displays, data gloves, body suits, and complex visual display systems ( Simpson, 2003 ). Several definitions have been offered that differentiate desktop VR from immersive VR. Simpson (2003 ) stated that desktop or non-immersive VR uses conventional desktop computers, multimedia, and distance learning systems. A recent online source defined desktop VR as 3D imagery that can be explored interactively at a personal computer by manipulating keys or the mouse so that the content moves and zooms in or out ( WhatIs, 2005 ). The present authors explained that desktop VR employs mouse, joystick, or sensorball-controlled navigation through a 3D environment on a graphics monitor under computer control ( Ausburn & Ausburn, 2004 ). Desktop VR is presented in the form of on-screen “movies” that are created by taking a series of digital still images and then using special VR software to “stitch and blend” these images into a single panoramic scene that the user can “enter” and explore interactively. The user employs a mouse or other navigation device to “…move and explore within a virtual environment on his/her computer screen as if actually moving within a place in the real world ( Ausburn, Ausburn, Cooper, Kroutter, & Sammons, 2007, p. 9). Movement can include rotating the panoramic image to simulate physical movements of the body and head, and zooming in and out to simulate movements toward and away from objects or parts of the scene. Object movies can be embedded in panoramas to permit the user to “pick up,” rotate, and examine individual items, and clickable “hot spots” can interlink multiple panoramas and embedded objects. What characterizes these movies is that the user, “…chooses when and where to move and what actions to take, rather than being controlled by the pre-production decisions of a videographer” ( Ausburn & Ausburn, 2004, p. 41).

A critical feature for educators of this new desktop VR technology is its technical and financial accessibility. In their discussion of the defining characteristics of desktop VR, Arts and Humanities Data Services (2002 ) asserted that desktop VR systems can be distributed easily via the Internet or on CD and that users need little skill to install them and only a standard computer with a simple software viewer to play and explore them. Creation of desktop VR movies requires a hardware/software investment of around $4000 plus a high-end off-the-shelf computer, and can be mastered by computer-literate instructors with a few days of training. It is this new form of desktop VR that is the focus of the study reported here.

Review of Literature

Perhaps because of its properties of user immersion and interaction, VR appears to be frequently effective as an instructional technology. Early in the VR research literature, Winn, Hoffman, Hollander, Osberg, Rose and Char (1997) claimed that three factors contribute to the capabilities and impacts of VR: (a) immersion, (b) interaction, and (c) engagement and motivation. Selwood, Mikropoulos, and Whitelock (2000 ) proposed that VR’s power as an educational tool stems from its ability to exploit the intellectual, social, and emotional processes of learners. More recently, Seth and Smith (2004 ) asserted that the effectiveness of all types of VR comes from its ability to let learners experience a strong sense of presence in, and interaction with, a scene. They also attributed the success of VR technology to its ability to provide depth cues through stereo imagery that helps to convey to learners three-dimensional spatial relationships more realistically and accurately than conventional visualization tools. The accumulated research evidence indicates that enthusiasm for VR is generally high among educators and trainers who have tried it. An extensive review of the literature revealed several major themes that appear to have emerged in the study of instructional VR.

Concerns about Cost, Technical, and Instructional Design Challenges of VR in Instruction

Researchers have pointed out several areas of concern in using VR for teaching and learning. One important concern is the high levels of skill and cost required to develop and implement many VR systems ( Mantovani, Gaggiolo, Castelnuova, & Riva, 2003 ; Riva, 2003 ). Concerns have also been raised about the as-yet unknown physical and psychological effects of VR (Mantovani, et al.; Riva) and the complexity of the high-end computing equipment it requires. For example, Riva (2003 ) and Sulbaran and Baker (2000 ) both discussed the “latency problem” in VR, which arises when inadequate computers or online band width dramatically limits the response time for navigation and interaction and destroys its usefulness as a reality simulation. Also cited as concerns for VR instruction have been weak instructional designs that: (a) fail to achieve adequate sense of reality and “presence” to allow VR training to transfer to the real world (Riva); (b) present a poor or incomplete analysis of a learning task (Wong, Ng, & Clark, 2000); or (c) have overly complex navigation control, poor guidance of learner exploration ( Chen, Toh, & Ismail, 2005 ), unappealing look and feel (Sulbaran & Baker), or poor simulation of concrete hands-on experience (Young, 2000).

Generally High Level of Enthusiasm and Research Support for VR as an Instructional Technology

Despite the concerns and issues for VR discussed in the literature, the general consensus of its effectiveness and benefits in teaching and learning has been extremely high (e.g., Ausburn & Ausburn, 2004 ; Boehle, 2005 ; Pantelidis, 1993 ; Riva, 2003 ; Selwood, Mikropoulos & Whitelock, 2000 ; Sulbaran & Baker, 2000 ; Watson, 2000 ). This apparent enthusiasm has been supported by considerable empirical evidence of the motivational properties and instructional benefits of VR (e.g., Mantovani et al., 2003 ; Pantelidis, 1993 ; Sulbaran & Baker, 2000 ; Winn et al., 1997). Watson’s (2000 ) conclusion that “Most would consider that…[VR] systems provide strong potential…for the educational process,” (p. 231) appears to represent well the general position and expectation of virtual reality researchers and users.

Training Success of VR in a Variety of Occupations and Industries

Published research literature documents many positive effects of VR. The field most actively reported in the VR literature is medical/dental education, where large numbers of published studies have attested to VR’s benefits ( Imber, Shapira, Gordon, Judes, & Mitzgar, 2003 ; Jaffe & Brown, 2000 ; Jeffries, Woolf, & Linde, 2003 ; Mantovani et al., 2003 ; Moorthy, Smith, Brown, Bann, & Darzi, 2003 ; Patel, Gallagher, Nicholson, & Cates, 2004 ; Riva, 2003 ; Seymour, et al., 2002 ; Urbankova & Lichtenthal, 2002 ; Wilhelm, Ogan, Roehaborn, Caddedder, & Pearle, 2002 ; Wong et al., 2000). Engineering education has also reported considerable success with virtual reality instruction ( Sulbaran & Baker, 2000 ).

A variety of occupational and technical education programs have reported positive performance results in the research literature. These have included auto spray painting ( Heckman & Joseph, 2003 ), firefighting ( Government Technology, 2003 ), forestry machine operation ( LaPoint & Roberts, 2000 ), meteorology ( Gallus, 2003 ), and welding ( Mavrikios, Karabatsou, Fragos, & Chryssolouris, 2006 ). Use of VR for both training and product development has also been reported in a variety of industries such as aerospace, petroleum, equipment design, vehicle prototyping, lathing and manufacturing, accident investigation and analysis, law enforcement, anti-terror response, hazard detection, crane driving, aircraft inspection and maintenance, and facilities planning (e.g., Flinn, 2005 ; Government Technology, 2003 ; Halden Virtual Reality Center, 2004 ; Jezernik, 2003 ; Sandia National Laboratories, 1999 ; Scavuzzo & Towbin, 1997 ; Sims, Jr., 2000 ; Shneiderman, 1993 ).

Research Focus on Immersive and Technically Complex VR Systems

In an extensive review of the literature on instructional VR, Ausburn and Ausburn (2004) reported that the published studies have focused almost exclusively on complex immersive VR technologies, with an absence of reported research on the instructional effects of new desktop VR technologies. While a few published studies have supported the effectiveness of desktop VR ( Jeffries, Woolf, & Linde, 2003 ; Lapoint & Roberts, 2000 ; McConnas, MacKay & Pivik, 2002 ; Scavuzzo & Towbin, 1997 ; Seth & Smith, 2002 ; Wong et al., 2000), these are very small in number and as yet fall far short of establishing firm empirical support for instructional uses of desktop VR environments. These studies also fail to incorporate the recent technical improvements in digital cameras, software, and computer graphics that dramatically improve the realism, navigation, and immersion value of desktop VR. The lack of current research support for desktop VR is problematic because it is this new PC-based technology that brings VR within the reach of most schools and teachers, both technically and economically. The embryonic status of research on desktop VR, combined with its recent dramatic technical improvement and its instructional potential, provided the impetus for the study reported here and the line of inquiry arising from it.

Theoretical and Conceptual Framework for the Study

The theoretical underpinnings for this study are found in supplantation theory, Dale’s Cone of Experience for media concreteness, Bandura’s self-efficacy construct, and current research on technology-related age and gender differences. The study’s conceptual frame is found in the Aptitude-Treatment Interaction (ATI) research models that emerged in instructional technology and design research of the 1970s to study interrelationships among learning task requirements, learner characteristics, and instructional treatment features.

Salomon’s (1970 ) classic definition of supplantation identified this process as the explicit and overt performance or alteration of a learning task requirement that learners would otherwise have to perform covertly for themselves. The present authors have defined supplantation operationally in the context of design of technology-based instructional treatments as “…the use of an instructional treatment to either capitalize on learners’ strengths or to help them overcome their weaknesses” ( Ausburn & Ausburn, 2003, p. 3).

Supplantation-based instructional design is specifically based on an intersection or interaction of three critical components identified in the Cronbach and Snow (1977 ) Aptitude-Treatment Interaction (ATI) research model: a learning task with specific requirements, learners with specific capabilities/aptitudes related to the task, and an instructional treatment that bridges any existing gap between the two. At the psychological heart of supplantational instructional design is the notion that when learner characteristics are related to specific learning task requirements, an instructional treatment can be expected to have a positive effect on learner performance when it helps learners perform task requirements by “bridging” gaps between the task requirements and learner capabilities ( Ausburn & Ausburn, 2003 ). This proposes that the process underlying the use of an instructional treatment that strengthens or completes the learner/task link through explicit performance of a learning task requirement will have positive impact on learning performance. This is the process identified in supplantation theory.

In a learning task requiring spatial orientation and memory for details in a complex scene, the task is made more difficult by the need to hold and manipulate in memory complex sets of visual details and relationships from image to image. Mental retention and manipulation of these visual components and their relationships are critical to learning performance when the task is presented via a sequence of individual still pictures, as is currently the standard technique in instructional graphics intended to introduce learners to a new environment. However, when the task is presented via desktop virtual reality, the mental imagery requirements are supplanted by the presentation medium. The VR presentation mirrors physical reality in which learners can see all visual components and their relationships simultaneously in a seamless environment and can move within the entire scene at will to examine visual details and relationships. In fact, the virtual reality goes beyond the physical one by giving learners a way to continue to explore and re-visit the environment indefinitely. In this study, application of supplantation analysis led the researchers to theorize that a desktop VR presentation of a complex visual scene would supplant for learners the difficult mental imagery processes required when the scene was presented via a series of still images, and thus lead to improved scenic comprehension. Because such a complex visual environment is frequently encountered in technical education, evidence supporting this supplantational capability of desktop VR could support its value as an instructional technology in CTE.

A second theoretical support for the predicted efficacy of VR in this study came from Dale’s Cone of Experience. This icon of instructional design theory, based in Piagetian psychology’s proposition of concrete versus abstract reasoning, proposed that (a) various types of learning experiences and media representations vary in their “concreteness,” (b) more concrete forms of experience and media are truer and more complete representations of reality, and (c) media representations that are more concrete can facilitate learning, particularly when reality is complex and unfamiliar to learners ( Dale, 1954 ). Dale’s classic Cone of Experience presented various types of learning experiences in a pyramid with direct real-life experience at its base as the most concrete learning medium, verbal symbols (i.e., words) at its apex as the most abstract medium, and various other types of learning experiences and audio-visual representations arranged from base to apex in increasing abstractness as they move up the pyramid. The more completely and accurately an experience or representation presents “reality,” the greater its level of “concreteness” in Dale’s theory. One of the primary characteristics of VR is the fidelity of its presentation of the reality of a 3D environment and the relationships of items within the environment. Thus, application of Dale’s Cone and the theory of media concreteness led the researchers to hypothesize that VR would provide a more accurate and realistic experience of a complex visual scene than would be possible with other forms of media representation and would add to the supplantation advantages of the medium. The combination of supplantation and experiential concreteness theoretical foundations led to a substantive theory for VR efficacy that could be called supplantation-concreteness.

This study was developed as an exploratory pilot to test the VR supplantation-concreteness hypothesis and to trial research procedural techniques in a general application with relevance in CTE before proceeding to tests in specific and more technically challenging CTE environments. The study also tested established general instructional design theories of supplantation and media concreteness in the context of the new desktop VR technology. Instructional design research history has demonstrated repeatedly that generalized theoretical concepts do not always apply predictably to new technologies with unknown characteristics and must therefore be thoroughly tested.

The study also incorporated in its design possible relationships of two other variables with desktop VR: (a) effects of VR on learners’ perceived confidence in mastery of a complex visual environment, and (b) potential performance differences related to age and gender.

Theoretical support for inclusion of learners’ perceived confidence as a dependent variable in the study is provided by Bandura’s (1997 ) construct of self-efficacy, which he defined as belief or confidence in one’s ability to take appropriate actions to successfully perform a certain task. Bandura asserted that one’s level of self-efficacy, regardless of its truth, could impact actual performance. In this study, it was hypothesized that the supplantation-concreteness properties of VR might increase the technology self-efficacy of learners, thus increasing their perceptions of confidence and mastery as well as their learning performance.

Support for the inclusion of age and gender as intervening independent variables in this study came from several lines of research on digital technologies. Studies of the relationship of age and technology have established the deeply-ingrained technological skills and confidence of the digital natives of Generation Y and the Millenials, born after 1980, compared to those of the digital immigrants of earlier generations (Howe & Strauss, 2001; Prensky, 2001 ; Tapscott, 1998 ). Research has also shown a specific relationship of gender to virtual technologies. Twenty-five years of history with paper-and-pencil tests (e.g., Bennett, Seashore, & Wesman, 1973 ) have revealed consistent gender differences in skill in mental rotation of objects, with females generally having more difficulty than men. Evidence suggests that this gender gap in mental rotational skills is exaggerated in virtual environments, and that men and women often perceive virtual experiences quite differently, with men preferring more interactive environments than women ( Space, 2001 ; University of Washington, 2001 ). Findings such as these led to a speculation in the present study that VR might interact differently with the technology self-efficacy and the performance of learners of different ages and genders and provided a rationale for stratification of the sample on these variables.

Purpose and Hypotheses

The purpose of this study was to compare the effectiveness of desktop VR in presenting a complex scenic environment to presentation with traditional still color images, in the context of the supplantation design model. The study served as a pilot for a line of experimental inquiry into desktop VR grounded in theory-based instructional design. At issue was application and testing of established instructional design principles to a new technology with unknown characteristics and comparison with a graphic medium currently extensively used in CTE. Specifically, the study addressed three aspects of learning outcome by comparing scores of learners who received a desktop VR presentation of a complex scene with those who received a still imagery presentation of the same scene. Null hypotheses tested were:

1. Learners receiving a VR presentation of a complex scenic environment perform no differently on a test of scenic orientation than those receiving a still imagery presentation.
2. Learners receiving a VR presentation of a complex scenic environment perform no differently on a test of recall of scenic details than those receiving a still imagery presentation.
3. Learners receiving a VR presentation of a complex scenic environment report the same level of perceived confidence in scenic comprehension as those receiving a still imagery presentation.
4. There are no main or interaction effects for age and gender with VR and still image presentations.

Methodology

Research Design

Following procedures described below, the study used a quasi-experimental design to compare the instructional effectiveness of desktop VR and still color imagery in presenting a complex visual environment via standard PC computer. The research used a posttest-only design, with two levels of experimental treatment rather than a treatment/control configuration. This design lacks the random selection of subjects that define “true” experiments ( Campbell & Stanley, 1963 ; Fraenkel & Wallen, 2006 ), but is frequently used of necessity in education research. While the subjects were not randomly selected, they were randomly assigned in clusters to receive either a VR or a still image presentation treatment, which strengthened the study’s design.

Convenience samples were used, with built-in controlled representation from both genders and two age groups. Groups of subjects were randomly assigned to receive VR or still image instructional treatments, resulting in a fully factorial design. The groups were given the experimental treatments, and their learning performances and confidence perceptions were compared with descriptive statistics and two-way analysis of variance.

Subjects and Sampling Method

The subjects were 80 adults solicited by members of the research team (instructors and students in a graduate course in advanced technology and research) from people in their sphere of influence who fit specific age and gender requirements. The sample was purposively stratified to include equal numbers of males (n = 40) and females (n = 40) and equal numbers of representatives of the 18-35 (n = 40) and the 36-60 (n = 40) age groups, with each member of the research team identifying and testing an equal number of subjects in each gender/age subgroup. No limitations or requirements were imposed for selecting subjects, except that they have the required gender and age characteristics. Researchers were randomly assigned to use either the VR or the still imagery treatment presentation for their chosen subjects, thus creating a random cluster assignment of subjects to treatments. The sampling and treatment groups are summarized in Table 1.

Table 1
Subject Sub-Groups and Numbers for VR and Still Imagery Treatments (N = 80)

Gender/Age Groups	VR Treatment	Still Images Treatment
Males, age 18-35	n = 10	n = 10
Males, age 36-60	n = 10	n = 10
Females, age 18-35	n = 10	n = 10
Females, age 36-60	n = 10	n = 10
	n for Treatment = 40	n for Treatment = 40

The sampling procedure used, while frequently employed in quasi-experimental research designs, raises cautions about generalizing the findings of the study to CTE populations in education/training settings. However, this sampling design was felt to be adequate for the exploratory stature of the study and its preliminary test of the VR supplantation-concreteness hypothesis.

Learning Task Environment and Presentation Treatments

Selection of task environment. The scenic environment for this experiment was a house interior showing several interconnected rooms and a complex set of furniture and other decorative elements. This task environment was chosen because it satisfied several criteria needed in order to preserve the internal validity of the experiment.The following criteria were met by the house interior selected for the study:

1. No subject could have had prior exposure to this particular scene, giving each subject an equal baseline for the learning tasks and eliminating possible prior knowledge as a confounding variable.
2. The scene had sufficient number and complexity of details to discriminate among learner performances.
3. The scene was generic and non-technical, eliminating previous experiences and comfort with a technical environment by some subjects but not others as a confounding variable.

There were two other important considerations in selecting the task environment for this experiment. First, the chosen environment needed to provide a “clean” test of the supplantation-concreteness hypothesis. This necessitated an environment that was familiar to all participants and would require no labels, pop-ups, or other graphic identifiers of scenic components and details that would create a visually complex field. Such an environment field could add an element of visual field complexity that could interfere with the supplantional and concreteness properties of VR that were under analysis in this study. The house interior met this experimental control requirement, leaving possible interactions of supplantation-concreteness with other treatment variables for later studies. Second, the task environment for the study needed to have relevance to CTE and training in industry. The relevance of a house interior to training in real estate and advertising is obvious. However, more importantly, it is representative of a whole class of environments requiring mastery of locational relationships and assessment of details that are frequently encountered in CTE and industry training. Laboratories, production shops, equipment interiors, operating rooms, crime scenes, and construction sites all represent technical versions of the kind of environment represented generically by the house interior. This provides a link from the house environment used in this study to many conceptually similar CTE and workplace task applications.

Task presentation treatments. A desktop VR QuickTime® panorama movie (created with VRWorx® software) and a set of eight still color photographs of the house interior scene were produced to serve as the instructional treatments for the study. The same digital camera with the same lens was used for shooting both treatments, and identical visual information was present in both sets of materials. The still images made a static presentation of the components of the house scene. The VR movie allowed the user to control and explore the scene interactively by “walking” within it via horizontal and vertical panning, in/out zooming, and clickable hot spot navigation. Both treatments were presented via computer under learner control, which represents the way similar instructional treatments would be presented in actual instructional environments.

Two PowerPoint® presentations were developed, one to present each treatment. In the VR treatment, the VR panorama movie was accessed from within PowerPoint via an Action Button. In the still photo treatment, the eight still photographs were presented sequentially in a PowerPoint slide sequence, and then all eight were presented simultaneously as “thumbnails” on a single slide. The two PowerPoint presentations contained identical instructions to the subjects for completing the research task, which ensured uniformity of task presentation protocol across all researchers collecting data.

Learning Task Instrumentation

Operationalizing the performance variables of scenic orientation, recall of scenic details, and confidence level in scenic comprehension was challenging, as no guidance from any similar studies was found in the literature. The instruments developed for the study were pilot tested with representative subjects from the general population and refined based on their feedback to assure all items were clear and posed no problems with interpretation. Further work regarding instrument validity and reliability will be required as this line of inquiry progresses; however, the instruments were viewed as suitable for use in this exploratory study.

The task instrument completed by each subject comprised three sections designed to measure the aspects of learning performance and confidence perception of interest in the study. The first measure was identified as scenic orientation. This was operationalized as a 15-item multiple-choice test requiring subjects to mentally position or locate themselves within the scene and identify the location of designated objects in relation to their position. The performance measure was number of correct responses out of 15. A sample question is shown in Figure 1.

Figure 1. Example from 15-Item Multiple Choice Test of Scenic Orientation

The second performance measure was identified as recall of scenic details. This was operationalized as the number of correct and non-duplicative items, excluding large pieces of furniture, found in the house scene that the subjects could recall and list within a time of one minute. The one-minute time limit was established through preliminary testing as appropriate for discriminating among learners on this task.preliminary testing as appropriate for discriminating among learners on this task.

The third measure was identified as perceived confidence level in scenic comprehension. This was operationalized as the subjects’ self-reported confidence in their understanding of the details of the scene and the accuracy of their test responses, using the following five-point Likert-type scale:

The confidence or self-efficacy variable was included as a corollary and alternative to learning performance in assessing the effects of VR, similar to what frequently occurred in early research on color in instructional treatments. In this early research, it was often demonstrated that while color had no measurable effect on learning performance, it positively impacted learner attention to, interest in, and satisfaction with the learning experience. The implication was that increased learner approval must have some motivational impact on learning and that learning gains could be demonstrated if the right measures could be found. The exploratory nature of the performance measures used in this study suggested that the addition of a confidence variable was beneficial in examining a fuller range of potential effects of VR and suggesting alternative avenues for assessing impacts on learning performance.

Procedures

Members of the research team were randomly assigned to use either the VR or the still imagery presentation treatment with their subjects. Using a standardized written research protocol to minimize differences in data collection and recording procedures, members of the team selected their own subjects within the designated gender and age groups. Subjects were tested individually in a familiar location (home, workplace, etc) of their choice. Each subject was given the assigned treatment presentation, shown how to operate it, allowed as much time as desired to study the presentation, and then asked to complete the multiple choice test, the timed detail recall activity, and the Likert-type scale confidence question. Once subjects had completed their study of their presentation and begun the testing process, they were not permitted to see the VR movie or still images again.

All data for each subject were recorded on a standardized data sheet. The data were then coded and entered into an SPSS file for statistical analysis. Analysis was performed using descriptive statistics and full-factorial univariate linear model for fixed-factor 2-way Analysis of Variance

Findings

To analyze the data, descriptive statistics were first calculated for the various gender, age, and treatment groups on each of the three performance variables. These data are shown in Tables 2, 3 and 4.

Table 2
Descriptive Data for Gender, Age, and Treatment Groups on Scenic Orientation Score (Number of Items Correctly Answered Out of 15)

Gender or Age Group	Presentation Treatment	Mean	Std. Dev.	N
Male	Still Images	9.05	2.39	20
Male	Virtual Reality	9.60	3.55	20
Male	Gender Total (Still + VR)	9.33	3.0	40
Female	Still Images	10.05	2.28	20
Female	Virtual Reality	12.30	2.23	20
Female	Gender Total (Still + VR)	11.18	2.50	40
18 – 35	Still Images	9.25	2.24	20
18 - 35	Virtual Reality	11.55	2.93	20
18 – 35	Age Total (Still + VR)	10.40	2.83	40
36 – 60	Still Images	9.85	2.50	20
36 – 60	Virtual Reality	10.35	3.47	20
36 – 60	Age Total (Still + VR)	10.25	2.90	40
	Treatment Total – Still Imagery	9.55	2.37	40
	Treatment Total – VR	10.95	3.23	40
	Treatment Total (Still + VR)	10.25	2.90	80

Table 3
Descriptive Data for Gender, Age, and Treatment Groups on Recall of Scenic Details (Number of Correct Details Listed in 1 Minute)

Gender or Age Group	Presentation Treatment	Mean	Std. Dev.	N
Male	Still Images	4.80	1.85	20
Male	Virtual Reality	5.80	2.57	20
Male	Gender Total (Still + VR)	5.30	2.27	40
Female	Still Images	5.90	2.10	20
Female	Virtual Reality	8.35	4.45	20
Female	Gender Total (Still + VR)	7.13	3.65	40
18 – 35	Still Images	4.85	2.25	20
18 -35	Virtual Reality	7.25	1.92	20
18 – 35	Age Total (Still + VR)	6.05	2.40	40
36 – 60	Still Images	5.85	1.69	20
36 – 60	Virtual Reality	6.90	5.11	20
36 – 60	Age Total (Still + VR)	6.38	3.79	40
	Treatment Total – Still Imagery	5.35	2.03	40
	Treatment Total – VR	7.08	3.81	40
	Treatment Total (Still + VR)	6.21	3.16	80

Table 4
Descriptive Data for Gender, Age, and Treatment Groups on Perceived Confidence Level of Scenic Understanding (5 Point Scale)

Gender or Age Group	Presentation Treatment	Mean	Std. Dev.	N
Male	Still Images	3.05	.83	20
Male	Virtual Reality	3.30	1.08	20
Male	Gender Total (Still + VR)	3.18	.96	40
Female	Still Images	3.00	.86	20
Female	Virtual Reality	3.95	.89	20
Female	Gender Total (Still + VR)	3.48	.99	40
18 – 35	Still Images	3.00	.73	20
18 -35	Virtual Reality	4.00	.65	20
18 – 35	Age Total (Still + VR)	3.50	.85	40
36 – 60	Still Images	3.05	.94	20
36 – 60	Virtual Reality	3.25	1.21	20
36 – 60	Age Total (Still + VR)	3.15	1.08	40
	Treatment Total – Still Imagery	3.03	.83	40
	Treatment Total – VR	3.63	1.03	40
	Treatment Total (Still + VR)	3.33	.98	80

Two separate sets of 2-way ANOVAs were then performed: one set for gender by instructional treatment on each of the three performance variables, and one set for age group by treatment on each of the three performance variables. Statistical significance was set at the .05 level; trend was identified as p ≤ .16; effect size was measured with the eta squared statistic, with moderate effect size identified as η ² ≥ .06 ( Green & Salkind, 2005 ). ANOVA data for the gender and age analyses are shown in Tables 5 and 6, respectively.

Table 5
ANOVA Data: Gender x Instructional Treatment for All Dependent Variables

Performance Variable	Source	df	F	p	η 2
Scenic Orientation (Score on 15-item multiple choice test)
	Gender	1	9.618	.003*	.112 +
	Treatment	1	5.508	.022*	.068 +
	Gender x Treat	1	2.030	.156**	.026
	Error	76
	Total	79
Recall of Details (Number recalled)
	Gender	1	7.780	.007*	.093 +
	Treatment	1	6.950	.010*	.084 +
	Gender x Treat	1	1.228	.271	.016
	Error	76
	Total	79
Confidence (5-point scale)
	Gender	1	2.134	.148**	.027
	Treatment	1	8.537	.005*	.101 +
	Gender x Treat	1	2.905	.092**	.037
	Error	76
	Total	79
*   Statistically significant p ≤ .05 ** Trend .16 ≤ p > .05 +   Moderate effect size

Table 6
ANOVA Data: Age x Instructional Treatment for All Dependent Variables

Performance Variable	Source	df	F	p	η 2
Scenic Orientation (Score on 15-item multiple choice test)
	Age	1	.226	.636	.003
	Treatment	1	4.918	.030*	.061 +
	Age x Treat	1	2.032	.158**	.026
	Error	76
	Total	79
Recall of Details (Number recalled)
	Age	1	.224	.637	.003 +
	Treatment	1	6.311	.014*	.077 +
	Age x Treat	1	.966	.329	.013 +
	Error	76
	Total	79
Confidence (5-point scale)
	Age	1	2.970	.089**	.038
	Treatment	1	8.727	.004*	.103 +
	Age x Treat	1	3.879	.053**	.049
	Error	76
	Total	79
*   Statistically significant p ≤ .05 ** Trend .16 ≤ p > .05 +   Moderate effect size

The statistical results allowed rejection of all four of the study’s null hypotheses. Results favored VR over still image presentation on all three performance measures. In addition, several main effects for gender and age and several interactions between gender/treatment and age/treatment were observed.

On the scenic orientation variable, a significant (p ≤ .05) main effect of moderate effect size (η ² ≥ .06) for treatment was found in favor of VR for both gender and age. Similarly, significant main effects of moderate effect size in favor of VR were also found for the recall of details measure for both gender and age, and for perceived confidence level for both gender and age.

Several other findings were of interest. Significant main effects of moderate effect size were found for gender, in favor of the females, for both scenic orientation and recall of details. A main effect trend (.16 ≤ p > .05) for gender in favor of females was also observed on the confidence variable that may be of interest in further research. A main effect for age, in favor of the younger group, on confidence level approached significance (p = .09). Interactions on confidence level for both gender by treatment ( p = .09) and age by treatment (p = .06) also approached significance. Both these interactions were ordinal in nature, with both groups benefiting from the VR treatment. In the gender by treatment interaction, the females had greater benefits from the VR than the males; in the age by treatment interaction, the younger group had greater benefits than the older group. Two additional ordinal interactions showed trends at levels (.16 ≤ p > .05) that suggested they may be of interest in further research. These were interactions on the scenic orientation variable by both gender (p = .16), with females making the greatest gains under VR, and age (p = .16), with the younger group benefiting most from VR.

Discussion, Conclusions and Recommendations

Limitations of the Study

This study was a pilot, and should be regarded as such. Decisions on how to design and present VR and still image treatments, what performance variables to measure and how to measure them, and protocols for interacting with subjects and collecting data were all exploratory in nature. Difficulties were encountered in all these operationalization factors that will need to be refined in future research on desktop VR. One major decision for further research will be whether the learning performance and confidence variables used in this study are appropriate tests of the effects of VR and what additional outcome variables should be addressed. A second major consideration must be the refining of the performance measurement instruments and establishment of their validity and reliability, weakness in which limited the validity of this pilot study. Another limitation of this pilot was that it used subjects from the general population and a treatment task and naturalistic presentation settings that were general rather than technical in nature. These limitations affecting the study’s population and environmental external validity mean that direct transference of the results to a CTE population, training task, and classroom environment should not be made without further research.

Another limitation of the study was imposed by its use of a posttest-only design. Performance on the three dependent measures could have been influenced by prior skills and experiences of the subjects rather than by this exposure to VR. Additionally, without pretest measures as baselines, it was not possible to actually measure changes in learning performance or learner confidence under the treatment conditions or to verify that any improvements occurred from the VR treatment.

Conclusions and Implications

Despite its limitations, this study served several useful purposes, provided valuable methodological information to aid in the transference of desktop VR research to a CTE environment, and offered preliminary evidence of the value of desktop VR in a training task with implications for CTE. Several conclusions can be drawn from the study’s findings. First, the study supported the efficacy of desktop VR for improving learner performance and confidence in mastering a complex scenic environment. This has implications for CTE, because such environments are frequently encountered in CTE programs (e.g., laboratories, operating rooms, interiors of complex equipment, workplace sites, etc.), and if VR can be shown to improve mastery of such locational environments , this would suggest that its use may be suitable for similar learning tasks in CTE programs. In this pilot study, VR did indeed result in better scenic orientation, recall of details, and learner confidence across genders and age groups than did a presentation based on conventional still images.

Second, this study supported the authors’ supplantation-concreteness theory for predicting and explaining the effectiveness of VR. Based on supplantation ( Ausburn & Ausburn, 2003 ; Salomon, 1970 , 1972 ) and media concreteness theories ( Dale, 1954 ), it was hypothesized that the VR presentation would provide a highly realistic or concrete representation of a visual environment and would overtly perform the complex image retention and manipulation required to master a detailed scenic environment. These properties of VR were predicted to result in improved mastery and feelings of confidence or technology self-efficacy ( Bandura, 1997 ) by learners. The results of the study supported this supplantation-concreteness hypothesis, thus providing at least the beginnings of a theoretical rationale and framework for research on VR applications in CTE environments. While these results are encouraging, the generalizability of the supplantation-concreteness theory is far from established at this point. New studies by the authors are suggesting that the theory is not yet complete and that there are variables such as complexity in the VR visual field that can override the supplantation-concreteness benefits of VR and actually disadvantage the medium if not controlled.

Finally, the study’s findings of gender and age differences in performance and confidence under VR and still image treatments, and particularly the possibility of interactions of the learner variables and treatments, suggested that the supplantational and concreteness effects of VR may not be uniform across all types of learners and that some of these interactions may be contrary to expectations. In this study, findings of greater confidence overall by the younger age group and greater gains by this group in both confidence and scenic orientation performance with VR appeared to support the documented strong technology self-efficacy of these technology-savvy digital natives (Howe & Strauss, 2001; Tapscott, 1998 ). The finding that the VR treatment also yielded slightly greater confidence in the older age group may suggest a benefit for VR for the less technologically efficacious digital immigrants that merits further investigation.

The study’s findings on gender revealed some outcomes contrary to expectations. The superior performance of the females overall in scenic orientation and recall of details and their trend for greater confidence were unexpected based on a lengthy research history of stronger skills in mental spatial manipulation among males in both paper-and-paper and virtual environments ( Space, 2001 ; University of Washington, 2001 ). An explanation of these unexpected gender findings may be suggested in the related findings of greater gains in both spatial orientation and perceived confidence levels by females than by males under the VR treatment. This raises the possibility that the greater supplantation benefits were felt by the group with the greater need for the supplantation effects. This interpretation of disordinal interactions has been frequently implicit in the aptitude-treatment-interaction research model ( Cronbach & Snow, 1977 ; Salomon, 1972 ).

The interaction findings of this study suggest that in future research on the effects of desktop VR in CTE, simple main effects hypotheses for the benefits of VR should be replaced by the type of aptitude-treatment interaction (ATI) hypotheses advocated by Salomon (1972 ), Cronbach and Snow (1977) , and the present authors ( Ausburn & Ausburn, 2003) supporting supplantational instructional design. Of value may be the age and gender variables that showed potential in this study, and particularly learner variables that concern preferred styles and capabilities in cognitive processing. Although Chen, Toh, and Ismail (2005) found that VR with guided navigational tools benefited learners irrespective of their learning styles, supplantation theory suggests that there may be important interactions between VR instructional treatments and learner style characteristics, particularly when those characteristics are defined in terms of individual differences in how information is perceived and processed. These style differences were referred to in an extensive body of instructional design and psychology research as cognitive styles or cognitive controls, defined by Ausburn and Ausburn in an analysis of instructional design implications ( 1978 ) as “…psychological dimensions that represent consistencies in an individual’s manner of acquiring and processing information” (p. 338). Many classic dimensions of cognitive style/control, such as field independence/field dependence (Witkin, 1950; Witkin, Dyk, Faterson, Goodenough, & Karp, 1962); reflective/impulsive cognitive tempo and visual field processing ( Kagan, Rosman, Day, Albert, & Phillips, 1964 ); flexible/constricted field control ( Santostefano & Paley, 1964 ); and visual/haptic perceptual types ( Lowenfeld & Brittain, 1970 ) deal with methods and capabilities in perceiving and processing visual information. Cognitive processing weaknesses in visual field perception variables such as separation of figure from ground, accuracy and speed in visual scanning and details, visual distractibility, and processing of mental imagery, as defined by various dimensions of cognitive style, may be precisely what the supplantation capabilities of VR can ameliorate. Thus, may be through research applying supplantation theory and ATI hypotheses to VR instructional treatments and learners’ cognitive style characteristics that a detailed understanding of the effects of desktop VR is eventually gained. As this understanding is currently lacking, its advancement could give CTE a leadership role in instructional design research on an important emerging learning technology.

Recommendations

Based on this exploratory study and on enthusiastic reception from CTE educators in demonstration presentations, a line of inquiry on the effects of desktop VR in technical instruction is recommended. These VR studies should be moved into specific CTE applications where mastery of locational orientation, relational placement of objects, and recall of details in complex scenic environments are critical. Based on supplantation theory and instructional design, VR studies should apply ATI designs, with attention to learner gender, age, cognitive style/information processing, and technology experience variables which may interact differentially with VR’s supplantational capabilities. Depth and refinement might be added to experimental data by the addition of qualitative interviews with CTE instructors and students who are using VR treatments for teaching and learning. The opening of such a line of inquiry may lead to empirical demonstration of the benefits of new desktop VR technologies in CTE environments, an understanding of the tasks and learners for whom these technologies are most beneficial, and instructional design guidelines for effective CTE applications of VR at the desktop.

Ultimately, the critical test for desktop VR in CTE will be to compare its instructional effectiveness with that of first-hand learning experiences and to determine its usefulness in augmenting or replacing physical reality. The authors offer several recommendations to guide this line of research. First, we must learn how and when to use VR effectively. Training in many occupations requires mastery of environments that are expensive, complex, dangerous, or nearly impossible to conquer without significant risks to resources or personnel. For such situations, the efficacy of VR in the learning process would be highly beneficial for CTE. However, in the pursuit of this goal, it is critical to discover not merely that VR works, but how, why, and when it works, for it is only through this understanding that sound instructional design principles can elevate this new tool from unpredictable techno-trend to functional and reliable learning medium. Most new technologies have walked this path before, as discussed by Moore and Kearsley (2005) in their description of an anthropology approach to media research as travelers’ stories reporting personal experiences with a new technology and how well it worked. Their warning is that despite sophisticated data analysis in such studies, they remain anecdotal and can do nothing more than “point the way for research that is more controlled and systematic and that might give results that could be generalized beyond the particular case” ( 2005, p. 239).

Controlled and systematic inquiry into the efficacy of desktop VR in CTE applications rests on two pillars: (a) grounding in theory, and (b) careful evidence accumulation. Research on this new technology should avoid the naïve and simplistic assumption that “established” instructional design theories and principles necessarily apply to new technologies or to all applications of any technology. A long history of instructional technology research has shown this to be false. The authors propose that desktop VR research in CTE must be predicated on gradual accumulation of a body of empirical evidence gained through small steps in theory-based controlled tests to establish the technology’s suitability for specific CTE learning tasks, learners, and conditions. This study began this exploration by examining from a set of theoretical propositions some specific learning effects of desktop VR in a particular type of learning task commonly encountered in CTE, comparing it to the still imagery methodology that currently dominates CTE textbooks and visual instructional presentations.

Conclusion

Virtual reality (VR) technology has a record of enhancing learning performance that has been well documented in recent literature. Until recently, despite its documented instructional success, VR has had limited classroom applications because of technical complexity and very high associated costs. However, recently improved new desktop VR now offers access to this technology to classroom instructors and to students with off-the-shelf computing hardware and realistic technology skills. Desktop VR has intuitive appeal to learners who are part of what Turkle (1995) called a new culture of simulation, in which digital technologies make it possible to create, explore, and interact with real and hypothetical “worlds” in which people increasingly work and play. This new technology also has been shown to be an effective instructional tool in a small number of empirical studies. However, research on the effectiveness of desktop VR has as yet been minimal, and there has been no attempt to explore and explain its effects in terms of theoretical perspectives and models. In summary, research on desktop VR is still embryonic: little is yet known about if and when this new technology is effective, and nothing is known about why.

This study provided a first step in demonstrating positive instructional benefits of desktop VR in a specific type of learning task, within the context of a theoretical framework. The study was a pilot: small scale, highly exploratory, and constrained by limitations in both internal and external validity. However, its successful results have implications for CTE. The task environment used in the study is conceptually similar to many found in CTE, where mastery of locational orientation and comprehension of details in complex visual scenes are critical. The study also supported the potential of VR to aid the technology confidence or self-efficacy, and thus perhaps the motivation, of learners. Finally, the study was a first step in developing a theoretically-supported and interaction-based research model for discovering effective desktop VR instructional design through the view of VR as what Squire (2006) called “… designed experiences, in which participants learn through a grammar of doing and being ” (p. 19). As an emerging instructional technology that has both wide application in CTE and as-yet very limited research exposure, desktop VR offers CTE an opportunity to assume a leadership research and instructional design role. To borrow a metaphor from the technology itself, the door is standing wide open; CTE needs only to click on the hotspot, step through, and discover what may be waiting on the other side.

---

This research was supported in part by the Provost's Teaching Research Grant for the Institute for Teaching and Learning Excellence at Oklahoma State University, Stillwater, Oklahoma.

---

Lynna J. Ausburn is an associate professor of Occupational Education in the College of Education at Oklahoma State University. She can be reached at lynna.ausburn@okstate.edu.

Floyd B. Ausburn is adjunct faculty in Occupational Education in the College of Education at Oklahoma State University. He can be reached at fausburn@cimtel.net.

References