Engineering Student Outcomes for Infusion into Technological Literacy Programs: Grades 9 -12

Craig Rhodes and Vincent Childress

Introduction

In 2004, the National Center for Engineering and Technology Education (NCETE) secured funding from the National Science Foundation (NSF) to infuse engineering design into the schools through technology education. In order to reach this goal the researchers, in cooperation with NCETE, conducted a two phase study to identify outcomes for high-school students studying engineering. The first study (referred to as a Phase I) focused on students who intended to enter an engineering program after high school, answering the question:

What are the engineering student outcomes that prospective engineering students in grades 9 through 12 should know and be able to do and prior to entering into a post-secondary engineering program?

This initial study by Childress & Rhodes (2008) started with preexisting items selected from ten sources, including focus groups and national standards projects. At the end of the Delphi Round 3, very few of these items had been dropped since the consensus on all of them was high. Therefore, the researchers decided to have selected engineers categorize the outcome items into groups of conceptual likeness and to assign categorical names to the groupings. These groups then formed the basis of the instruments for the remaining Delphi rounds.

This resultant baseline of achievement outcomes for prospective engineering students was then used to design a modified Delphi instrument for the second study (Phase II) reported herein. This study focused on students who were enrolled in technology education for general education purposes and sought input from technology education teachers, teacher educators, and supervisors regarding the following question:

What are those engineering outcomes that should be taught in a high school technology education program in which the focus is general technology education program in which the focus is a general technological literacy and not pre-engineering?

Background

Wicklein’s “Five good reasons for engineering design as the focus for technology education” ( 2006 ; cf. NAE, 2004 ; Lewis, 2004 ) effectively identified the importance of this study. In presenting the advantages of infusing engineering into technology education, he stated that the optimization, analysis, and predictions are among the more important things and those students should know about engineering. This study takes the next step by identifying the related engineering design concepts to be included in a technology education curriculum.

Among recent efforts to identify engineering outcomes for high school students. Dearing and Daugherty (2004) used a modified Delphi technique to identify concepts that would prepare secondary students for postsecondary engineering, in the context of a course promoting technological literacy. Their experts were secondary and postsecondary educators.

Method

To develop the Delphi instrument for the present study, the results of Phase I were categorized and a ranked list of 40 outcomes, recommended by engineers for high school students who want to pursue postsecondary engineering education, was determined.

Twenty two technology educators served as Phase II Delphi participants: 7 high-school teachers, 6 administrators and 9 teacher educators (Table 1). Each was paid a participation stipend. Participants were selected based on the researchers' knowledge of their professional expertise, or in some cases, on the recommendation of a person deemed qualified for participation. Potential participants were asked to describe their level of knowledge of engineering and its relationship to technology education (Table 2).

Table 1.
Participant demographics

Gender
Female	13.6%	( n = 3)
Male	86.4%	( n = 19)
Race
Caucasian	95.5%	( n = 21)
African American	0%	( n = 0)
Hispanic	4.5%	( n = 1)
Age
Mean	46.8
Range	26 to 62
Years Teaching Experience
Mean	21.95
Range	4 to 37
Years Experience in Current TE Role
Mean	13.4
Range	4 to 31

Table 2.
Participant Expertise

ID No.	Yrs. Experience	Position	Self-Description of Knowledge
1	7	Classroom teacher	Described STLs and relationship to engineering
2	25	Supervisor	Strong
3	7	Classroom teacher	Past president of TE related association
4	7	Classroom teacher	BS in Mechanical Engineering
5	4	Classroom teacher	MS in Bioengineering
6	14	Teacher educator	High
7	36	Supervisor	Taught engineering for 15 years; curriculum writer
8	32	Supervisor	Chair of a standards team; wrote cross reference of standards with PLTW
9	6	Classroom teacher	Did specific research on it in college; writing standards for state
10	19.5	Classroom teacher	Role is to prepare students to major in engineering
11	36	Teacher educator	Consulted with engineering firms regularly, co-inventor of medical research apparatus, keep in close contact with engineering departments
12	37	Teacher educator	Have studied and written about engineering concepts
13	25	Teacher educator	Unsure. Called for follow-up. Is not an engineer but understands role of engineering in technology education
14	29	Teacher educator	Above average
15	18	Classroom teacher	Teach engineering
16	31	Teacher educator	Know enough to be dangerous Works with College of Engineering and has published and consulted on engineering related projects.
17	15	Teacher educator	Understand principles of engineering through applied teaching and learning activities and involvement
18	24	Supervisor	High
19	27	Supervisor/Classroom teacher	Understands the basic engineering process and tools needed to be an engineer like math and teamwork skills, knowledge of specific engineering fields is very minimal
20	30	Teacher educator	On a scale of 1 to 5, I am a 4 – better than average, not expert
21	33	Teacher educator	Very high
22		Supervisor	(none provided)

No one participating in the study was an employee of an institution that is a full partner in the NCETE. Participant #4 and #22 dropped out at Round 1. All participants who completed Round 3 were paid a fee for their full participation.

In Rounds 1 through 3, participants were given the list of outcomes items from Phase I. They were informed that the items had been selected by engineers for inclusion in high school pre-engineering programs. The Phase II participants were asked to rate these items, reword them if needed, add and rate new items, and provide comments. They were also asked to rank the importance of the items using a five point scale (Least Important, Less Important, Important, More Important, Most Important; Clark & Wenig, 1999 ). The descriptions for each of these categories are reported in Table 3. The interquartile range (IQR) was used as the statistic for variability of rating responses; an IQR of 1 was determined by the researchers to indicate consensus on an item ( Wicklein, 1993 ).

Table 3.
Explanation of ratings

	Rating Statements
1.	Least Important: Not necessary for inclusion in a technological literacy program.
2.	Less Important: Less than necessary for inclusion in a technological literacy program.
3.	Important: Necessary for inclusion in a technological literacy program.
4.	More Important: Essential for inclusion in a technological literacy program.
5.	Most Important: Most essential for inclusion in a technological literacy program.

Thirty-five of the 44 items from Phase I achieved consensus after Round 3. Three additional items were identified by participants after Round 1 and these items achieved consensus after Round 2. After Round 3, consensus had been reached on 38 engineering outcome items. Eleven items were rated as “Important” to include in a technological literacy program, 22 items were rated “More Important” and five items were rated “Most Important.” For comparison, after three rounds the participants in Phase I reached consensus on what should be taught 1 ^st , 3 ^rd , and 7 ^th places in order of importance. However, in Phase II after three rounds, the participants reached consensus on the order of only one grouping, Emerging Fields of Engineering, and it was ranked lowest in importance. In Phase II, no engineering outcomes items were dropped due to a low median importance ratings; however, those outcome items that were not included in the final list were those for which consensus was not reached. The final engineering outcome item ratings and group rankings for Phase II are presented in Table 4. A complete statistical analysis of all data, (including non-consensus items) is available at http://www.ncete.org/flash/Outcomes.pdf.

Table 4.
Third round consensus ratings of engineering outcomes and rankings of the seven outcome groupings

Item Rating	Rank	Group: Engineering Design Regarding engineering outcomes for Engineering Design the student in grades 9-12:
* Indicates a consensus item added by the participants themselves.
4	Rank Undetermined	Is aware of how engineering principles must be applied when designing engineering solutions to problems.
4		Understands that creativity is an important characteristic for engineers to apply in design.
4		Believes in his/her ability to design a solution to a problem.
5		Recognizes that there are many approaches to design and not just one “design process.”
3		Understands engineering as it is actually practiced as a future career option.
		Group: Application of Engineering Design Regarding engineering outcomes for Application of Engineering. Design the student in grades 9-12:
4	Rank Undetermined	Designs, produces, and tests prototypes of products.
4		Understands that there is no perfect design. Designs that are best in one respect may be inferior in other ways (cost or appearance). Usually some features must be sacrificed as trade-offs to gain other features.
3.5		Conducts reverse engineering and can analyze how a product or process was designed and created.
4*		Understands how to work well on multidisciplinary teams.
4		Applies research and development and experimentation in the production of new or improved products, processes, and materials.
		Group: Engineering Analysis Regarding engineering outcomes for Engineering Analysis the student in grades 9-12:
4	Rank Undetermined	Uses models to study processes that cannot be studied directly.
4		Applies mathematics and science to the engineering process.
4		Understands that knowledge of science and mathematics is critical to engineering.
3		Uses a physical or mathematical model to estimate the probability of events.
3		Uses optimization techniques to determine optimum solutions to problems.
		Group: Engineering and Human Values Regarding engineering outcomes for Engin. and Human Values the student in grades 9-12:
4	Rank Undetermined	Practices engineering ethics.
5		Is aware of how societal interests, economics, ergonomics, and environmental considerations influence a solution.
5		Understands how other factors, such as cost, safety, appearance, environmental impact, and what will happen if the solution fails must be considered when designing engineering solutions to problems.
4		Takes human values and limitations into account when designing and solving problems.
5		Understands that the solution to one problem may create other problems.
		Group: Engineering Communication Regarding engineering outcomes for Engin. Communication the student in grades 9-12:
4	Rank Undetermined	Understands basic personal computer operations and uses basic computer applications such as word processors, spreadsheets, and presentation software.
4		Provides basic technical presentations, graphics, and reports, and communicates verbally information related to engineering processes
4		Uses technical drawings to construct or implement an object, structure, or process.
4		Visualizes in three dimensions.
3		Understands computer-aided engineering.
3		Applies the rules of dimensioning and tolerancing.
4		Uses computer-aided design to construct technical drawings.
		Group: Engineering Science Regarding engineering outcomes for Engineering Science the student in grades 9- 12:
5	Rank Undetermined	Develops basic ability to use, manage, and assess technology.
3		Applies knowledge of basic ergonomics to the engineering process.
4		Develops basic skill in the use of tools for material processes.
4		Applies basic power and energy concepts.
3.5		Applies knowledge of the processes for manufacturing products to the engineering process.
4		Applies knowledge of material processes to the engineering process.
4		Applies knowledge of basic mechanics to the engineering process.
3		Applies knowledge of basic dynamics and motion of rigid bodies and particles to the engineering process.
4*		Understands open and closed loop systems.
3*		Describes the sources, basic chemical structure, recycling potential, and environment al impacts of widely used industrial materials.
		Group: Emerging Fields of Engineering Regarding engineering outcomes for Emerging Fields of Engin. the student in grades 9-12:
3	Rank 7 th	Understands the importance of nanotechnologies in developing the next generation of innovations (less power, smaller).

As indicated in Table 5, the consensus rate for technology educators in Phase II was very similar to the consensus rate for the engineers in Phase I.

Table 5.
Comparison of percentage of consensus outcome items per round per phase

Round	Tech Educators (Phase II)	Engineers (Phase I)
1	41%	42%
2	62%	63%
3	80%	78%

With the exception of the Emerging Fields of Engineering group, it was difficult for both groups of participants (engineers and technology educators) to reach consensus on outcome grouping. Among the plausible reasons are that both phases dedicated only three rounds to consensus building for the groupings, and the initial groupings were juried instead of being crafted by a complete Delphi process.

Discussion

As of Round 3, there are several consensus items that provide reinforcement of the importance of the engineering design processes that the NCETE has selected as its professional development focuses: constraints, optimization, prediction, and analysis (COPA). Other technology educators may be interested in these outcomes also. Those items are presented in Table 6.

Table 6.
Consensus items related to constraints, optimization, prediction, and analysis

		Round 3			Round 2			Round 1
Item	Outcome	IQR	Mdn	SD	IQR	Mdn	SD	IQR	Mdn	SD
14*	Uses models to study processes that cannot be studied directly.	0*	4	.49	.75*	4	.92	1*	4	1.05
15*	Uses optimization techniques to determine optimum solutions to problems.	1*	3	.51	1*	3	.74	1*	3	.98
16*	Applies mathematics and science to the engineering process.	.75*	4	1.02	1.75	4	1.09	1.25	4	1.01
17*	Uses a physical or mathematical model to estimate the probability of events.	1*	3	.85	1.75	3	1.05	1.25	3	1.09
19*	Understands that knowledge of science and mathematics is critical to engineering.	1*	4	.95	1*	4	.95	1*	4	1.00

The use of models for indirect study such as analysis or prediction (Item 14) had an IQR of zero. Typical comments in support of the More Important rating include the following:

• I believe the ability to use models—mathematical, physical, and virtual, is one of the most important skills we can teach in technology education.
• Modeling is part of the 3 – 12 STL Standards and should be central to what we teach since modeling is such a powerful and universal tool.
• Making decisions based on models before construction is valuable.

Optimization (Item 15) was rated as Important to include in a technological literacy program. Comments related to optimization include the following:

• Calculus is not needed for problems solved related to a technology education course.
• For technological literacy, applying math and science to a design process would be vital for optimization and adhering to constraints, as well as analyzing data during testing.
• This is what engineers do. Don’t bother to teach “engineering” if you take the math and science out. There is plenty that can be done with high-school appropriate mathematics.

Mathematical analysis-related Items 16 and 19 and predication-related Item 17 were rated More Important and Important respectively to include in a technological literacy program. Supporting comments related to the use of mathematics and science for the analysis items included the following:

• STL currently calls for recognition of such connections between science and mathematics to all technological processes.
• The way this is stated, it seems more like a value or feeling than a technical concept or skill and therefore not that difficult to comprehend. It does not state that one know how to apply science and mathematics to the study of engineering which in my opinion would warrant greater importance.
• Learn by doing

Comments supporting the prediction outcome included the following:

• While the background needed to develop mathematical models on their own may be lacking, students need to be able to work with these models in a meaningful way—not just plugging in data, but interpreting formulas and results, optimizing, etc. “Probability” is really the wrong term, though.
• Virtual modeling is critical to understanding the engineering process because it allows for iterative processing without cost of trials. Perfect for tech ed because you get to simulate what engineers do with limited resources.
• We in TE need to step up and model our solutions and outcomes more

Table 7.
Consensus items related to constraints

		Round 3			Round 2			Round 1
Item	Outcome	IQR	Mdn	SD	IQR	Mdn	SD	IQR	Mdn	SD
21*	Is aware of how societal interests, economics,…environment al…influence a solution.	1*	5	.604	1*	5	.606	1*	5	.790
22*	Understands how other factors, such as cost…must be considered when designing…	1*	5	.995	1*	5	1.08	1*	5	1.07

Constraints (Items 21 and 22) were rated as Most Important to include in a technological literacy program. See Table 7. Comments in support of the importance of constraints were.

• Considers when designing should be used
• This is why technological literacy (an outcome we should hope to attain in all students, not just those seeking engineering careers in the future) is so important
• Definite connection to real life: "each action or decision could affect others."

Technology educators, interestingly, rated approaches to the design process differently than did engineers. The technology educators also rated technical drawing slightly higher than did the engineers. After Round 3 for this grouping, the engineers ended with an IQR of 1 while the technology educators has an IOR of zero, as required in Table 8.

Table 8.
Consensus items related to the design process

		Round 3			Round 2			Round 1
Item	Outcome	IQR	Mdn	SD	IQR	Mdn	SD	IQR	Mdn	SD
6*	Recognizes that there are many approaches to design and not just one “design process.”	1*	5	.605	1*	5	.605	1*	5	1.06
28*	Uses technical drawings to construct or implement an object, structure, or process.	0*	4	.459	0*	4	.887	.5*	4	.669

Supporting comments related to recognizing that there is a variety of design processes included:

• General technological literacy ability.
• It is important, but learning the design process as something that is discursive would be fundamental, and perhaps learners who can be identified as more creative would be encouraged to take multiple approaches to design…
• Multiple solutions approach to learning is a characteristic of a technologically literate person

Comments related to support of the technical drawing outcome included.

• Without technical drawings to plan the construction, students are just creating art, no? How do you replicate a design if needed?
• Free hand sketching is needed and can be taught. No time for CAD class prior to TE class. Good class if you go to college for Eng[ineering] Vocational[y].
• I will move to the majority here because your designs and ideas are only as good as they can be expressed to another who might have to use your graphic representation of them in your absence.

These findings were similar to those of Dearing and Daugherty in their modified Delphi study.

Technology educators added an item that did not appear on their Round 1 instrument because it was not a consensus item for the engineers. However, it did appear on the Round 1 instrument for the engineers. This item relates to the NAE’s prediction that working on teams will be very important to engineering in the future (Item 47 below in Table 9).

Table 9.
Consensus item related to design teams

		Round 3			Round 2
Item	Outcome	IQR	Mdn	SD	IQR	Mdn	SD
Added 47* add to Grp 2	Understands how to work well on multidisciplinary teams.	1*	4	.67	1*	4	1.07

Comments related to support of the multidisciplinary teams outcome included:

• This is a critical skill addressed by ABET that is applicable to all students in technology education.
• Understands how to work well on multidisciplinary design teams.
• Important addition

Technology education participants did not reach consensus on the importance of managing the engineering design process. Related comments are included following Table 10 below.

Table 10.
Statistics for non-consensus item related to managing the engineering design process

		Round 3			Round 2			Round 1
Item	Outcome	IQR	Mdn	SD	IQR	Mdn	SD	IQR	Mdn	SD
8	Organizes and manages the engineering. design process that includes optimal use…materials. processes…	2	4	1.02	2	3.5	1.03	2	4	1.12

• Optimizing is covered below. I do not feel that “engineering management and organization” as a process should play a significant role in technological literacy.
• Important for engineering literacy but not essential for technological literacy. I would prefer the word “engineering” be deleted from this learning outcome statement.
• Important for engineering literacy, but less important for technological literacy.

Dearing and Daugherty (2004) described a modified Delphi study that they conducted with technology teachers, technology teacher educators, and engineering educators. The purpose of the study was to identify those curricular concepts that are necessary to teach high school students in order to prepare them for postsecondary engineering education, while preserving the mission of teaching technological literacy. Dearing and Daugherty developed a predetermined list based on information from Project Lead the Way, CORD’s Principles of Technology, The Standards for Technological Literacy, ASEE, and others. Participants were to decide if a concept should be included in such a curriculum.

Dearing and Daugherty measured consensus in terms of an item’s standard deviation. Fifty-two concepts on their list met the criterion for consensus and were retained. While the purposes of the Dearing and Daugherty study differed from this study, they were both focused on technological literacy. Twenty-nine of the engineering consensus items in the Dearing and Daugherty study overlap conceptually with the consensus items in this study.

Finally, it should be noted that the Corporate Member Council of the American Society for Engineering Education has been working on a set of “…National Content Standards for K-12 Engineering/Engineering Technology…” ( Morrison, 2007 , p. 1). Most of the 70 outcomes derived from these standards overlap with the findings of this study. However, it should be noted that the purposes of the Corporate Member Council’s study differ from the purpose of this study. The final findings of the Corporate Member Council’s study were not available at the time this article was written.

Recommendations

The following recommendations are offered to technology education teacher educators, technology education teachers, and technology education administrators.

1. Conduct professional development in which teachers are provided the opportunity and guidance to infuse those engineering outcomes agreed upon by the participants into the teachers’ own technological literacy curricula.
2. Enhance technology education by infusing selected engineering outcomes into the technology education curriculum for non-pre engineering curricula.

Implications for Technology Education Curriculum and Instruction

There are at least two primary points of view regarding the application of the items identified in this study to general technological literacy programs—those that are not pre-engineering. One is that a curriculum could be designed that is heavily influenced by engineering contexts. Those outcome items identified herein that overlap with the Standards for Technological Literacy would address the same standard, but the delivery would be set in the context of engineering. A second point of view is that the outcomes identified herein would be used sparingly in a program intended to develop technological literacy; engineering would be one of many contexts and topics of study included. Although agreement was reached on only a few groupings, they may still be of use to teachers insofar as the engineers agreed that Group 1, Engineering Design, was the group of outcome items of primary importance in a curriculum that is crowded and has limited time to dedicate to engineering outcomes. Likewise, Group 7, Emerging Fields of Engineering, represents the least important grouping that one would want to teach if time were limited. Finally, a crowded curriculum will actually provide less opportunity for students to learn about engineering in meaningful ways—meaningful ways that will tend to attract underrepresented populations to STEM areas. Just because consensus was reached on as many as 38 different outcome items in this study does not mean that all of them should be taught in one course. They should be applied as needed and when pertinent over the span of the ninth through twelfth grades.

---

Craig Rhodes ( rhodesc@ncat.edu )is an Associate Professor and Vincent Childress ( childres@ncat.edu ) is a Professor in the Technology Education Program at North Carolina A&T State University, Greensboro

References

Author Notes

The researchers would like to thank the participants in this study for their dedication and thoughtfulness. The researchers would also like to thank the National Center for Engineering and Technology Education for its support of this study.