A Comparison of the Effectiveness of Two Computer-Assisted Instruction Methods for Learning Architectural Concepts in Design Studio
Taeyeol Park and Valerian Miranda, Ph.D.
Department of Architecture
Texas A&M University
College Station, TX 77843-3137, USA
Keywords: Architectural precedent study, Concept learning, Design Studio teaching, CAI (computer-assisted instruction)
In design studio, during the preliminary stage of a project, learning architectural concepts through the study of precedents is a common activity. A few computer-based tools for precedent study have been developed, however, most have focused on delivering information without considering aspects of learning concepts. Although the tools apply digital media, such as hypermedia and multimedia, their instructional method is limited to direct instruction for presentation of information. The design and development of the tools neglect the need of the study of concept learning, learners, and learning methods.
This paper describes a pilot study to compare two different instructional methods, direct instruction and discovery, in computer-assisted instruction (CAI) format, with a view to find which is more effective for learning architectural concepts through the study of precedents in design studio. In order to attain the objective, this study is designed to find differences of students' cognitive gains from concept learning by the two different methods, and to see how these differences interact based on differences in learning tasks.
Two different CAI lessons based on the two different methods and a test are designed and placed on the Web. Both of the lessons represent the same concepts related to spatial composition in architectural design, but their instructional methods are different. The lesson based on the direct instruction method outlines the concepts with concept labels and definitions, and proceeds from them to specific examples which illustrate the concepts. The lesson based on the discovery method presents examples and non-examples, and then encourages students to abstract a certain concept from them. The test consists of 38 multiple-choice items to measure the subjects' concept acquisition according to concept learning tasks: recall, distinction, and recognition.
Seventeen undergraduate architecture students at Texas A&M University were randomly assigned to two groups, each group was administered a "lesson" conforming to the instructional techniques being examined. All participants were then given the same test. Results from the test and subsequent questionnaires are analyzed and compared in this paper.
Design Studio is at the core of architectural education and there is much variation in the way studio is conducted, however, it is common practice to study precedent examples of buildings in the preliminary stages of a studio project. This study of precedents is used by instructors in different ways, one of which is to understand the concepts of spatial organization. Typically, an instructor prepares material containing examples of the concepts to be learned. This material is then presented to students in the form of slide lectures, library reference lists, handouts, etc., before any discussion takes place. This system has been an effective method for many decades, however, it has always suffered from the constraints of time and place, i.e., all concerned must be gathered in the same location at the same time for the method to have greatest effect. With the progress and popularity of digital technology, it is now possible to accomplish much of the same learning in a more flexible manner, possibly in a more effective manner.
Drawing on previous work of others in the Computer-Assisted Instruction (CAI) field, the authors set out to examine the relative effectiveness of two particularly suitable CAI strategies, the Direct Instruction Method and the Discovery Method. The Direct Instruction Method closely resembles the presently used technique of presenting material of example buildings that illustrate the concepts by way of drawings, pictures and text descriptions. Instead of the materials being presented in person, these materials were formatted in a manner conducive to be reviewed by students in a hypertext format using any popular browser. The Discovery Method involved constructing "lessons" in a hypertext format in which the concepts to be learned were not directly presented, rather, the students were encouraged to review material of examples and non-examples and formulate the rules and characteristics of the concepts being studied. In education research these two methods have their supporters and critics with no group being clearly at an advantage.
DESCRIPTION OF EXPERIMENT
For the purpose of describing the experiment for this paper, we have divided the experiment into the following steps:
Selection of Material
The study of precedents to be used in the experiment were taken from a design studio at the undergraduate level. The concepts to be learned were those of spatial organization with an emphasis on symmetry, repetition , and hierarchy. The concepts and their relevant examples were adapted from Clark (1985), Ching (1979), texts that are commonly used as design studio references. Each example consists of a main plan, a whole view, an axonometric view, an elevation or section, information about the architect, design date, the location and the name of the building.
Designing the lesson was based on Merrill's (1992) design guide for a concept lesson. Implanting the lesson in a CAI format was based on the CAI lesson design principles of Hannafin (1988) and Rieber (1994). Applying the CAI lesson to the Web environment was based on Web design guides of Morris & Hinrichs (1996) and Weinman (1996). The lesson systems were prepared with HTML (HyperText Markup Language) and CGI
(Common Gate Interface), and placed on an Internet site to be accessed only by the students of a design studio class.
Design and Programming the "lessons"
The Direct Instruction Method lesson provides students a concept label and definition which includes attributes and elements of the concept, and presents examples of the concept and information about the example. Figure 1 shows the structure of the "lesson" for the Direct Instruction method. This "lesson" seeks to impart knowledge about the spatial principle ORDER through the understanding of the concepts of SYMMETRY, REPETITION, and HIERARCHY. Examples of each concept are grouped by the elements AXIS, POINT, SIZE, SHAPE, and LOCATION. Each element group has two examples designated as EX 1 and EX 2.
The lesson consists of a series of pages, each page of which consists of three sections, the top section shows this lesson structure as a map. This section also provides a locator and a navigation button. The arrowhead tells a user's current position in this lesson, and the underlined text is a button which takes the user to the next page. The middle section is for information of the concepts, the elements, and the examples. The bottom section contains two types of directions, one is for going to the page, and the other for going back to the previous page. The lesson starts from the principle, ORDER, and goes to the concept and to its elements and its example. The progression for learning the concept SYMMETRY is as follows:
ORDER > SYMMETRY > AXIS > EX 1 > EX 2 > POINT > EX 1 > EX 2
The procedure for learning the other concepts is the same as the above. Figure 2 shows a sample page which presents the concept, SYMMETRY. Figure 3 shows a sample page which presents the examples of a concept element, AXIS. Figure 4 shows a sample page which presents the information about an example of the concept.
In the Discovery Method lesson, students are asked to find similarity from four instances (two examples and two non-examples) of a concept and to select two examples based on the similarity. After their correct selection, they are asked to label the concept of the examples and its elements from the definition of the concepts and the information about the examples of the concept. Figure 5 shows the structure of the "lesson" for the Discovery method. This lesson is for the same three concepts SYMMETRY, REPETITION and HIERARCHY but are labeled as CONCEPT 1, CONCEPT 2, and CONCEPT 3 which are parts of spatial organization principle, ORDER. The examples of each concept are grouped by their characteristic elements, AXIS, POINT, SIZE, SHAPE and LOCATION, but are labeled as ELEMENT 1, ELEMENT 2, ELEMENT 3, etc.. Each element group has examples labeled as EX 1, EX 2, etc..
This lesson also consists of a series of pages, each page of which has three sections, the top and middle sections are the same in each page, and the bottom section are different in every page through the lesson. The top section shows this lesson's whole structure as a map. The arrowhead takes a user to the pages he/she needs. The popup menu box is used when the user is asked to select names of the concepts the user learned in this lesson. The middle section has definitions of concepts and the end button. The definitions are used when the user is asked to select names of the concepts and the elements. After learning the all concepts and selecting all the names of concepts and elements, the user clicks the end button. The bottom section provides main place for information about concepts. It also contains directions for the lesson process.
The lesson starts from the principle, ORDER, and goes to the concept and to its elements and its examples. In the CONCEPT page of the lesson, four instances (two examples and two non-examples) of the concept are presented, and students are encouraged to abstract a certain concept from them. Students must discern the common attributes and elements from the instances. They are asked to select the examples of a concept from the given instances. Figure 6 shows a sample page from which they must select a group of two examples. Figure 7 shows a sample page which contains detailed information about the example building. After selecting correct examples and reviewing the information of the examples, they are asked to label the concept of the examples and its elements from the definitions of the concepts. After learning all the concepts and selecting all the names of the concepts and elements, they are asked to review the names selected.
Constructing and administering test
The purpose of testing students was to find a difference between the two groups scores according to instructional methods and concept learning. The test was designed to measure a students capability to identify correctly the class membership of an example of the concept learned from the CAI lesson. The capability is measured according to the three types of prototype concept learning tasks, namely, recall, distinction, and recognition, as defined by Martorella (1972).
The test consisted of 38 multiple-choice items to measure students' concept acquisition according to the concept learning tasks and concepts in the lesson. Table 1 shows the distribution of questions according to the concepts to be learned and the learning tasks. Test questions for recall give students a concept definition or attribute and ask them to select a name of the concept as shown in the example in Figure 8. Test items for distinction give students a concept name and ask them to select an example of the concept as shown in the example in Figure 9. Test items for recognition give students examples of a concept element and ask them to select the element name as shown in the example in Figure 10.
In order to insure that the test does measure concept learning performance, the design of the test was based on Merrill's(1992) guidelines of a concept classification test. For the validity of the test, all test items are newly encountered instances and sequenced randomly to prevent students from detecting patterns and guessing. The collection of examples and non-examples for the test items were reviewed and determined by a design faculty committee.
Administering "lessons" and test
Seventeen students from a design studio were administered the CAI lessons. The students were randomly assigned to two groups, one was administered the Direct Instruction Method lesson and the other was administered the Discovery Method lesson simultaneously. No time limits were set, though, through pre-testing of other individuals, the researchers expected the duration to be between thirty and forty-five minutes. Individual students accessed the lessons and test through a password system and the duration of the lesson and test was recorded by the computer. The answers were recorded by the computer and forwarded to an e-mail account. After completing the test students filled out two questionnaires, one dealing with the content and interface of both the lesson and the test, and the other dealing with their perceptions of the usefulness, benefits, difficulty, future directions, etc.. of the CAI lessons.
Collecting and analyzing data
As stated above, the data from the test was collected electronically and additional data was collected manually through filling out questionnaires. Test scores were collated, analyzed and summarized. Questionnaire data was sorted, categorized and analyzed. The results were then compared and tentative conclusions are described below.
COMPARISON OF INSTRUCTION METHODS
Tabulated test scores shown in Table 2 and Table 3 indicate subtle differences between the two instructional methods. The tabulation has been prepared under the two major headings of Learning Tasks as evidenced by Recall, Distinction and Recognition and Concepts as evidenced by the concept elements of Symmetry, Repetition and Hierarchy. Total Scores and Time Taken have also been tabulated.
Scoring in learning tasks - As seen in Figure 11 mean test scores of the two groups (Direct 87, Discovery 79) were slightly different in the Recall category, but the mean scores were almost identical in the Distinction (Direct 74, Discovery 74) and Recognition (Direct 66, Discovery 67) categories. If we consider only the scores, we can conjecture that instructional method does not affect concept learning, distinction and recognition. Based on the students' responses and comments in the evaluation and the survey, however, we realize that the discovery method can stimulate, challenge and interest students. These are important factors in the effectiveness of concept learning.
Scoring in concepts - Though the Discovery group obtained a lower mean score in Symmetry (Direct 91, Discovery 85) and Repetition (Direct 75, Discovery 66) concepts, the group achieved a higher mean score in the Hierarchy (Direct 61, Discovery 67) concept than the Direct group. The graph in Figure 12 shows that the mean scores dropped significantly from Symmetry to Hierarchy indicating the difficuly level of the concept and also indicating that the Discovery method may be more effective with difficult concepts that the Direct method.
Scoring in concept learning - The difference of the mean test scores between the two groups (Direct 76, Discovery 73) was very small, however, as can be seen from the box plot in Figure 13 the variation was much greater for the Discovery group. This may indicate that individual differences of learners should be considered in designing a lesson based on the discovery method.
Time for lesson and test - There was little difference between average times for the lesson and the test of the two lesson groups (Direct 37 min., Discovery 39 min.). This result was a surprise as the researchers expected the time to be much longer for the Discovery group.
Summary of Survey Questionnaires
After taking the lesson, students were asked to comment on the use of the lesson for learning architectural concepts. The comments of the Discovery group were more enthusiastic than those of the Direct group. Although many of the students from the Direct group made no significant comments, students from the Discovery group made more detailed comments showing their interest. The comments indicated that:
a) The CAI lesson was more "interesting that looking up a book".
b) Students took a little time to get used to the format.
c) More detailed information on the examples would be welcome by the students.
d) The lessons were helpful.
In response to questions related to difficulty of spatial concepts, difficulty of learning concepts, and interface of the lessons, the questionnaire data showed:
Difficulty of spatial concepts to be learned - One of the items in the questionnaire asked the students to rate the difficulty of the concepts SYMMETRY, REPETITION and HIERARCHY they learned in the lesson. The graph in Figure 14 shows the summary of their responses and indicates that all three spatial concepts were more difficult for the Discovery group than for the Direct group. The graph also indicates that the concept HIERARCHY is the most difficult concept to be learned in both groups. These indications are validated by the test scores for each spatial concept.
Difficulty of the learning concepts in the two lessons - The students in the Direct group indicated little difficulty in the learning concepts. However, there were many more responses that indicated difficulty of the learning concepts in Discovery group. Most of the difficulty, seemed to disappear in the middle of the lesson and students seemed to understand the initial difficulty was necessary for this method because they unfamiliar with the learning method. Below is a summary of reasons for the initial difficulty students of the Discovery group encountered.
Interface of the two lessons - Students were queried on the quality of the guidelines, instructions, organization and ease of navigation in and any difficulty posed by the interface. Responses show that there was no difficulty in learning or using the interface of the lesson in either group. This equivalence of the interface in the two lessons helps the comparison of the instructional methods by eliminating interference normally caused by interfaces.
At the present time, the Direct Instruction lesson, the Discovery lesson, the test and the questionnaire are being fine tuned. A larger number of participants will be used in the next phase of this work in order to obtain results that are statistically valid. The use of qualitative assessment techniques will also be used to gather information that we hope will be useful in developing Computer-Assisted Instruction materials for a number of different subjects that have relevance to architectural education. With the continuing advancements in digital connectivity and communication much of this CAI material can be made available to students all over the world, thereby diminishing the traditional constraints of time and place.
Ching, F. D. K. (1979). Architecture: Form, Space & Order. New York: Van Nostrand Reinhold Company.
Clark, R. H., & Pause, M. (1985). Precedents in Architecture. New York: Van Nostrand Reinhold.
Hannafin, M. J. & Peck, K. (1988). The Design, Development, and Evaluation of Instructional Software. New York: MacMillan Publishing Company.
Martorella, Peter H. (1972). Concept Learning: Designs for Instruction. San Francisco: Intex Educational Publishers.
Merrill, M. D., Tennyson, R. D., & Posey, Larry O. (1992). Teaching concepts: An instructional design guide. Englewood Cliffs, NJ: Educational Technology Publications.
Morris, M. E. S. & Hinrichs, R. J. (1996). Web Page Design: A Different Multimedia. Mountain View, CA: Sun Microsystems.
Rieber, L. (1994). Computers, Graphics, and Learning. Madison, WI: Brown & Benchmark.
Weinman, L. (1996). Designing Web Graphics. Indianapolis, Indiana: New Riders Publishing.
DIRECT INSTRUCTION METHOD GROUP
|ID number||Recall||Distinct-ion||Recog-nition||Symmetry||Repetition||Hierarchy||Total Score||Time Taken|
DISCOVERY INSTRUCTION METHOD GROUP
|ID number||Recall||Distinction||Recognition||Symmetry||Repetition||Hierarchy||Total Score||Time Taken|