ANSI Standards Committee on Dental Informatics - Working Group Educational Software Systems
V. 1.2 8/1/2001Please note: Version 1.2 of the Guidelines is a significant revision of the prior version based on input from multiple groups and individuals during the 1999-2001 period. The Guidelines were also modified as a result of applying evaluation instruments derived from them to educational software. We thank all individuals and groups who participated for their help in improving the Guidelines.Please send comments to:Titus Schleyer, University of Pittsburgh School of Dental Medicine (email@example.com)
Lynn Johnson, University of Michigan School of Dentistry (firstname.lastname@example.org)Development of these guidelines supported by:University of Pittsburgh Center for Dental Informatics
University of Iowa Instructional Technology and Research Program
American Dental Association
DentEdEvolvesDevelopment of this Website partially supported by:SIPP Global Healthcare Applications Project
The development of the Guidelines for the Design of Educational Software began in 1995. At that time, the Accredited Standards Committee Medical Devices 156 (Task Group on Dental Informatics) established Working Group 5 (Educational Software Systems). The charge of this working group was to propose standards for student, patient, and practitioner educational software. More recently, the ASC MD 156 Task Group on Dental Informatics was elevated to the ANSI Standards Committee on Dental Informatics. The Working Group for Educational Software is continuing to develop the Guidelines within the new committee.
The Guidelines have undergone significant development since the inception of this project. In 1998, the project was first described in the literature (1). Subsequently, an evaluation study of the preliminary guidelines with 157 Web-based continuing dental education courses was published (2, 3). The guidelines were then continuously refined, and served as the basis for the inaugural Educational Software Competition of the American Dental Education Association (4). In the spring of 2001, the guidelines were reviewed and augmented in a working meeting of DentEdEvolves, a European consortium of dental schools. The latest revision also includes changes made in response to comments received in early 2001 as a result of posting of the Guidelines for public comment by the Standards Committee on Dental Informatics. The current release, V. 1.2, is the result of this process.
The purpose of the Guidelines is to promote quality in educational software. They do so in two ways. First, developers can use the guidelines to ensure that their products are of high instructional quality during development and evaluation of their products. Second, end users can compare educational software programs with the Guidelines to recognize quality products. For the second goal, we are deriving validated rating instruments for use by instructional design experts and software end users.
We plan to continue to refine the current Guidelines for the Design of Educational Software considerably through discussion among instructional design experts and application to instructional design projects and evaluation studies. We encourage you to review these guidelines and send us your comments using the Comment Form.
The Working Group for Educational Software has chosen to focus on proven educational strategies that work in a computer-based environment. We recognize that technology is constantly changing and many of today's technological standards will be outdated tomorrow. Thus, the Guidelines provide a conceptual framework for quality assurance that does not restrict technical innovation, and that provides developers a large degree of flexibility in design.
The Guidelines are also not domain-specific. Despite the fact that they are being developed in a dental standards committee, they are purposely conceived and formulated to be general. Thus, the Guidelines are applicable to any domain, not just the dental or medical one.
For the foreseeable future educational software will be delivered in two environments -- stand-alone (or in closed environments, such as local area networks) and on the World Wide Web. Stand-alone programs are broadly defined as instructional software packages that operate as an executable program on a single microcomputer. World Wide Web (Web)-based software operates over the Internet and uses a Web browser. The Working Group's major objective is to establish guidelines for quality software, that is, instructional software that has clear learning objectives and fulfills those objectives in a meritorious way that permits the learner to focus on the content and not on operating the software.
The standards proposed by the Working Group center around features of educational software independent of technology. To a certain degree, the guidelines can be applied to educational materials of any type. Other recommendations apply directly to computer-based educational media. Thus, the recommendations allow for innovation on a technical level, while providing a framework to help providers develop offerings of high educational and pedagogical value.
The Working Group attempted to identify existing standards applicable to the design of educational software in dentistry. Unfortunately, currently no such standards exist. The Working Group then reviewed the educational literature to develop recommendations for computer-based educational software.
The recommendations for educational software cover two levels. The first level is applicable to any type of educational software, while the second is specific to certain types of software (such as tutorials, drills, games and simulations). This Website only covers guidelines for the first level.
Quality standards applicable to any type of software include Pedagogical Issues; Subject Matter; Language, Grammar, and Format; Surface Features; Assessment; Feedback; Invisible Functions; Off-line Materials; Evaluation; and Other. Pedagogical Issues address aspects such as: Is the computer appropriate for the type of materials? To what degree can users control aspects of the software (such as flow and pacing)? How are users motivated? How do they interact with the software? Is the use of multimedia appropriate? Subject Matter covers content-related aspects of the software, such as goals and objectives, information presented, content emphasis, and organization and sequence. Language, Grammar, and Format includes aspects such as reading level, cultural bias, technical terms, spelling, grammar, punctuation, and formatting and layout. Surface Features consist of esthetics, look and feel, presentation modes, basic control features, and user input. Assessment addresses how the software should assess learning outcomes and how the user interacts with the software to demonstrate those. Invisible Functions address software functions that are not visible to the user, such as record keeping within the program, security and accessibility, and the ability of the program to restart after failure. Off-line Materials provides guidelines for materials that accompany the software, such as manuals and workbooks. Evaluation covers the important aspects of formative and summative evaluation. A recently added category, Other, provides a classification for criteria that cannot be directly associated with any of the previous category. Over time, items from the Other category may be grouped into new categories.
Pedagogy is the most important general aspect of any instructional project. Principles of pedagogy help the teacher select the most appropriate methods to achieve a specific instructional objective. Instructional software must use proven and effective instructional techniques that address the needs of the audience.
Course designers must address general pedagogical issues such as appropriateness of the computer, appropriateness of the methodology, student practice, lesson length, and mastery level. Where possible, educational software should adapt to the learner's skills and knowledge.
Educational software should motivate the user. Use of the computer alone does not guarantee motivated users. Interaction is a valuable tool to keep the users' interest, and to reinforce learning content.
The media used in the course should be appropriate for the content. For instance, video clips should illustrate processes and images visual and/or spatial concepts.
If the software is interactive, students should be able to control the program in several ways, such as by determining pace, navigating randomly, or book marking places in the program.
General pedagogical aspects pertain to all types of educational software, regardless of content, objectives and/or audience.
1.1.1. Is the computer appropriate? The courseware should take advantage of the unique capabilities of the computer that are not available in another medium such as videotape, a book, or a live patient.
1.1.2. Is the methodology appropriate? The instructional methodologies selected should match the content and audience requirements.
1.1.3. Is spaced practice encouraged? Practice in applying information should occur not only immediately after instruction, but should be spread out over a number of occurrences.
1.1.4. Is lesson length appropriate? Lesson length should be based on the content and the audience. Ideally, instructional content should be a series of short lessons spread over several sessions.
1.1.5. Is mastery level appropriate? The mastery level needs to match the audience and the methodology. Tutorials are often used to provide basic instruction and have a lower mastery expectation than drills that are intended to establish mastery.
1.1.6. Does the lesson adapt to the learner? Whenever possible, the lesson should be customizable. There can be different versions for different languages and different audiences. A restorative dentistry continuing education course may be completed by both dentists and dental assistants, but have different paths, one for each audience. Another example of adaptation is a pre-course questionnaire that tests the users' knowledge in order to customize the course on the fly.
The software should support the user's motivation to learn. The use of the computer alone is insufficient to guarantee motivated learners. Motivating factors must be built-in and anxiety minimized.
1.2.1. Is motivation intrinsic? Intrinsic motivators are found within the instruction. Techniques that have proven effective include game techniques, exploratory environments, visual embellishments, increased student control, challenge, and encouraging feedback.
1.2.2. Is computer anxiety minimized? This should be done via an user-appropriate interface and safety nets. Safety nets prevent accidental termination or loss of data by the user.
1.2.3. Is the level of challenge appropriate? As students progress the level of difficulty, or challenge, should increase.
1.2.4. Are curiosity and confidence maintained? Curiosity can be maintained by building in novel or unusual techniques. Confidence is maintained by recognizing accomplishments and giving supportive feedback.
1.2.5. Is competition appropriate? Competition between the computer and an individual is viewed as being safer than competition between individuals. Competition should support instructional goals.
1.2.6. Is motivation balanced with other instructional factors? Balance between learning and motivation needs to be maintained. Learning should not be sacrificed for motivating factors.
Interaction provides opportunities for the student to react to the educational content. Interactions should be frequent and varied, and should have an instructional purpose.
1.3.1. Is the frequency of interaction appropriate? Different types of educational programs may require different frequencies of interaction. Games, for instance, are highly interactive, while textual didactic material may not be very interactive. Interaction is also dependent on the educational objective.
1.3.2. Is comprehension enhanced? If a question is used to increase interaction, it should not be restatement of the information presented. Instead it should ask the learner to apply the information to a new situation.
1.3.3. Is memory enhanced? Techniques that ask the learner to apply new information intermittently over time help move information from short-term memory to long term memory.
1.3.4. Is transfer enhanced? Interaction can enhance transfer through the application of the new information in a simulation or game.
1.3.5. Are there a variety of types of interactions? The most frequently used form of interaction is asking questions. Another type, which is available for Internet-based software, is communication with the instructor. Other interactions include making decisions, evaluating, constructing, drawing and charting. Whenever possible, a variety of interactions should be used.
1.3.6. Can the student leave comments? Educational software should provide the capability for users to submit comments about the program. This information should be used to continually improve the product, and/or the comments should be taken into account as the learner's work is scored. Web-based software should also offer the opportunity to leave comments for the Webmaster, often a different person from the author of the courseware.
Media are forms of expressions for content. The combination of two or more media, such as text, video, audio, or images, is defined as multimedia. Multimedia should support the educational objective and not be used just because it is available.
1.4.1. Are the media relevant and important? The selected media should be directly connected to the goals and objectives. Multimedia should be utilized when the medium conveys concepts or issues more clearly and efficiently than text.
1.4.2. Is the level of detail appropriate? Users new to a topic may need simple visual aids that make points extremely clear. As the sophistication of the learner increases the level of details and complexity of the media can increase.
1.4.3. Are media of sufficient quality to achieve the learning objective? The quality of illustrations, images, video and audio is often subject to technical and creative constraints, such as required storage space, file format and availability of professional recording and postprocessing. The media should be of a quality that is high enough to achieve the learning objective.
1.4.4. Are media labeled with unique, context-independent legends? Sometimes, media are viewed independent of their original context (such as in pop-up windows due to screen size limitations). To make the media useful as standalone entities, they should have unique, context-independent legends.
1.4.5. If media can be magnified, is increasing detail visible? Many programs provide magnification capability for images and videos. In this case, the magnification should not simply increase the pixel size, but show increasing detail.
1.4.6. Can movie-like opening and closing sequences, if used, be skipped or switched off? Often, videos or animations are used to capture the learner's interest. If they do not contribute directly to the content of the program, the learner should be able to skip them or switch them off.
1.4.7. Can audio volume and tone be adjusted or switched on/off? The user should be able to adjust audio output.
1.4.8. Are alternative output methods for selected media available? Individuals who cannot take advantage of one or more output modalities should be able to choose an alternate method of output. One example is on-screen transcription of audio output with indication of the portions that already have been listened to. In certain countries, legislation may stipulate such capabilities (such as the Americans with Disabilities Act).