Quality Matters Online

Quality Evaluation of Online Courses

The basis for evaluation

Pat voice recording goes here

The process for evaluation the quality of online delivery of courses at PBA is still emerging. Best practices are being developed that are grounded in principles derived from:

1. Quality Matters, an inter-institutional quality assurance organization.

Quality Matters Website

QMLOGO.jpg

2. SACS accreditation standards applied to the online environment.

sacs logo.jpg

3. PBA's Quality Enhancement Plan (QEP) applied to enhance student learning in the online environment.

QEP.jpg

4. Office of Online Learning standards emerging from national standards of best practice, the research literature and instructional design priniciples

5. Best use of media informed by current and emerging technologies.

Quality Matters suggests:

studentPointofView.jpg

Definition of Design and Delivery

Design and delivery participants in course development

May include instructional designers, faculty content experts, technologists, formative and summative evaluators.

Quality Matters defines the difference between design and delivery:

designvdelivery1.jpg

For example:

designvdelivery2.jpg

 

The Design and Delivery Process

ISD.jpg

The 5 Step Design Process in the Office of Online Learning

1. Analysis/Assessment

Definition: define the goals and objectives based upon existing parameters

This stage is comprised of a review of

Interviews with the content expert;

Inventory of existing content;

Master Syllabus Learning Outcomes;

First Day Syllabus;

Existing course documents and learning objects;

Course texts and publisher documents and learning objects.

Audience analysis and relational objectives defined;

Expectations of the appropriate dean or chair.

Design

Definition: planning the course layout within the guidelines of the Office of Online Learning. .

This stage is comprised of:

The instructional designer and the content expert creating a development plan;

Set-up of the architecture to organize lesson delivery;

and developing course assets;

Creating an instance of the course in eCollege;

Customization of the Office of Online Learning eCollege template for the specific course;

Alignment of Master Syllabus Learning outcomes and First Day Syllabus assessments;

Preparation of a prototype.

Development

Definition: creation of learning objects and placement of assets into the course.

This stage is comprised of:

SoftChalk assignment documents prepared and deployed into eCollege;

Lecture materials prepared and deployed i.e. SoftChalk, PowerPoints, audio files, video files;

Text documents prepared and deployed;

Media elements produced and deployed;

Unit introductions written and deployed;

Discussion questions finalized and deployed;

Integration of faith finalized and deployed;

Artwork and third party media selected and deployed;

Formative evaluations by the content expert, the Office of Online Learning, and department deans or chairs.

Implementation

Definition: making the course available to students.

This stage is comprised of:

Technical assistance with accessing course materials;

Making adjustments to course materials and learning objects;

Requesting feedback.

Evaluation

Definition: post-delivery summative evaluations.

This stage is comprised of:

Feedback from student evaluations;

Delivering professor evaluations;

School or department evaluations;

SACS compliance and QEP administrators' evaluations;

Adjustments to the curriculum.

Quality Matters' Eight Standards and Alignment

Eight Standards Rubric

rubric.jpg

What is alignment according to Quality Matters

What is alignment.jpg

Evaluation Rubric

The Office of Online Learning standards for course design are under development. In one sense the process will always be emerging because the standards will be informed by new professional research and summative evaluations. At the present time the Office of Online Learning will utilize an adaptation of the the rubric shown below as the basis for online course evaluation.

Design Process.jpg

Note: The rubric below is from the "Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values" available on the Quality Matters website.


Rubric

Course Overview and Introduction

1.1 Instructions make clear how to get started and where to find various course components. (3)

1.2 A statement introduces the student to the purpose of the course and to its components; in the case of a hybrid course, the statement clarifies the relationship between the face-to-face and online components. (3)

1.3 Etiquette expectations (sometimes called "netiquette" for online discussions, email, and other forms of communication are stated clearly.

1.4 The self-introduction by the instructor is appropriate and available online.

1.5 Students are asked to introduce themselves to the class.

1.6 Minimum student preparation, and, if applicable, prerequisite knowledge in the discipline are clearly stated.

1.7 Minimum technical skills expected of the student are clearly stated.

Learning Objectives

2.1 The course learning objectives describe outcomes that are measurable. 2.2 The module/unit learning objectives describe outcomes that are measurable and consistent with the course-level objectives. 2.3 All learning objectives are stated clearly and written from the students' perspective. 2.4 Instructions to students on how to meet the learning objectives are adequate and stated clearly. 2.5 The learning objectives are appropriately designed for the level of the course.

Assessment and Measurement

3.1 The types of assessments selected measure the stated learning objectives and are consistent with course activities and resources. 3.2 The course grading policy is stated clearly. 3.3 Specific and descriptive criteria are provided for the evaluation of students' work and participation. 3.4 The assessment instruments selected are sequenced, varied, and appropriate to the content being assessed. 3.5 "Self-check" or practice assignments are provided, with timely feedback to students.

Resources and Materials

4.1 The instructional materials contribute to the achievement of the stated course and module/unit learning objectives. 4.2 The relationship between the instructional materials and the learning activities is clearly explained to the student. 4.3 The instructional materials have sufficient breadth, depth, and currency for the student to learn the subject. 4.4. All resources and materials used in the course are appropriately cited.

Learner Engagement

5.1 The learning activities promote the achievement of the stated learning objectives.

5.2 Learning activities foster instructor-student, content-student, and if appropriate to the course, student-student interaction.

5.3 Clear standards are set for instructor responsiveness and availability (turn-around time for email, grade posting, etc.)

5.4 The requirements for student interaction are clearly articulated.

Course Technology

6.1 The tools and media support the learning objectives, and are appropriately chosen to deliver the content of the course.

6.2 The tools and media support student engagement and guide the student to become an active learner.

6.3 Navigation throughout the online components of the course is logical, consistent, and efficient.

6.4 Students have ready access to the technologies required in the course.

6.5 The course components are compatible with current standards for delivery modes.

6.6 Instructions on how to access resources at a distance are sufficient and easy to understand. 6.7 The course design takes full advantage of available tools and media.

Learner Support

7.1 The course instructions articulate or link to clear description of the technical support offered.

7.2 Course instructions articulate or link to an explanation of how the institution's academic support system can assist the student in effectively using the resources provided.

7.3 Course instructions articulate or link to an explanation of how the institution's student support services can help students reach their educational goals.

7.4 Course instructions answer basic questions related to research, writing, technology, etc., or link to tutorials or other resources that provide the information.

Accessibility

8.1 The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses.

8.2 Course pages and course materials provide equivalent alternatives to auditory and visual content.

8.3 Course pages have links that are self-describing and meaningful.

8.4 The course ensures screen readability.

More on the ADDIE Model of Instructional Design