Figure 1. Example success factors radar chart for an IsL participant

The original IsL study uncovered six success factor areas. These six success factor areas are summarized in the companion paper entitled, “Conducting a Self-Audit of Your Institution’s Online Learning Activities.” They are:

• Compelling motivation
• Commitment and prioritization (leadership)
• Programmatic approach
• Faculty support
• Student services
• Goals and measurements

In our view, these six factors provide a system by which the likely perception of success of an online initiative can be measured. From the original study over 110 specific factors were ascertained that contribute to one or more of the six success factors. The self-audit study participants complete surveys that measure their responses on a five-point Likert scale. The responses are collected into the six success factor categories and averaged to provide a set of scores for each initiative being audited.

A participating online initiative may involve completely online delivery, hybrid delivery, or simple web-support of a classroom delivered course. It may represent a single program, a single department or other operating unit such as a division of professional studies, or it may represent an initiative that is integrated across an entire institution.

The participating initiative leadership team selects the number and type of respondents. There is a common survey for administrators and faculty. There is a shorter student survey. Initiatives are encouraged to bring up to 50 administration and faculty and 150 students to get a balanced view of progress, however most start with a small team of leaders. The process is set up for longitudinal comparison. Repeating the process with the same respondent group or larger is recommended for longitudinal comparison. In this way progress can be measured yearly or at whatever interval desired.

Each participating initiative receives a private benchmarking report that has three primary purposes:

• Facilitate discussion among the respondents on progress, strengths, weaknesses
• Give a sense of how the initiative measures up against the field of initiatives, including a defined peer group
• Facilitate discussion among the respondents on future priorities

Based on a comparison of the 110 factors, each participant initiative is given a summary table that summarizes ranking in each success factor area at one of three levels based on a statistical ranking among the field and peers:

• Improvement needed
• Keeping pace
• Best practice leader

To categorize into these three ranking areas the scores for each success factor are roughly divided into thirds with those receiving the best practice leader designation in the 66% or higher percentile, those receiving the keeping pace designation between the 33% and 66% percentile, and those receiving the improvement needed designation below 33%.

Since the individual percentile rankings in each of the 110 survey items, a recommendation is provided into possible improvement areas based on which items are most seriously lagging the field and peers. The participants also receive a table of the twenty specific items where they most significantly lagged their peers.

A typical summary spider diagram is shown here in Figure 1. This diagram gives the participant a simple visual representation of where they are relative to the field and peers in each of the six success factor areas and overall.