So far we have found out that perceived success in Internet-supported learning appears to be correlated with strong motivations and balanced leadership. We next turn our attention to how institutions are measuring themselves with respect to their Internet-supported learning initiatives.

Respondents were asked to provide in free text format the top five measures of success. The answers were interpreted and classified into about 20 categories. The results are shown in Figure 9. Fully 50% of the measures were related to student outcomes or satisfaction, consistent with the strong motivators to move online to provide better service to students. 21% of the measures related to growing enrollments, consistent with the mission drivers of providing increased access and increased enrollment, revenues, and market reach. Faculty satisfaction came in at a distant but strong 10%.

A very interesting result was that measures that involved the shear volume of courses or sections online or using CMS technology were only mentioned in 4% of the responses.

A few of the institutions had put in place relatively sophisticated measures of Return on Investment (ROI), but this was a small minority. The majority favored direct input from students to gauge progress.

Consistent with the focus on student outcomes, several of the institution profiles (see Section VI) point prominently to the equal or favorable student outcomes, as measured by retention and quality ascertained through student interviews, in comparison to face-to-face courses. How much progress is being made in terms of quality and the “no significant difference” phenomena (Twigg, 2001)? The recent Sloan “Entering” Study indicates that 40.7% of schools offering online courses agree that students are “at least as satisfied” with the online course in comparison to the face-to-face course (Allen & Seaman, 2004). Only 3.1% disagreed. The remainder were neutral, which we assume is interpreted as “not sure”.

In this study of institutions that considered themselves successful at e-Learning we explicitly asked how it was believed most faculty and students involved in these efforts would assess certain quality-related issues. The results are shown in Figure 10.

Our interpretation of the results is that a majority of the institutions (59%) believed that their courses or programs were of higher quality due to the e-Learning initiative. A slightly smaller majority (54%) felt that they had revolutionized the teaching process. A larger majority (68%) felt that e-Learning providing students with two distinct advantages over the classroom experience: The ability to cover more material and the ability to be more participative in the learning process.

The “Thwarted” Study portrayed e-Learning as a “Boom Gone Bust” (Zemsky & Massy, 2004b). So, a natural question to ask of these self-admitted successful institutions was if they were more or less successful than they expected to be. Surprisingly, they were more successful than they expected, indicating that they would probably not characterize their experience as a boom gone bust (see Figure 11).

As previously mentioned, the Sloan “Entering” Report predicts growth in online enrollments of 24.8% in the coming year (Allen & Seaman, 2004). Based on the expectations of our participants, the majority of which selected enrollment growth in the range of 15% to 25% (for those that chose enrollment as a key metric), the Sloan prediction appears to be on the high end of expectations. If we had to venture a guess based on our limited sample we would guess closer to the last year’s growth reported by Sloan of approximately 20%.

Should there be some standard metrics of success that all institutions of a given type compare themselves against? Our assumption going into this study was that institutions and the public would be best served by institutions developing and communicating their own mission-specific metrics. However, at the request of some participants we included a question in the final survey that asked respondents to give their opinion on the usefulness of each of a list of 18 metrics in comparing their progress to like institutions. The results indicated a set of metrics that were perceived as absolutely essential or very useful and a few that were considered by the majority to be poor comparison metrics. These are summarized in Table 2.

Table 2. Most and Least Favored Comparison Metrics

Consistent with the previous findings, the measures considered most useful are those concerning learning outcomes and quality as perceived by the students.

Our summary interpretation regarding measures of success and expectations is as follows: