wilzed

The emergence of e-Learning has become an increasingly popular alternative to traditional training activities. From this, there have been many arguments for and against e-Learning. However, all can agree that it is really important to discover how e-Learning can provide credible Return on Investment (ROI) for organizations. ROI can be defined as “a comparison of the money earned (or lost) on an investment to the amount of money invested” (Wikipedia 2006, para.1). In the following is a discussion about a current practice of evaluation, which includes measuring the ROI of e-Learning programs in organisations.

As e-Learning is a growing field, there are differing perspectives on measuring its ROI. However, there is a commonly used framework to evaluate an e-Learning program that includes measuring ROI. According to Kypreos (2006), Deeny (2003) and Setaro (2001) this framework is based on Kirkpatrick’s and Phillips models. These two models combined contain a “chain of effect” principle.

Each level of the model impacts the previous and next levels of evaluation.

The Levels are as follows:

· Level 1 – Participants reactions. This determines whether the learners enjoyed their Learning experience.

· Level 2 – Determines the competencies gained.

· Level 3 – How has the Learner applied the content learnt into their workplace?

· Level 4 – Shows the Learners improved performance.

· Level 5 – Monetary value produced compared to the cost of training. This level provides a measure of quantifiable ROI which can be calculated using the below formulas.

TACTP – TACNP = PNS

This formula involves subtracting the total administrative costs of the new program (TACNP) from that of the old training program (TACTP). This gives the projected net savings (PNS) for training administration.

This measurement involves multiplying the total benefits (TB) of training in dollars by 100 and dividing that by the total training program cost (TTC). Total Benefits can be derived from performance factors such as increase in productivity and improvement in quality.

Even though these formulas provide accurate, quantitative ROI for e-Learning, they do no evaluate the effects of less tangible benefits, such as improved communication skills and higher staff morale. These benefits are currently measured qualitatively by using methods such as surveys.

Below are two Case Studies that feature organisations evaluating the benefits of e-Learning.

Department of Humans Services Case Study

In the Department of Human Services (DHS) in Salem, Oregon, employees experienced the immediate benefits of conducting online training courses. The initiative started when a group of employees in Eastern Oregon needed to travel 8 hours to attend a half-day computer training course in Salem. This meant that Learners would spend three days off the job and 16 hours driving to take a three hour course. To rectify the situation, Rod Roehnelt – the distance learning co-coordinator, downloaded a copy of Microsoft NetMeeting. The savings have been particularly beneficial, especially with having around 700 people completing the computer course in eight weeks instead of three months.

Today Roehnelt now delivers 150 courses per year using the same software in which he named NetCast. With NetCast, Learners from around the state can view presentations online at the same time and share opinions electronically. As NetCast has been so convenient, course offerings have increased by 40%. The cost of providing these courses in a classroom environment would have been more than $4 million. The benefits of online training have not only been from saving traveling costs, but have also provided greater accessibility. People who haven’t taken training for a long time are attending because of the increased convenience (Gale 2002).

Cisco Systems Case Study

Cisco Systems is an Internet networking Solutions Company based in San Jose, California. In Cisco, evaluation for e-Learning is based upon five categories.

They are:

1. Efficiency: Training that helps you work faster

2. Effectiveness: Training that helps you perform better

3. Cost Avoidance: The money saved from not having classroom training such as traveling, etc.

4. Customer Satisfaction: How the training impacts the customer

5. Business Revolution: Skills learnt that radically changes the way employees work.

Cisco finds that the most important measures to be met are the first two. Efficiency must be attained, but not at the price of effectiveness. At Cisco, evaluating e-Learning is based on both Quantitative and Qualitative measures, providing a reliable picture of the effects of an e-Learning initiative. Even with qualitative data collection methods, such as surveys, Cisco asks its employees quantitative questions. As a result calculations can be made so that evaluation can be as quantitative and qualitative as possible. For example, a Cisco recruiter after taking a course, becomes more discerning about whom she interviews. This reduces the amount of time spent on interviews from 20-10%. If she makes $80,000 a year, the return on the training would be $8,000 (10% of salary – cost of the course per user) (Gale 2002).

Compare and Contrast

Upon reviewing the above Case Studies, there are a few common factors present. They are:

1. Both Organizations have benefited from e-Learning

2. Both Organizations measure the saved cost, time and resources as a result of e-Learning.

3. Both organizations provide quantifiable proof of the benefits of e-Learning.

There are also a few differences in both of the above Case Studies. They are:

1. DHS seems to focus on measuring the cost saved from their e-Learning initiatives against the pre-existing classroom training for evaluation. ROI is only measured in accordance to the costs saved. They would most likely have used the formula TACTP – TACNP = PNS to calculate their projected savings of $4 million. Cisco however seems to use a greater variety of formulas to evaluating their e-Learning program, including the measuring of monetary benefits. It is quite probable, that they would be using the below formula.

2. Cisco also seems to have had a more developed and systematic way of measuring ROI than DHS. This may have been the result of differing factors in both organizations. This may also indicate that Cisco’s management is more concerned with reaching immediate ROI from their e-Learning initiative.

In conclusion, ROI for e-Learning in organizations can be measured using the Kirkpatrick and Phillips models. ROI can be measured with quantifiable data that can be used in formulas to calculate the dollar value of the benefits of e-Learning. This aids in providing a stronger case for e-Learning to management, who are mainly concerned with increasing profits and lowering costs. However, ROI cannot measure qualitative data from sources such as surveys and questionnaires. Qualitative data is useful in evaluating parts of an e-Learning program that are not quantifiable, such as improved interpersonal skills and higher staff morale.

References

Deeny, E. 2003, ‘Calculating the real value of e-learning’, Industrial and Commercial Training, Vol.35,No.2/3, viewed 25 April 2006, http://proquest.umi.com.ezproxy.lib.uts.edu.au/pqdweb?index=0&sid=1&srchmode=1&vinst=PROD&fmt=3&startpage=-1&clientid=20928&vname=PQD&RQT=309&did=334899571&scaling=FULL&ts=1145500900&vtype=PQD&rqt=309&TS=1145500918&clientId=20928

Gale, S.F. 2002, ‘Measuring the ROI of e-learning’, Workforce, Vol.81, No.8, viewed 25 April 2006, http://proquest.umi.com.ezproxy.lib.uts.edu.au/pqdweb?index=17&did=154476501&SrchMode=3&sid=1&Fmt=4&VInst=PROD&VType=PQD&RQT=309&VName=PQD&TS=1145501808&clientId=20928&aid=1

Kypreos, T. 2006, Building a Business Case for E-Learning, eLearn Magazine, viewed 25 April 2006, http://elearnmag.org/subpage.cfm?section=case_studies&article=13-1

Setaro, J. 2001, E-Learning 1.0: Many Happy Returns: Calculating E-Learning ROI, Learning Circuits, viewed 25 April 2006, http://www.learningcircuits.org/2001/jun2001/elearn.html

Wikipedia, 2006, Rate of Return,Wikipedia: the free encyclopedia, viewed 25 April 2006, http://en.wikipedia.org/wiki/Return_on_Investment