A common issue that arises late in the sales cycle when trying to close a deal to deliver sales training is “How do you measure the impact of your course?” This question is loaded with danger…and opportunity for the forward-thinking training organization. The stack of research and literature on measuring the impact of training is high and deep, but the usual approaches have significant limitations. These limits have to be addressed to have a rigorous and meaningful assessment of the effectiveness and impact of sales training.

Level 1

Post training evaluations do a good job of measuring participants reaction to the training event and often include a component asking respondents to predict applicability to their job performance. This self-reported data correlates at very low level with future business impact (r < .2) and a learners ability to recall specific information from the training (Alliger, Tannenbaum, Bennett, Traver & Shotland, 1997). Predictions about future performance and ability to retrieve information and apply it are usually overconfident (Zechmeister & Shaughnessy, 1980; Thalheimer, 2007). While post-training evaluations are a good starting point for measuring sales training effectiveness, it does not complete the job.

Level 2

Post-training tests that measure a learner’s ability to retrieve information from the course are widely used in situations that require compliance (such as in the pharmaceutical industry) or technical proficiency (for certifications). Since recall is necessary to enable on the job application, post training tests are clearly a useful component when measuring the potential for impactof a sales training program, but the ability to answer questions about training materials does not necessarily translate into higher sales performance. Tests often have questions that are biased - too easy due to clues contained within the question or so difficult that learners are discouraged from getting the right answer. Online tests are often treated as “open book” exams, which in itself is not a bad thing (at least people learn how to find the correct answers!) but it does not mean that the learner can apply the knowledge, skills or information in a real sales situation.

Level 3

Measuring training transfer (Level 3) is best accomplished by an objective third party. Ideally, sales behavior in the field is observed and assessed by someone who is familiar with pre-training performance and is trained to be able to spot the differences after training. In reality, sales managers are best positioned to do this, but with a myriad of responsibilities and multiple salespeople to manage, this is rarely accomplished with a high degree of precision. Many training organizations resort to self-reported data, which predictability results in over-estimated transfer and application.

Level 4

Evaluating business impact is logistically difficult, for a number of reasons:

1. Isolating the Impact of Training

There are a variety of variables that must be controlled in order to isolate the training impact. Some are controllable - compensation and sales incentives, stabilizing territories, and keeping sales teams intact for the period of measurement. Others are uncontrollable, such as market conditions and competitive landscape.

2. Forming Control Groups

One client did all of the right things in designing a strong Level 4 evaluation, including setting up a control group of non-participants with the same tenure, experience, historical performance and baseline training. After the training, the pilot group could not help themselves but to share what they learned with the control group as they saw an immediate positive impact on their own sales results (which they attributed to the training). Thus, the control group was contaminated and the data being collected for the Level 4 evaluation was no longer valid.

3. Maintaining Vigilant Measurement

Sales organizations cannot “freeze” in the hopes of a valid scientific pursuit of impact data. The best sales teams are always tweaking, adjusting, compromising, and trying new things in order to grow their book of business. It is unreasonable to expect professionals tasked with driving revenue to ignore the very individual needs and wants of their customers to maintain precise measures of the impact of a training program. In our experience, the heavy lifting associated with measuring business results is often abandoned before the prescribed measurement period elapses.

Level 5

The holy grail of trainers is to unequivocally prove that the financial investment in training is worthwhile. The usual method is to take the fully loaded costs of training and compare those costs to the financial impact. The trick is to make sure that all costs associated with training are accounted for in the analysis. The costs that are almost always captured (due to the ease of identification) are course development costs, travel costs, logistics (shipping, printing, etc.) andthe wages and salaries of all of the staff involved in developing, delivering and attending the program (Lilly, 2001).

However, costs such as lost productivity (sales, customer response time, customer satisfaction) and administration (marketing, testing, registration, documentation, postage, conference calls) are more difficult to pinpoint and are sometimes left out of the analysis. Another commonly neglected area is the reduction in performancethat is commonly referred to as the learning curve. Performers often struggle to adopt new skills and this produces a temporary negative impact on results in the period immediately following a training event (Buelow, 2004).

Conclusion - There is Light at the End of the Tunnel

After all of these pitfalls, you may be wondering what we should do as trainers to assure our customers that it is indeed worthwhile to invest in training? The answer lies within a comprehensive strategy that encompasses all of the Levels defined here, and making incremental improvements to all levels. In upcoming posts, we will examine and recommend ways to enhance the precision for each level.

For more information on SNI’s training and measurement practices, please visit http://www.shapironegotiations.com for a white paper - The Direct Path to Training ROI.

As the reverse auction increases in popularity as a way for buyers to drive vendor costs down, more suppliers are choosing not to participate in them. Suppliers are resisting the pressure to reduce profitability and commoditize their products and services. However, a strategy to avoid reverse auctions is not the best or only way to prevent erosion of profits.

As in any “negotiation” the best way to get what you want is to help the other side get what they want. And today’s buyers are certainly interested in more than simply cutting costs (although in this economy, it is a primary interest). Suppliers need to find a way to provide customers with the benefits of reverse auctions without succumbing to the pitfalls of the strategy.

The figure below shows the existing model for a typical reverse auction.

Arthur D Little, a management consulting company based in the UK, developed a process that meets the needs of buyers, while protecting the interests of suppliers. In this process, the buyer takes the additional step of defining key criteria that increase the value of each proposal to the buyer. Buyers would indicate which value-added products or services would be most important to meeting their needs.

Suppliers would then assess the willingness of the buyer to pay extra for various value-adds. This gives the supplier a chance to adjust pricing based on real criteria, instead of having to guess at these options and hope to make the “final” round and then negotiate in a needs-based manner.

Buyers would then accept the lowest adjusted price bid, and save the time and trouble of the post-auction negotiation. The figure below reflects this basic process:

The buyer benefits from an increased array of goods and services that will be capable of auction; a decreasing the number of bidders who fail to deliver value; and an award based on total value. Suppliers have price transparency, but the real value of their bid is concealed in the details and it minimizes the “final round” of negotiations after the auction by reducing buyer discretion to change specs or to demand discounts. In short, it provides for negotiation in a controlled and structured manner.

Source: Arthur D Little is the world’s oldest management consultancy, founded in 1886. This process was originally defined in Dr. Daniel Deneffe’s research paper titled “How to Design Reverse e-Auctions to Realize Their Full Potential.”

A reverse auction (also called procurement auction, e-auction, sourcing event, e-sourcing or eRA) is a tool used in business-to-business procurement. It is a type of auction in which the role of the buyer and seller are reversed, with the primary objective to drive purchase prices downward. In an ordinary auction (also known as a forward auction), buyers compete to obtain a good or service. In a reverse auction, sellers compete to obtain business.

According to recent research, the number of companies purchasing goods via online auctions is increasing by 5% annually. Large organizations (>$100M U.S. in goods purchased) use the reverse auction more often than smaller companies.

The risks for suppliers are clear - reduced pricing on auctions won, future price reductions in face-to-face negotiations (due to the bad precedent), inability to differentiate on value, a removal of the direct relationship with buyers and constraints on the terms and conditions of the contracts associated with reverse auctions.

There are some benefits to a supplier too. Reverse auctions give you the time to effectively prepare a strong bid. Suppliers can also develop a new channel for excess inventory and reduce the costs of acquiring new customers. Transaction costs are reduced in a reverse auction and suppliers can improve the standards for specifications (creating a more level competitive playing field).

Here are some key strategies to try when participating in a reverse auction:

1. Use Precedents to Your Advantage

If a customer is thinking about using the reverse auction for the first time, you should share the following common complaints about the strategy:

• It is slower and ultimately more expensive than a face to face transaction. Time spent drawing up the specs, qualifying applicants, identifying bidders, and choosing the winner all contribute to a longer, more drawn out process.
• Reverse auctions drive out competition. Some suppliers cannot compete on price alone, which, over time, erodes the competitive advantage by consolidating the supplier base. Eventually, bargaining power will shift back to the supplier.
• Sometimes the winner cannot meet the demands of the buyer at that low price. As a result, the buyer has to start the process all over again.
• Reverse auctions carry the risk of exposing sensitive information that can be shared among bidders and buyers.

2. Establish a Walkaway

By setting a reserve price (a price at which you will not accept a penny less) and establishing strong, non-negotiable evaluation criteria, suppliers can play the same game as buyers. One example is to set an “exploding deadline.” Set a limit to the amount of time your bid is valid. Do not let the auction end then wait for a “final” decision. Reserve the right to close your bid if a contract is not in place within a reasonable amount of time.

3. Use Pricing Software

Prevent unprofitable bids by investing in pricing software to quickly analyze profitability, especially if you participate in multi-offer reverse auctions where you can see who is winning the business during the auction. Vendavo, MapInfo, Visstar and Zilliant all provide different types of software that can protect your bids from becoming losses.

In the next entry, I will outline some alternatives to reverse auctions that can help both supplier and buyer get the most of what they want from their deal.

Many training initiatives are reactive in nature. An influential executive notices a skill gaps or deficiency and prescribes training as the remedy without careful consideration for the long-term benefits. In fact, training alone will rarely, if ever, provide for lasting and productive change in the way people do their jobs. People are able to change only if they have an understanding of the reasons for that change and can recognize a personal benefit if they do change their behavior (Langdon, Whiteside and McKenna, 1999).

Organizations must recognize that training alone is rarely the solution. Training typically impacts individual performance whereas in a business setting, results are typically produced by a team of people acting in concert to achieve an objective. These people may have different incentives and barriers to their individual performance, such as different compensation plans, tools and systems to use in doing their jobs and the types of rewards and recognition that are available. These barriers and incentives must be addressed in order to determine if training is part of the right solution to drive the desired behaviors and results.

To successfully design and implement a training program that delivers positive results for your organization, we recommend the following steps:

1. Assess Your Business Needs

The first step is to determine precisely what gaps exist between desired outcomes and the current state. Determine this gap by:

a) Assessing the existing skill set and level of your employees.

b) Analyzing the organizational practices that drive or impede new ways of doing things.

c) Reviewing existing training curriculum to measure the extent to which it delivers necessary skills, knowledge and attitudes.

Use this information to establish a baseline of today’s performance and to identify the proper solutions.

2. Align Your Solution with Corporate Strategy

The solutions you choose must be aligned with the long-term strategic direction of your organization. In the case of a training solution, we recommend that your company:

a) Align training expenditures in direct proportion to corporate goals.

b) Centralize accountability for producing high-impact training.

c) Solicit input from all levels of the organization on alignment and proper solutions.

For a complete version of the white paper “The Direct Path to Training ROI“, please call Shapiro Negotiations Institute at 410 662 4764.

According to the latest annual report by Training Magazine, U.S. organizations spent $58.5 billion for training (including payroll and training budgets) with $16.3 billion spent on external learning products and services. While growth in the training industry tailed off (6% growth versus 7% in 2006), staffing and training budgets increased by 4.8%. Breaking it down further, the average spend per learner in 2007 was roughly $1,202 which is about the same as the $1,276 spent in 2006. These are significant numbers in troubled economic times since companies are making tough budgeting decisions for 2008 as you read this and all signs point to an extended period of recession.

Top Priorities

1. Increase the Effectiveness of Training

Forty-four (44%) percent of companies responding to the annual Training survey listed increasing the effectiveness of training as a top priority in 2007, and I predict that figure will increase when the 2008 survey results are published. Companies are focusing more attention on implementing programs that maximize learning transfer to the job and that have a direct impact on the bottom line. Gone are the days of 3-day “flavor of the month” in-house workshops with extended role playing. Today’s training provider has to deliver the goods in a short, memorable program and follow through with easy-to-use job aids.

2. Reduce Training Costs

“Increased effectiveness” is often code for “better training for less money.” This is understandable when training directors and business-line leaders are making decisions on headcount, technology and other key business drivers and trying to justify training expenditures at the same time. Twenty-nine (29%) percent of respondents to the Training survey reported that vigilance over training expense is their #1 priority in 2007. This will undoubtedly increase as the economy continues to stumble.

3. Measure Training Impact

At SNI, we have experienced a significant increase in the number of clients expressing a desire to carefully analyze the effectiveness of our training programs. More and more companies are looking for partners who are willing and able to assess learning transfer and its impact on the bottom line. While many companies continue to struggle with the measurement process, most are willing to spend the time and money needed to more closely assess bottom line impact. According to Bersin & Associates, most companies will choose one or two programs that are closely aligned with results for which there is readily available data for the measurement effort.

Conclusion

This blog will explore the emerging trends and outcomes of the “new training economy” to identify the best survival tactics for today’s training professional. The first step in forming a strategy for success is knowing precisely what the training buyers and participants need from us. The good news is that while increased effectiveness, reduced costs and accurate impact measurement all seem to be incongruent (”Do more with less and prove it”)it creates the perfect storm of opportunity for us. Plato (apparently?) said “Necessity (which) is the mother of invention.” Perhaps this is our opportunity to take all of the tools, technology and training we have and use it to meet these emerging needs in our customers.