A Trial Final Exam

Quick Summary: Ask the customer at the end of the trial for a candid assessment.


Completing a trial successfully based on your criteria is not enough.  There are three “final exam” questions that you need to carefully answer to determine if your product or service can be the commercial success that you intend.  The questions involve the ability to provide the product in high volumes that has the required features and functionality as defined by the customer, and confirmation that it addresses a problem that prospects deem is worth solving.

After a trial is complete, there is a simple, three-question final exam that will be the most important test to determine the success of the trial.  Even if all of the features and functions worked flawlessly as envisioned and specified, the three final exam questions will be the primary determining factors to consider before moving forward or, perhaps, retrenching. 

Did you and the customer agree that the features and functions that you provided work as expected and the customer was able to use the product or service on their own with no continuous help, coaching, or intervention after the initial deployment and training?

As described in some of the articles in the Development chapter of this collection, there is a rule-of-thumb that states the “good path,” in terms of a product or service operating properly when every condition encountered is as expected, represents about 10% of the overall product development effort. This does not imply that all of the factors are perfect.  It assumes that all factors are within their normal bonds.  However, the “bad path,” which occurs when some of the conditions vary from the “standard” expectation, represents about 90% of the effort in the development of a product that can be used and supported in actual real world conditions.   The “bad path” implies that something varied from the expected conditions.  It could be user error, a related system not being available, or a component being out of tolerance.  It is not uncommon that companies are so anxious to begin a trail that they start long before the “90% Bad Path” items are properly addressed.  In those cases, when “bad things” occur during the trial, they most likely will require intervention that is beyond the capability of the customer.

To be fair, a large part of any trial is to discover situations that may occur in “real world” deployments with customers that are not obvious or were not considered in the product’s development.  Therefore, attempting to address all “90%” of the possible anomalies is not practical.  However, when discovered, some of these anomalies could have fundamental impacts on the product that could require significant re-trenching to address their occurrence in the future.  Others, perhaps, can be addressed by minor changes in labeling or user documentation.  Since multiple support personnel may be involved during the trial process, it is critical to document and discuss each unexpected condition or involvement to understand its root cause and the long term corrective actions required to address the underlying issue.

Does the customer agree that the product not only works as expected, but is also a viable addition that can be added to their existing processes and environment?

In some situations, a product or service offering may work exactly as intended and meet all of the customer’s expectations, but may still not be a viable alternative.  We all experience this situation when we walk down the aisles of virtually any retail store.  We see “new and improved” items that are more attractive than the item we currently have and would like to have the new item, but simply cannot justify replacing what we have.  Products in this category fall into the “nice to have” instead of the “must have” category.  For personal items, the barrier between “nice to have” and “must have” may be passed quickly and based on emotion instead of sound financial analysis.  Not so, in commercial settings where the new purchases may be scrutinized by others or the change impacts are substantial.  With the company’s laser focus on their new offering, it is quite common for them to miss a customer’s reluctance to change, no matter how “nice” the new product is. 

Did the trial uncover, or were certain aspects of the product identified, that cannot be easily scaled prior to the product’s commercial launch?

It is easy to fall into the trap of focusing a trial on the steady state operational aspects of a product, assuming that it was initially installed properly and the user was trained.  Many, if not all of us, have experienced the situation that once a product was working, it worked!  However, the effort to get it to initially work was traumatic.  The difficulty often stems from the incredible variations in the product’s environment.  Anyone who has attempted to load new software on a PC has probably experienced that situation.  In the past few years, software developers have made great strides in simplifying this task, but calls to help desks are still quite common.

As another example, does the product require custom configuration before it is shipped to a customer?  For a limited trial, this approach may be acceptable, but for high volume shipments, perhaps through a stocking distributor, may be totally impractical.  Finally, if the product requires connectivity to some back office service, can that service support high volume, simultaneous users?  A more general case of considering the impacts of scalability is discussed in the article, “Process: When to Start.”  In that article, the concept of “adding a zero” was discussed.  The exercise consists of considering what would happen if it was required to ship or support ten sales instead of one, or one hundred instead of ten.  After the impacts of adding one zero are understood, repeat the exercise by adding two zeros; ship one hundred instead of one or one thousand instead of ten.  This exercise will quickly identify the first breaking point long before it occurs and allow you to consider methods to avoid it.

Candidly taking the trial final exam and understanding the results can dramatically impact your next steps after a trial.  The advantage of taking the trial final exam is that you can grade it internally, long before prospects and customers take the exam with you and judge the results by themselves.


Article Number : 5.020404   

A Handy Reference Guide for Executives and Managers at All Levels.

9 Volumes 37 Chapters ~572 Articles

Browse Select Read Download



The weight of your world does not have to be on your shoulders.
The articles in this site will help to lift that weight from your shoulders.
Pick an article similar to how you pick a route on a page of an atlas.
There is no need to look at other articles, just as you ignore other pages in an atlas.
It is easy to start a business but it is hard to run. Bumps and unexpected sharp turns in the road are always present.
Others have traveled the road before you; learn from them. This site may help.