When I started my business evaluating PC-based accounting software in 1983, side-by-side checklists with vendor ratings was a common way to evaluate and even select manufacturing software. Millions of dollars of software have been purchased based on a vendor which has 94% of a feature list as opposed to another vendor who only shows they can do 91%! This approach is still sometimes used but does it really make sense anymore?
To be fair, in a macro sense, the checklists still have value, but only if the criteria is distinguishable, meaning that the items are not all functions that every system can do in one way or another. This is becoming more and more of an issue since, over the years, vendors have added missing features or simply copied other program’s capabilities. In other words from 20,000 feet off the ground, a lot of manufacturing software programs are starting to look alike.
Using RFPs, RFIs:
There are numerous issues with using RFIs or RFPs to gather vendor capabilities. Depending on who is doing the ratings and how well they understand their software and how they interpret the question, you can wind up with a lot of misleading information. In point of fact, vendors hate filling out RFPs of any length and often will not even bother doing it unless the prospect is a large company that represents a significant revenue opportunity.
There is a difference in asking a salesman or technical person a question about whether his/her software has a certain capability. You have a chance, but not necessarily a good chance, of getting an accurate answer. Some software company salesman are notorious for not really explaining how the software can do it, but might say something like “Yes, the software can do that.” And they say it with such sincerity you are reluctant to question them any further.
RFIs are a much better way of doing initial screening when you use high level questions that require some kind of qualitative rather than quantitative response.
Smart consultants don’t invest a lot in vendor representations anymore. They understand that business processes and qualitative information is far more informative than simple yes, no’s and scores. The really important part of evaluating capabilities is in the demonstrations where they have to show clients how the software really works both with sample data and, in round 2 of demos, with client data. Experienced consultants have gotten to the point where they place very little reliance on what the vendor says the software does. They have to show the client it works and, for really important functionality, references would be provided.
Software Buyer’s Challenge:
The hardest thing for the software buyer to do is to sit down and really focus on exactly how his/her company operates and whether there might be an opportunity to improve processes so that steps are either combined or eliminated and still achieve the necessary result. This is often not possible without outside help since company staff is simply too invested in current ways of doing things.
Evolving Evaluation Criteria:
With proper focus, and sometimes coaching, users can see clearly what software they want and what they prefer to work with day in and day out. Software selections are not about counting up scores and rankings anymore. There are too many qualitative factors such as user comfort with the software interface and functional operation. There is no way to establish this comfort level without extensive testing of the software.
Unfortunately, most smaller manufacturing companies don’t really understand this and most vendors won’t cooperate by allowing them to test the software. Therefore, purchase decisions can be hit or miss. Larger manufacturers who are making six and seven figure bets on manufacturing software cannot be as casual. They must apply the time and have the proper expertise to screen their options and arrive at the optimum solution based on business goals and how the software supports those goals. It’s definitely not about counting feature rankings anymore!
While feature ratings chart may still have some value, they are no longer the essential factor in evaluating manufacturing software options as they once were.