Focusing Clients on Solution Features
April 29th, 2010 11:54 pm Category: Scheduling, by: Joe Litko
When using a rapid iterative development process, a client may see several solutions to his problem that are not perfect. Getting the best feedback from the client at every cycle of the development is important. If you get only the most obvious evaluations, the progress towards an acceptable product is erratic, possibly unsuccessful, and very painful.
Recent experience was a scheduling application where we did not have direct contact with the final user. Here we were working for an intermediate third party. It finally became clear that feedback was going to be limited to a comment about the feature the customer wanted to focus on. Usually that would represent some aspect of the solution that was either unacceptable or very different from current practice. Some current practices would require new resources and changes in parts of the system that could not be easily changed.
The limited feedback means that each improvement would come along one dimension at a time. In most cases there are several important elements to a good solution. Although it might sound reasonable to focus on one at a time, it can turn out that the last problem to be overcome means a substantial revision to modeling logic. That can happen even when the last problem is relatively minor. In other words the next-to-last solution is very close to the desired solution, but the model logic requires substantial revision or an entirely new concept to succeed. In a coast-to-coast driving trip, the Grand Canyon represents just a small part of the distance. Crossing it requires a different approach than driving across Kansas.
Of course, this is like a traffic accident. Regardless of who is at fault here — NO ONE WINS. Model development is a joint responsibility. It requires a lot of effort by the developer and the customer to be successful and efficient.
At the start of the process it will be difficult to get a customer to read through some defects to find more subtle problems. But it is worth the effort for both the developer and the customer. One possible approach to this is to develop a word picture or checklist of what an entirely acceptable solution looks like. In the example mentioned above the end product is a schedule of work for some crews. So the checklist could involve things like the duration of the workday, the order of activity completion, limits on available resources as implied by the schedule, and correlation between activities in different parts of the system. In the example given here it was the lack of correlation that was never identified as a model requirement until very near the final product. Creating that correlation required substantial model revision, search for appropriate parameters, and another series of regression tests.
Also essential is an appropriately large set of data for testing. Looking at a single data set at a time magnifies the problem of getting a customer to look at all aspects of a solution. Very often one data set does not exercise all parts of a model. If tests are done on live data provided each day, new deficiencies are discouraging and the prospects of success can seem bleak.
There seem to be some good principles here. Do not assume that aspects of model behavior not commented upon are acceptable. Develop a checklist or word picture of the perfect or acceptable solution. Keep that checklist growing as new information is uncovered. Get comments on all parts of the checklist/picture at every stage they are observable. Start with data that represents the variety you would see in the system over a reasonable period of time.