For the sake of this article we will assume that there is no foul play; the bugs are not added on purpose by a developer trying to sabotage the project. With that in mind we can determine that a bug is a feature that does not function the way it is intended. This means that the coding of that feature has not been completed yet and that is the only difference between a bug and a change request. A bug means the current goal for this feature has not been met yet, a change request means a new goal for a completed feature.
Another important question is who is accountable for the time spent fixing a bug. If we agree that a bug is an incomplete feature then the time to resolve a bug counts towards the total development time of a feature. If this actual time spent does not match the estimated time you could say accountability lies with the developer. But this comes down to any guarantee made. Did the developer promise a fixed price - common with freelancers, or is there a pay-per-hour agreement in place - common with larger vendors and in-house staff.
Fixed or Per-hour?
For a fixed-price project a bug results in the developer having to continue developing the feature until it is working as intended, at no charge to the client. Accountability lies with the developer so the risk of this happening is assessed by them at the start of the project and the fixed-price quote for the project is increased to mitigate this risk. There are 2 outcomes to this:
- The developer over-estimates and the client pays more than the actual work done.
- The developer under-estimates so they end up working for free.
Neither of these outcomes are particularly fair, even if the actual hours worked only differs by a couple of percent from the estimate. A perfect estimate/actual outcome is very rare, certainly on larger custom development projects.
For a pay-per-hour project a bug results in the developer having to continue development on the feature and the client being charged for the additional hours needed. This risk can still be assessed by the developer up-front to help manage the client’s expectations. The result is that the client always and only pays for actual hours worked by the developer but there is always a chance that the developer underestimates the time needed. An estimate is after all an educated guess and not a guarantee.
Client Frustration
There are typically 3 sources of frustration for clients around bugs.
- The actual hours to develop a feature, including resolving bugs, exceeds the estimated time that the developer gave. Most clients would agree that “pay only for actual hours worked” is a fair system. It is having to pay more than they initially thought would be needed that is causing a feeling of unfairness and frustration.
- Bugs being found after the developer declares the feature as completed. Inadequate testing is common among development agencies and this is often vastly underestimated in custom application development. Here the frustration is also in expectations not being met, the client is told that the feature is completed and they assume that this means it is working perfectly in all situations.
- New features breaking old features. This is closely related to the above situation of insufficient testing and the common perception that individual features mean there is no link between them. More often, a given feature is dependent on other features or is itself a dependency of other features. Meaning that if that feature is changed, it can affect other features in the application and these need to be retested accordingly. In some cases this can result in testing time being equal to or even exceeding development time and this is often underestimated or insufficiently explained to clients.
Managing Expectations
So how do we minimize client frustration and help them manage their expectations better around bugs?
When estimating projects always ensure sufficient margin of error in the estimates. A client who received a 10 hour estimate when only 8 hours were needed will be far happier than a client that received a 6 hour estimate even though the final cost is the same. Clients also frequently need educating in terms of budget management. We always recommend our clients to have 30% additional budget on top of whatever we estimate as a safety margin. This safety margin includes additional time for dealing with any bugs that might arise after development.
If a client has a fixed budget we recommend that we only plan in features covering 70% of their available budget and keep the remaining 30% as an error margin. If we complete the project without needing the 30% we can then plan in additional features and exceed expectations. If unforeseen issues arise the 30% can be used to resolve them and we meet expectations in terms of features delivered.
Testing should be at least 30% - 50% of your estimate depending on complexity of project. Front-end coding will typically need more user-testing for the different browsers and resolutions that it needs to support. Back-end coding will typically need more security and stress testing.
Our development partner EZB has 2 layers of testing. The first layer of testing is done by their internal testers as part of the development cycle. The second layer of testing is done by an external party at the end of the project. This fresh set of eyes results in far more bugs being found and resolved before the project is considered completed and presented as such to the client.
A realistic timeline is also essential to managing client expectations. Rarely do custom developed applications go from a development cycle to fully live. Phases such as closed beta and public beta are very important in these projects as they provide developers and clients a period of transition from development to fully live where additional issues can be found and addressed while at the same time managing user expectations. Most users understand that a beta product will not be bug free and generally use the platform accepting this and even help by reporting issues they encounter.
Summary
To summarize my opinion on the matter:
- A bug in a feature means that that feature has not completed development yet and additional time from the developer will be needed.
- This additional time may exceed the estimate that was given for the feature so it is essential to manage client expectations and buffer sufficiently in the estimate.
- Ensure that sufficient testing is estimated and executed for all new features and any features that they may interact with.
- Plan for a transition period from development cycles to live application where any missed issues can be found and resolved and ensure you estimate your time for this transition period to help manage client expectations.