Typical brief
The client creates a brief that contains the requirements of the project. This is shared with a number of development agencies who provide an estimate as best they can based on their understanding of that brief. Each of these vendors has their own experiences, ways to implement the project and come up with their own unique solution for the project and its challenges.
Regardless of which vendor the client chooses, by the end of the project they are presented with a final invoice that brings the total project cost significantly above the initial estimate. In most cases the expected timeline also does not pan out causing issues with launch dates.
The client ends up frustrated by their expectations being completely missed and demands reasons. And these are always the same; Changes were made during development, the requirements were not made clear before hand, what we estimated did not include certain activities such as testing or management, etc. Overall a bad experience with each party quick to point fingers at the other.
I came across a quote recently that is especially applicable to this situation.
The single biggest problem in communication is the illusion that it has taken place.
- George Bernard Shaw
And this is exactly what happens when clients brief vendors. The client assumes that their brief is sufficiently detailed. After all, the brief was created by the project owner or project manager who have a keen understanding of the requirements and planning. But they typically do not have an understanding of the technical implementation and challenges.
The vendor receives a brief and assumes that this is the full and detailed scope of the project. They compare it to past similar projects and make some educated guesses to come up with a preliminary plan that their estimate is based on. Most vendors look at the brief as an instruction manual. To them the brief is clear, the client wants to achieve a certain goal, they know how to write code that will help them achieve this. Not much further discussion should be needed.
And this seems great on the surface. Client has briefed the vendor, it contains all the requirements that the users need. The vendor understands it and sets off to program the application. But so many assumptions are being made here that the perceived mutual understanding is entirely an illusion.
There are 2 concepts I repeat frequently in my writing and both of them apply to this situation.
- The 90/10 rule
90% of a custom project is neither understood nor seen by most clients. They only see the visible 10%. In the case of a brief, the non-technical project manager has written a brief that sits mostly in that 10%. Very little of the brief will be in the 90%. But let’s be optimistic and say the project manager briefed the full visible 10% and another 10% in the unseen 90%. That means that the full detailed brief provided by the project manager to the vendor is still only 20% of the total project. And this is a best-case scenario. - The house analogy
This analogy draws similarities between a custom application project and a building house. What is happening in our above example, is that the project manager is going directly to the construction builders and asking them to start laying the foundations for a new building based on the interior designs. Put into this context you can see why the outcome rarely meets the initial expectations.
Solution Architect
While some senior developers might give feedback on the requirements, it is never going to be enough to create the illusive “100%” brief. This is because developers have a deep understanding of the technical aspects of a project but they usually struggle to view things from a business perspective. Having both the technical and business understanding is essential to creating a complete project brief.
For briefs to cover more than 20% of the project, you need a Solution Architect. This is a professional that can view the project from the client’s perspective as well as the developer’s and, most importantly, bridge that gap. They can improve the 20% that the client has and fill in the 80% of the project brief that is missing.
I believe there are 2 reasons most clients don’t use a solution architect.
- They don’t know they exist. While the role of solution architect is not a new one, they are still not commonly known outside of technical circles. Most would see them as a technical project manager or experienced developer. This would mean they are part of the development agency’s team making them “someone to involve later”.
- Most solution architects are only found at larger development agencies. This poses 2 problems. First, these agencies are not cheap, so most smaller companies won’t even consider it. Secondly, larger clients have strict procurement guidelines, they would not be able to engage a development vendor without first doing due diligence by briefing and getting estimates from multiple vendors.
The first reason we are addressing by educating potential clients through articles such as this and podcasts with our development partner EZB. The second is exactly the reason why Binary Thinktank exists. We offer independent solution architecture consultancy without any commitment to development services. Past clients have also asked me about hiring a solution architect in-house. This is only cost-effective if the client is developing at least 8 projects a year however.
Briefings that work
The issue we need to address is to ensure the project plan covers most of the project before any commitment is made to the estimate and schedule. Let's assume that you need to rely entirely on development agencies and their in-house solution architect and still have to meet procurement requirements while achieving the "all information before estimate" goal.
Step 1
Create the non-technical brief. Consider the following when creating the brief:
- Research as much as possible first. For internal applications confirm with each department what their requirements and guidelines are. For example, perhaps the IT department will require the application to run on your local network and not on the cloud. This is crucial information for a precise estimate. For a public application, research the market first. Make sure the application is solving a real, existing problem and that there is a need for it.
- Look at competitors, similar products and off-the-shelf products or SAAS that offer a similar service that you need. Can these be used instead of developing a custom solution? If no, then list out the specific reasons why not per product. It would also help to include what you like about each existing product in the brief.
- Focus on creating your requirements by looking at the goals of real users. Not simply listing out functionality. For this you need to talk to each type of user on the platform to get their feeedback. These requirements are called “User Stories” and I have written about this in more detail here: Requirements Gathering
- If there are specific deadlines or a maximum budget to meet, be sure to include this information. For budgets work backwards to your development budget. Start with your total project budget, set aside 30% for risk mitigation and divide the remaining 70% between development and marketing or training.
Invite a number of vendors to provide a ballpark based on your brief. Make sure to specify that you are expecting a ballpark only, not a detailed estimate as it would be unlikely to be accurate at this point. Let the vendor know that you do not work fixed price (because of the reasons mentioned here: There are no winners in Fixed Price) and that you expect their ballpark to include every relevant cost to the project such as development, testing and management - there should be no hidden costs. Enquire if they include risk margins in their estimates.
Ask each vendor to provide their profile, their day rate(s), their contract, an overview of their workflow and details on their testing and quality assurance processes. This information will help you to get a better understanding of the vendor and their capabilities.
Step 2
Evaluate each vendor and their ballpark. For the outliers with a ballpark price significantly above or below the average, try and find out why. Walk them through the brief and their reasoning for the ballpark. Evaluate the processes they have in place. Do they have a well-thought out approach to projects including research, documentation, development, testing and QA? Do they have redundancies and adequate backup facilities in place? Do they work with versioning services such as GIT? Most importantly, do they have a solution architect or similar role in their team? Agencies without a solution architect will typically start developing as soon as possible and suggest to iterate while developing. An agency with a solution architect will suggest (or even require) extensive planning, research and documentation before any code is created.
Shortlist 2-3 vendors at most and get a quote from them for the solution architecture. You know have 2 options depending on your budget and the vendor’s terms. If you have sufficient budget and the vendors offer the option of committing only to solution architecture and not development, you could consider buying the solution architecture documentation from each vendor. This would give you the maximum level of insight into the capabilities and proposal from each vendor before you commit to one of them. If this is not an option, then you will need to select a vendor based on your evaluation of their profile, the ballpark and their solution architecture estimate.
Either way, you have met the procurement requirements and done your due diligence but it is essential that you do not commit any internal budget based on the ballpark. Only the solution architecture estimate should be approved and committed to at this point.
For the next steps we assume that a vendor was selected prior to the solution architecture.
Step 3
Work with the vendor on the solution architecture documentation. This process will typically take 1-4 weeks, will include research, interviews, discussions and perhaps some prototyping to gain a better understanding of complex features or third party dependencies. The solution architect should manage most of this process independently and communicate to you when something needs reviewing and what information is needed from specific stakeholders.
A completed solution architecture document is typically 30-60 pages, including flowcharts and sometimes wireframes. More about that can be found here: Solution Architecture II
Most importantly, the solution architecture should contain your detailed estimate and proposed schedule. Typically the estimates should be within a margin of error of 10-15%, but this may vary between architects depending on their risk model.
Step 4
Once you have the detailed estimate you can commit to the project. Before assigning budget mark the estimate up by 30% for an additional risk margin. This is the total amount that you need to get budget approval for. The PO can be submitted and the project development can start.