Time to read: 3 minutes.

The author has a bone to pick with the software industry, and is trying to find quantitative data.

I contacted market research firms and also spoke with documentation managers in leading software firms. I asked them the following questions:

1. Do you have any estimates for the cost of software documentation? I would like to know:
   1. Cost to software manufacturers and internal development groups
   2. Cost to end user companies that struggle with substandard documentation.
2. I am particularly interested in documentation for APIs, especially Java APIs with attendant Javadocs:
   1. Estimates on the number of Java APIs
   2. Total cost apportioned to writing them
   3. Cost to tech support groups who field calls because of inadequate documentation
   4. Cost to end user companies in salary for programmers' wasted time
   5. Opportunity cost to end user corporations
   6. Cost of lost sales to vendors because programming to their APIs was not cost effective.
3. I am interested in any information about the percentage of companies that are using Javadocs as opposed to other means of generating documentation.

I was surprised to learn that such information was not readily available. Software companies live and die in part according to the quality of their documentation, and yet no business oriented analysis could be found. Surely this is an indication of the immaturity of the discipline of software engineering.

One documentation manager at a well-known Fortune 500 software company said ‘we just assume that documentation is important, and try to do the best we can with the resources that are made available to us. We have never considered measuring the impact of the quality of work that we do on the success of the product, or the product's profitability.’ To be sure, few senior executives are likely to allocate resource to a project because it is the right thing to do – they want tangible justification, usually in terms of increased revenue, decreased cost, improved market share, etc. in order to grant approval. Possibly this disconnect explains why so much documentation is of low quality.

Online Documentation Survey

Unhappy with the lack of information on this subject, the author undertook an online survey of a cross-section of computer professionals. The results support the material that follows.

Thumbnail Estimates

As a technical business consultant, I once had to explain to a customer how one could use estimate the number of pianos to a fair degree of certainty in the City of New York by just looking at the yellow pages. By counting the number of piano tuners, one could guess the number of pianos from simple arithmetic and a few basic assumptions. By varying the assumptions, one could perform a senstivity analysis of the end result for each of the assumptions. In this way one could predict the number of piano tuners in the City of New York with a known degree of certainty – without ever going to New York.

In the absence of any hard data, I tried a similar exercise by using a few assumptions about software documentation costs for Fortune 500 companies. Feel free to alter the assumptions and plug in different values into the spreadsheet model – the results are most interesting regardless of the actual figures used!

From the point of view of large corporations, the training costs for badly written API documentation might be computed as follows:

30%	Effectiveness of newbie learning API
4	Weeks to learn new API to 70% efficiency
2.8	Person-weeks lost per API learning experience
$4,167	Loaded labor rate ($usd/week)
$11,667	Cost per person to learn an API
50	IT staff programming to APIs
$583,333	Learning cost per enterprise per API
6	Number of enterprise APIs utilized by average programmers
$3,500,000	Cost for all programmers to learn the enterprise APIs that they use
15%	Annual turnover
5%	Annual growth in IT staff
20%	Annual API learning growth
$116,667	Annual cost for newbies to learn APIs
10%	Percentage of APIs that become obsolete every year
$361,667	Annual cost to train for replaced APIs
500	Number of companies
$180,833,333	Total cost in Fortune 500 for API learning
$18,083,333	Savings from 10% increased productivity

The 10% increase in productivity cited in the last line above is admittedly arbitrary and not defensible. It is the author's opinion that for many software vendors the actual savings might be closer to 35%.

I spoke with a technical support manager at an enterprise software vendor who must remain nameless. He computed the cost to them of for their own poor API documentation as:

$70	loaded hourly rate for tech support personnel
1	Average number of hours for tech support staff to resolve each case
$10,500	Average daily cost for tech support staff to resolve cases
$656,250	Average quarterly cost for tech support staff to resolve cases
50%	Percentage of API issues
35%	Anticipated reduction in API case load
$114,844	Quarterly cost to tech support due to poor documentation

Enterprise software vendors frequently sell their product through system integrators. The same software vendor that generated the previous table also estimated the costs to their system integrators for poor API documentation as:

4.5	Average number of months for new consultants to become productive
30%	Average productivity for newly training consultants (compared to experienced consultants)
69.3	Average lost productivity (person-days) for new consultants to become productive
$70	Loaded labor rate for new consultant
$9,702	Lost productivity cost for new consultants (average per new consultant per quarter)
$10,000	Course fees and travel for basic, system administration and programming courses
$19,702	Total training cost per new consultant
500	Total number of consultants world-wide with all channel partners
30%	Annual churn rate for channel partner's consultants
37.5	Number of new vendor consultants per quarter retained by all partners
$738,825	Total training cost for all new consultants worldwide by all partners
35%	Reduction in training cost (decrease of lost productivity)
$3,396	Anticipated savings per new consultant
$127,339	Total quarterly anticipated savings for all partners

Directions for Further Study

Costs for developing good documentation are difficult to come by. What percentage of development cost is typical for documentation? What impact on the bottom line profitability will this have on the software vendor, channel partners and end user corporations?

Case studies would be instructive.

Share to Twitter, Hacker News, LinkedIn.