Published 2018-09-12.
Time to read: 4 minutes.
posts collection, categorized under Evaluations.
I have performed evaluations for investors since 1985, and for legal cases since 1998. I typically assess the people, the processes they follow, and the technology they work with. Although the goals of assessments performed for investors are different from the goals of the legal team that retain me as an expert, the work that I do for both types of engagements is in many ways quite similar.
Aspects Common to Investment and Ligigation Engagements
The major steps that I follow for both types of engagements are:
- Examine evidence and interview the people involved
- Perform analysis
- Write a report
- Present the report, and defend or discuss it
Strategic risks typically involve technical and business considerations. I assess the strengths and weaknesses of the technical team, the technology they use, and the processes that they follow. I am also called upon to opine on product/market fit; is this a solution that addresses a burning unmet need in a potentially lucrative market in a given time period?
Some engagements require me to develop an opinion on whether significant intellectual property was developed and applied effectively. I develop an opinion on whether the company in question is or was able to generate and profitably maintain and evolve a product or service that meets compelling unmet customer needs in various market segments.
I generally inspect the following primary sources of data for assessments:
- My notes from personally using the product or service, stress-testing it, and performing failure mode analysis
- One-on-one interviews of employees and customers (when possible)
- Examining correspondence and working documents (Jira logs, git logs, internal product requirements and customer-facing documentation)
While assessments for legal cases have many similarities with assessments for investors, the approach is usually quite different. In addition, investors are often interested in sensitivity analysis, while legal cases generally do not require that.
Investments
Investors are primarily interested in minimizing risk and maximizing reward. Investors often request assessments of technology companies to determine if a company might be a good investment, to optimize the performance of an investment, to identify risk and opportunity.
Legal Cases
Legal cases often involve assigning blame, quantifying loss or missed opportunity.
Some cases engagements require an opinion on whether significant intellectual property was developed and applied effectively. Legal cases often require an assessment of how the suitability of a company’s product or service existed at a point in time, or how it changed over time. As an expert, I am often called upon to discuss the relative strengths of the arguments of the parties in a dispute from a technology perspective.
1:1 Interviews
I try to individually interviewing everyone in the company. To the extent possible, I take each person’s photograph, gather job descriptions, resumes and their work history. The information obtained from the lowest, most recently hired person is valuable and important. The most influential people cannot help but apply spin to their story. I look for evidence to support every piece of information that is presented to me.
Open-ended questions yield much more interesting and insightful responses than specific questions. Multiple-choice questions have limited value. When interviewing someone, I recognize that all experiences are individual; I never attempt to equate my experience with those that I am interviewing. These conversations are not self-promotional opportunities.
Each interview takes 45 minutes, followed by 5 minutes of note taking. I prepare for each interview by reading the resume, job description, and any other information before meeting with them. I can process about 40 people a week this way.
If there are more people than I can interview in the time available, I interview everyone from the front-line managers on up, plus a random selection of the others. I do not allow the company to select the lower-level people because I want to know of problems. Generally I interview the top level twice: a half-hour interview at the beginning, then a 90-minute second interview at the end.
Overspecialization vs. Resilience
The more highly specialized and optimized an animal or an organization is, the more fragile it is in an evolutionary sense. Attaining product/market fit is a continuous process, not a one-time event. Markets evolve, products mature, and organizations are social constructs subject to politics and an ever-changing historical interpretation.
I try to evaluate the interdependence, resilience and resourcefulness of the individuals in an organization. Cross training compensates for highly specialized jobs; highly trained people like PhDs generally know more and more about less and less. Extremely specialized people are generally unaware of the big picture; instead, they view events through their personal microscope.
Resilience Superposition
Resilience can only be measured by testing it, and testing an organization’s resilience changes it. This is similar to the story in the field quantum mechanics, which asks if Schrodinger’s cat is alive or dead within a box.
Change Is Traumatic
I generally try to anticipate the results of two sources of change:
- A large investment or a change in ownership
- Market changes
A large investment or a change in ownership can cause a fundamental behavioral change in the existing company's leadership. Replacing the leadership or changing their behavior is usually a traumatic event for the organization.
Because markets change, an organization that lacks resilience may crumble in the face of dramatic market shifts. I try to evaluate an organization’s ability to continually anticipate and respond to changes in product/market fit. It is all mental, a social construct, stories that the people who work at a company have internalized.
Anarchists Are The Best Assessors
One of the primary tasks of an assessor is to question the boundaries of the target market and the product or service. Rarely do those boundaries remain unchanged through the lifespan of the product or service, or the company that produces them.
Anarchists perform the most useful assessments. An assessor cannot be effective if they restrict themselves to coloring within the given outlines; by this I mean respecting the status quo cannot yield an objective or useful assessment.