Evaluating disparities in the U.S. technology transfer ecosystem to improve bench to business translation

Defining the commercialization pipeline

The AUTM Licensing Survey solicits responses annually from around 300 institutions, including universities, hospitals and research institutions, to quantify the total technology transfer activity at these institutions. These metrics are derived from a set of core questions that AUTM deems essential for assessing transfer and licensing activity. A detailed description of each metric from the AUTM survey data is given in Supplementary Table 1. We defined the “commercialization pipeline” (Figure 1) by identifying a set of key questions asked in each AUTM survey, and extracting relevant data from the 2010 to 2014 AUTM surveys. We use this commercialization pipeline to measure and compare relative levels of technology transfer activity at different institutions, and at different steps along the pipeline. The distributions of each metric across every surveyed institution are visualized as linear and log histograms, as well as empirical cumulative distributions, in Supplementary Figure 1 and Supplementary Figure 2.

Figure 1. Commercialization pipeline.

Each step in this pipeline corresponds to a metric in the AUTM survey. We use the health of the pipeline as a proxy for the overall health of the U.S. technology transfer ecosystem.

Identification of top performing institutions

We ranked each institution from the AUTM Licensing Survey data by each step in the commercialization pipeline. Any institution ranked in the top 10 (about the top 5%) in at least one stage of the pipeline was included in the our list of top performing institutions. This resulting list of 25 institutions (approximately 12% of all surveyed institutions) was then sorted based on mean reciprocal rank (MRR):

where N = 7 is the number of stages in the pipeline and R_i is the ranking of the institution in step i of the pipeline. We chose this scoring system to identify institutions with consistently high performance across the commercialization pipeline while avoiding heavily penalizing anomalous weak performances in just a single metric.

Calculation of the Gini coefficient

Given value x_i for institution i and x_j for institution j, we calculate G, the Gini Coefficient, such that:

The Gini coefficient is a measure of statistical dispersion used to assess inequality in a population. A high Gini coefficient indicates high levels of inequality where, in this case, a few institutions contribute a substantial amount of total translational activity. Conversely, a low Gini coefficient indicates that each institutions contributes an equal share.

Statistical analysis

Variance estimates (υ) for the Gini coefficient for each step were derived via jackknife resampling (Karagiannis & Kovacevic’, 2000; Yitzhaki, 1991):

where N is the number of observations, G is the Gini coefficient when all observations are considered, and G_i is the Gini coefficient value when the ith observation is removed. The confidence intervals of the log-normal fits were computed to the 95% confidence levels using the Jacobian of the parameter estimates assuming normally distributed residuals. All statistical analysis was performed in MATLAB 2016b (Mathworks, Natick, MA, USA).

Inequality between institutions through pipeline

Our goal was to understand how much each institution contributed to each step of the commercialization pipeline and to determine any notable overall trends in U.S. technology transfer. Histograms (Supplementary Figure 1 and Supplementary Figure 2) of contributions from each institution along the commercialization pipeline reveal highly skewed distributions. The distributions of each metric are generally well approximated by a log-normal fit. Note that the x-axes is on a log scale and therefore the significant skew in the distribution is not immediately apparent. The effectiveness of a log-normal fit decreases towards the end of the commercialization pipeline (Startups and Adjusted Gross Income).

The majority of institutions contribute a small amount to overall technology transfer regardless of how activity is measured (Figure 2 and Supplementary Figure 3). Specifically, the top 20% of institutions contribute over 60% of total commercialization activity. Importantly, this trend is robust to normalization by research expenditures, which indicate that differences in research funding do not explain the gap in productivity (Supplementary Figure 3). In fact, the top 10% of institutions contribute over 40% of “startups per dollar of research expenditures” and over 70% of “adjusted gross income per dollar of research expenditures”.

Figure 2. Contribution by top 1%, top 20% and bottom 80% of institution to each step of the commercialization pipeline.

A small number of institutions contribute to the majority of commercialization activity.

Highly performing institutions

We identified the 25 top-performing institutions by sorting all top-performing institutions by the average of their reciprocal ranking at each step in the commercialization pipeline (Table 1). Most organizations that perform well do so across the entire commercialization pipeline, indicating strong and broad technology transfer abilities (e.g. University of California and University of Texas Systems; MIT; and Stanford). On the other hand, some organizations excel in only specific parts of the commercialization pipeline (e.g. University of Washington in Licenses and Options Executed; California Institute of Technology in New Patent Applications; and University of Georgia in Licenses and Options Executed), which reveals focused, less-robust technology transfer capabilities.

Table 1. The 25 top-performing institutions.

Bar plots show the mean value over the years under consideration for each institution for each step in our commercialization pipeline.

Institution	Research Expenditures ($M)		Invention Disclosures		New Patent Applications (US)		Patents Issued (US)		Licenses and Options Executed		Startups
University of California System	5364		1605		1117		359		261		66
University of Texas System	2508		772		357		173		155		24
Massachusetts Inst. of Technology (MIT)	1515		646		514		232		107		18
Stanford University	855		492		308		210		116		16
Johns Hopkins University	1540		417		400		72		132		10
University of Washington/Wash. Res. Foundation	1010		401		172		75		225		12
California Institute of Technology	426		389		565		149		51		10
University of Michigan	1257		363		152		106		115		11
UW-Madison/WARF	1113		378		129		153		63		6
University of Pennsylvania	886		386		196		78		107		15
Columbia University	737		356		230		86		82		15
University of Illinois, Chicago, Urbana	972		354		159		102		87		12
Massachusetts General Hospital	744		326		185		83		132		9
University of Florida	548		330		167		84		128		14
Cornell University	781		363		175		82		130		10
University of Utah	401		223		99		67		82		18
University of Georgia	307		163		56		34		157		3
Georgia Institute of Technology	741		364		228		79		72		10
Harvard University	812		377		213		66		78		9
University of Colorado	809		239		297		39		53		9
University System of Maryland	998		292		180		64		38		9
Duke University	845		212		127		47		118		6
University of Pittsburgh	755		264		89		49		124		6
University of South Florida	441		177		89		89		58		9
Johns Hopkins University Applied Physics Laboratory	1101		219		59		18		29		3

Dispersion analysis

We extended this analysis by calculating the Gini coefficient, a measure of statistical dispersion that is often used to quantify income inequality (Gini, 1912). In this analysis, a low Gini coefficient indicates that each institution is contributing roughly equally to U.S. technology commercialization, whereas a high Gini coefficient indicates that a few institutions are producing the majority of the commercialization output.

As shown in Figure 3 and Supplementary Figure 3, high levels of inequality exist throughout the pipeline. For context, the Gini coefficient of patents issued in the U.S. is above 60%, while the Gini coefficient of all U.S. household income is 48% (U.S. Census Bureau). We believe this indicates that the majority of U.S. research organizations have significant untapped commercialization potential, the full realization of which could lead to new technologies and, overall, improved U.S. productivity.

Figure 3. The Gini Coefficient for each stage in the commercialization pipeline, with G of 0% representing complete equality and G of 100% represents complete inequality.

Error bars represent one standard deviation of uncertainty as estimated via jackknife resampling (Karagiannis & Kovacevic’, 2000; Yitzhaki, 1991).

Improving the pipeline

Many of the top performing institutions have invested significant effort and resources in supporting entrepreneurs at each stage of the commercialization pipeline. Top performing institutions have ensured continuity in their support structure to enable the efficient and effective translation and development of both institute-owned and student-created intellectual property. Table 2 highlights active programs at MIT and Harvard, two top performing translational institutions. Our summary of these initiatives span university incubators, student organizations, university venture capital funds and business plan competitions (Table 2).

The overview of successful programs (Table 2) provides a blueprint for universities that would like to foster improved technology transfer and innovation. While some of these programs would require a significant undertaking on the part of the university, many can be achieved in a straightforward and lightweight manner via the support of student-led activities and partnership with government and private organizations. Examples of grassroot student groups that have launched many new programs exist at both MIT and Harvard. For instance, the MIT Biotech Group group has partnered with the MIT Alumni Angels of Boston to launch a life sciences-focused track to improve access to capital for early-stage startups. The Harvard Biotechnology Club runs an incubator program to develop and translate academic research. These programs represent student-led efforts that require little to no university expenditure or resources. For larger undertakings, university/corporate collaborations can provide an efficient means to achieve significant progress. A prime example of this is JLABS @ M2D2, the medical device incubator partnership between Johnson & Johnson and the University of Massachusetts Lowell (McCarthy et al., 2013).

Supporting the commercialization pipeline

One salient feature of the top-performing institutions is their broad portfolio of commercialization-focused initiatives. Individually, these projects typically target only a few steps on our commercialization pipeline (for example, business plan competitions target the latter stages of the technology transfer process). However, the best performing universities have a large number of these efforts which, in aggregate, fully span the commercialization pipeline. This observation indicates a potential strategy for improvement of those less well served technology transfer pipelines; specifically, the cultivation of commercialization focused initiatives, such as incubators, business plan competitions, innovation prizes, law clinics, and student organizations. The value of these efforts goes beyond their immediate impact. For example, although when taken at face value, a business plan competition may seem to serve only the winning team, its merit truly stems from bringing together students, entrepreneurs, investors, and the media in a constructive setting. The resources required for such projects are small, and, given the disparity in commercialization, potential societal benefits are vast.

Outsourcing technology transfer to a third-party

A clear barrier to effective commercialization of university technology is the widespread lack of access to experienced, motivated, and well-resourced technology transfer offices (TTO). Many institutions are unable to support a comprehensive TTO, hampering efforts to introduce new technology into industry. The use of consultants can help alleviate some shortcomings, but faces its own barriers to widespread adoption (AUTM Technology Transfer Practice Manual).

Alternatively, a coalition of institutions could create a third-party technology licensing organization whose charter is to serve the technology transfer needs of those institutions. Like a sports agent, this third-party organization would use its expertise to strike technology transfer deals between institutions and licensees, freeing universities to focus on their strengths. Funded directly by the institutions and, in part, by licensing revenue, this organization would have the necessary resources and freedom to hire top-tier technology transfer professionals who can effectively interface between stakeholders in industry and in academia, while negotiating on behalf of the parent institutions. These teams would work to creatively package and license technologies to maximize their utility to society, as well as to assure that the parent institutions receive a fair return on their investment.

Operating outside of the university, this organization would be free to make decisions much more quickly than traditional TTOs. Similarly, its employees would be incentivized to work in the best interest of the parent institutions by ensuring the process is both efficient and maximizes value for all stakeholders. This outsourced model of technology transfer speaks towards the latent need for more efficient, properly incentivized, and more widespread efforts to commercialize academic research and development efforts.