When Do Research Consortia Work Well and Why? Evidence from Japanese Panel Data

We examine the impact of a large number of Japanese government-sponsored research consortia on the research productivity of participating firms by measuring their patenting in the targeted technologies before, during, and after participation. Consistent with the theoretical predictions of Katz (1986) and others, we find consortium outcomes are positively associated with the level of potential R&D spillovers within the consortium and (weakly) negatively associated with the degree of product market competition among consortium members. Furthermore, our evidence suggests that consortia are most effective when they focus on basic research.


When Do Research Consortia Work Well and Why?
Evidence from Japanese Panel Data By LEE G. BRANSTETrER AND MARIKO SAKAKIBARA* We examine the impact of a large number of Japanese government-sponsored research consortia on the research productivity of participating firms by measuring their patenting in the targeted technologies before, during, and after participation.Consistent with the predictions of the theoretical literature on research consortia, we find consortium outcomes are positively associated with the level of potential R&D spillovers within the consortium and (weakly) negatively associated with the degree of product market competition among consortium members.Furthermore, our evidence suggests that consortia are most effective when they focus on basic research.(JEL 032, 031, L52) If technological innovation is the most important force driving economic growth in the long run, then public policies designed to promote and encourage technological innovation take on substantial importance.This paper investigates the impact of Japan's decades-old experiment with one such policy instrument: publicly organized and supported research consortia.There is a longstanding debate concerning the role these consortia have played in Japan's technological development.This debate has implications beyond Japan's borders, because Japanese research consortia have been and continue to be emulated by nations in Europe, North America, and elsewhere in Asia.
Following A. Michael Spence (1984), a large theoretical literature has developed over the last 15 years that has analyzed the possible benefits of research consortia as tools by which R&D externalities could be internalized.Important contributions include Michael L. Katz (1986), Claude d'Aspremont and Alexis Jacquemin (1988), Morton I. Kamien et al. (1992), Kotaro Suzumura (1992), Dermot Leahy and J. Peter Neary (1997), Yannis Katsoulacos and David Ulph (1998), and Kamien and Israel Zang (2000), among others.Much of this theoretical literature has sought to identify the conditions under which consortia are likely to lead to welfare-improving outcomes.Up to this point, however, little has been done to confront the empirical predictions or implications of this literature with data in a systematic way.'In this paper, we seek to address this gap between theory and empirical analysis.While we will not directly test the propositions of these theoretical models, we will assess the empirical importance of consortium attributes emphasized by these models in explaining consortium performance.
We examine the impact of a large number of Japanese government-sponsored research consortia on the research productivity of participating firms by measuring their patenting in the targeted technologies before, during, and after participation.Among the empirical challenges we confront, the sample-selection problem-only firms with strong R&D capabilities participate in consortia-is probably the single greatest econometric problem facing any analysis seeking to measure the impact of government support on commercial R&D activity (Tor J. Klette et al., 2000).Our rich data set enables us to examine the impact of consortia at several different levels of aggregation, so that we can address issues of sample selection, causality, and the unmeasured heterogeneity of participating firms.Consistent with the predictions of much of the theoretical literature, we find that consortium outcomes are positively associated with the level of potential R&D spillovers within the consortium and generally negatively associated with the degree of product market competition among consortium members, though this latter relationship is less statistically robust.Furthermore, our evidence suggests that these consortia are most effective when they focus on basic research.
After we summarize the empirical implications of the theoretical literature in Section I, we provide in Section II background information on government-sponsored research consortia in Japan.Section III analyzes the overall impact of research consortia over time.Section IV examines the relationship between consortium characteristics and consortium outcomes.Section V concludes the paper.

I. Theoretical Predictions and Empirical Propositions
For concreteness, we review the empirical implications of one important early contribution to the theoretical literature-the work of Katz (1986).However, these implications also emerge from the other key papers in this theoretical literature.In Katz's model, the welfare impact of research consortia depends on the values of two key attributes.The first of these is the level of R&D spillovers within a consortium.In this model, the "effective" R&D of a firm is the sum of its own R&D expenditure and the R&D spillovers it receives through participation in a research consortium.2Through these R&D spillovers, firms can realize cost reductions (the "output" of R&D) above and beyond what they could obtain if they had to rely solely on their own R&D expenditures.3In general, the greater the potential level of R&D spillovers within consortia, the greater the equilibrium level of R&D expenditure by member firms.However, Katz points out that under certain conditions, R&D consortia can be welfare enhancingeven when they reduce the actual level of R&D expenditure-because the effective level of R&D (and therefore, R&D output) is raised through spillovers.Higher levels of effective R&D lead directly to higher levels of welfare since, in these models, R&D reduces production costs, increases output, and lowers prices.
The second key attribute is the level of ex post product market competition among the firms participating in the research consortium.Some-perhaps all-of the private benefits of cooperative R&D, in terms of raising firm profits, could be dissipated through product market competition.When the level of product market competition among participating firms is intense, the gain in profits to a single participating firm through reduced production costs is negated by the accompanying fall in the costs of its rivals.In this case, the principal effect of cooperative research is to lower the product price and raise consumer surplus, not profits.Anticipating this outcome, firms will seek to set lower levels of R&D in the cooperative "consortium" stage than would obtain in an equilibrium without research consortia.Ceteris paribus, a greater level of competition reduces the effective level of R&D input, and thus reduces R&D output, leading to a decrease in welfare.4 The net impact of R&D consortia will depend, in a critical way, on the values of these two key attributes.The key innovation of our paper is that we provide quantitative measures of these attributes for individual research consortia.Thus, we can empirically estimate the impact of these attributes on the R&D output of the participating firms in linear and nonlinear regression models, controlling for R&D input.Following a line of argument similar to Katz, we presume an increase in research output leads to increased social welfare.

II. R&D Consortia in Japan5
The empirical analysis in this paper uses data on all company-to-company cooperative R&D projects formed with a degree of government involvement from 1980 through 1992.This data set was collected from each ministry through direct contacts after examining a wide range of govemment white papers and other government publications, and is as close as possible to an exhaustive list of all the government-sponsored R&D consortia in Japan during that time period.6 In general, Japanese R&D consortia involved some government subsidization of consortia R&D expenditures, lowering the effective cost of R&D.Secondly, the government generally sought (not always successfully) to encourage complete dissemination of all research results to the participating firms.Furthermore, in select-ing participants for consortia formed since the early 1980's, the government generally sought to bring together firms with complementary research assets.7This implies the level of intraconsortium spillover could be quite high.Consortia often brought together firms that were not direct rivals in the product market, and they often targeted markets where Japanese firms played a small role in global production and trade.Both factors worked to minimize the potential negative effects of R&D consortia on industry profits.Finally, prior to 1990, many if not most of the patents that directly emerged from the research undertaken within government-sponsored research consortia were, by government directive, assigned not to the participating firms but to the research consortia themselves.8We obtained data on the patents assigned to these consortia as well as those assigned to participants.

A. Measurement of Consortium Outcomes
Before investigating the relationship between consortium characteristics and consortium outcomes, we first identify the overall impact of research consortia on participating firms' research output and the time path of that impact.In Katz's model, the only "output" of a successful R&D project is a reduction in the marginal production costs of firms.Because of the long, variable lags involved in converting a research advance into a process improvement (or a new product), it is practically impossible to measure the effect of R&D consortia on marginal cost with any degree of accuracy, particularly when the participating firms are large, multiproduct firms and the consortia each target only a small part of the firms' product/technology portfolios.
Patents provide a more direct and easily measured index of the innovative output associated with research consortia.We assume Japanese firms patent a fraction of their economically 4Following Spence (1984), if one thinks of a product as the services it delivers to the consumer, and R&D reduces the cost of delivering those services, then "product-innovation" R&D could be modeled in the same way as "costreducing" R&D. 5 We note that this section draws heavily from Branstetter and Sakakibara (1998). 6The data on consortia were originally prepared for Can Japan Compete? by Michael E. Porter et al. (2000).For further details on construction of these data, see Sakakibara (1997a, b).useful innovations, though this fraction can be allowed to vary across firms and industries.Provided this assumption is correct, the ultimate impact of a consortium on its members' innovation can be measured by looking at the increase in patenting in the targeted technologies registered by the participating firms during and after the consortium.9

B. Data
For every consortium, we know which firms participated, the total R&D budget of the consortium, the government's contribution, the consortium' s duration, and its technological goals.With the aid of the Japanese Patent Office, we have been able to construct a mapping from the description of the consortium goals in the official program documentation to codes or groups of codes in the International Patent Classification system.This mapping allows us to measure the patenting of consortium firms in the targeted technologies.'0When we aggregate across participating firms within a consortium, such that the consortium itself is the unit of analysis, we can add to our output measure the patents taken out in the name of the consortium itself.
In addition to information on firms' patenting in the targeted classes in Japan, we have also constructed measures of firms' patenting in the targeted classes in the United States.Having this alternative patent series provides us with a useful robustness check.In taking out a patent in the United States, a Japanese firm has to translate the patent application and reformat it in a way that conforms to the very different requirements and procedures of the U.S. patent system.Interviews with Japanese firms and empirical evidence suggest Japanese firms will only go through this additional trouble for the ideas that they perceive, at least ex ante, to have the most promise."Thus, the U.S. patent series provide us with a "quality-adjusted" stream of patents.'2Sample statistics of the data used for our empirical analyses are presented in Table 1.This gives data on the cross section of consortia.' 3 Matching the consortium data to firm-level data yields usable data on about 145 consortia.As the standard deviations for each variable make clear, there is substantial heterogeneity across consortia in terms of the mix of participants, their preexisting technological strengths in the targeted areas, and the total resources expended on the consortium.Data on R&D spending, patenting, and most other variables for participating firms are available from 1980 through 1994.

C. Overall Time Path of Impact
If R&D consortia were "successful," in that they stimulated or enhanced research productivity, then, controlling for research inputs and preexisting technological strength, we should observe a consortium-induced increase in patenting in the targeted area.However, unless we can compare ex post trends in the patenting of participating firms to those of similar firms that did not participate, it is difficult to get around the sample-selection problem identified in the introduction.Fortunately, our data set allows us to break up the individual consortia into the participating firms.For most consortia, these participating firms can be matched to a broadly comparable set of nonparticipants.This disaggregation allows us, at least in principle, to take 9 There are obvious issues of causal inference that later sections will address.Note that survey evidence presented by Akira Goto and Akiya Nagata (1997) suggests that Japanese firms patent a substantial fraction of their process innovations as well as their product innovations.
10 It is important to point out that our mapping from the technological goals of a consortium to the related patent classes almost certainly captures innovations that were related to but not necessarily part of the actual goals of the consortium.This is useful in that it allows participation in a consortium to have a positive impact on R&D in related fields.It is potentially problematic in that it may systematically overstate the actual innovative output resulting from a given consortium.However, if we find an increase in patenting in the classes related to a consortium at precisely the time a consortium is being undertaken, then it is likely that this increase may be driven-at least in part-by the consortium.
" Furthermore, whereas our Japanese data are based on patent applications, our U.S. data are based on patent grants (dated by year of application)-that is, applications that have been judged by the U.S. patent office to be sufficiently innovative to merit a patent grant.Data sources are discussed in the Data Appendix.
12 Rebecca Henderson and lain Cockburn (1996) have also used patents taken out in multiple countries as a "quality-adjusted" measure of innovative output. 13We emphasize that these sample statistics are drawn from only one of several alternative "cuts" of the data employed in this paper.a "difference-in-differences" approach to the data. 14  To estimate the consortium-induced increase, if any, on innovative output, we take the participation of firm i in consortiumj in year t as the unit of analysis, and we count all 14 Along with these advantages of disaggregation come two very important disadvantages.First, we cannot apportion the patents taken out in the name of the consortium to the individual firms, which means we cannot fully capture the innovative output of consortia at this level.Second, we do not have perfect information on how government subsidies or the participating firms' total private contributions to the consortium research effort were divided across firms and years.This means firms' R&D inputs are, at best, measured with considerable error.Because these measurement problems are slightly less problematic when one aggregates across the firms in a given consortium, we also experimented with regression analysis at the consortium level of aggregation.Some results from this level of aggregation will be presented in this paper, for comparison purposes.
the patents taken out in the targeted area(s) by each participating firm in the years prior to, during, and after the joint research project undertaken by the consortium.'5These annual patent counts are then regressed on a number of explanatory variables.16Real budget represents our estimate of the total research resources-private funds and public subsidies expended by firm i in consortium j in year t on the technologies targeted by that consortium.Unless otherwise specified, this variable and all other measures of research inputs are measured in logs, in order to obtain regression coefficients with an elasticity interpretation.Pre-project patents denote the average patenting by firm i in the technology classes targeted by consortium j prior to the start of that consortium, with the average taken over a five-year window prior to the official start date of the consortium.This variable is included to control for the preexisting technological strengths of consortium participants in the targeted classes.Real indirect inputs give our best estimate of the resources that may have "seeped in" to the inputs of firm i as a consequence of its participation in overlapping consortia (that is, other consortia whose technological goals overlap with those of consortium j) in year t.17Note that here, and throughout the paper, t is measured with respect to the inception of consortium j, rather than "calendar time." A skeptical view of research consortia would be that any positive impact on the innovative output of participating firms is produced entirely by the resources, public and private, expended on the project.If we find an increase in research output that remains even after controlling for consortium-related increases in R&D inputs, then this would constitute evidence that consortia actually enhance research productivity.We test for such an increase by estimating a set of dummy variables to capture the change in patenting associated with the individual years after the inception of the consortium.For instance, in our basic specification, year 0 dummy represents a dummy variable that takes a value of I in the year of the inception of the consortium, and 0 otherwise.Other dummy variables correspond to lags of set length after the inception of the consortium.18 A more complete exploitation of the information provided by our "control" (that is, nonparticipating) firms would require a modification to this approach in which we estimate two "time paths" using two sets of dummy variables.The first set of year dummy variables captures the changes over time in patenting in the targeted technologies that are common to Japan's technologically elite firms, both participants and nonparticipants.'9 If consortia are only created in "hot" technological areas, in which there would have been an increase in patenting even in the absence of the consortia, then this first set of year dummy variables will capture that effect.The second set of year dummy variables captures those changes that are unique to participating firms.If these year dummy variables are significantly different than zero, then that would suggest participation in consortia has a positive impact on the research outcomes of participating firms.

D. Results at the Firm-Consortium Level
For comparison purposes, column (1) of Table 2 provides the results of a regression of the initial specification on data aggregated up to the consortium level.Column (1) reports results from a linear specification, using the log of '5We thank Adam Jaffe for this suggestion. 16In estimating linear regression models, we confront the problem that a number of observations of our dependent variable (and some observations of our independent variables) have values of zero.To avoid the problem of taking the log of zero, we first add "1" to each observation.This transformation is standard in the R&D/productivity literature.Regressions using count data models yield qualitatively similar results.These results and other additional results whose tables are not presented here are reported in Branstetter and Sakakibara (2000). 17Issues regarding the measurement of these indirect inputs are discussed further in the Data Appendix. 18Consortium duration in our sample runs from 1 to 13 years, with half of all consortia ending 4 to 8 years after the official inception.
19 Sample firms were pre-selected on the availability of R&D data and patent data in both Japan and the United States.Obviously, we oversample R&D-intensive firms, hence our characterization of both the participants and the control sample as "technologically elite" firms.Figure 2, along with the 95-percent confidence bounds.Briefly summarized, these results suggest that both participants and nonparticipants tend to increase their patenting in the targeted technologies after the inception of a consortium.20However, the marginal increase of participants' patenting in the targeted area, relative to the control firms, is large and statistically significant.21The shape of this marginal effect on innovative output is also interesting.On average, this marginal impact tends to level off or decline in magnitude and lose statistical significance in the fourth to eighth year after the consortium's inception, increasing substantially thereafter.This is the point around which the average consortium's official duration is ending.
Why does Japanese patenting in the targeted area by the participating firms level off-even begin to decline-as most consortia are ending, then increase substantially thereafter?We suspect the reason is linked to the requirement, which held for many of the consortia we looked at, that patents taken out directly as a function of consortium research were to be assigned to the consortium, preventing the firm from fully appropriating the benefits of its research within the consortium.Given this restriction, there was obviously an incentive to delay patenting useful discoveries until after the consortium officially ended.The time path of benefits is quite consistent with the view that this "moral hazard" problem existed, and that firms behaved in an opportunistic manner.22 Our approach would more closely approximate the "difference-in-differences" studies undertaken in the labor economics literature if we included firm and consortium fixed effects.Unfortunately, the inclusion of fixed effects makes it difficult to estimate two statistically distinct time paths.However, it is possible to estimate a fixed-effects model using a more "parametric" approach to the estimation of the "time path of benefits."23We can construct for each firm-consortium-year observation (both participating firms and controls) a dummy variable corresponding to 1 during the official duration of the consortium (zero otherwise), a time trend which dates from the inception of consortium j, and the square of this time trend.Then we estimate another dummy variable equal to 1 during the duration of the consortium only if firm i is a participant in consortium j, zero otherwise.Similarly, we construct the time trend and its square where these variables are nonzero only if firm i is a participant in consortium j.This allows us to trace out, albeit less precisely, the general shape of the time path of changes in patenting in the targeted area for all firms and then the impact of participation on the participating finns only relative to that baseline.
Results from such a specification suggest participating firms receive a statistically significant boost in performance over the level of nonparticipants that persists over time.24However, the increase in performance is rather small in size (on the order of 5 percent using U.S. patents as the dependent variable).While the results given in Table 2 would seem to imply quite large and persistent positive effects, the specifications which most closely approximate a "differencein-differences" approach suggest that this impact is much more modest in size.

IV. Effects of Consortium Characteristics on Consortium Performance
In this section, we turn our attention to the question of which consortia are more successful-and why.As we noted in Section I, the theoretical literature highlights two characteristics of an R&D consortium as critical determinants of its impact: spillover potential of the consortium and the level of product market competition among consortium participants.So it is particularly incumbent upon us to come up with empirical measures of these characteristics.There are other characteristics that we wish to take into account as well, including the govemance structure of consortia, their technological orientation (basic versus applied research), and the mix of participants in terms of industry affiliation.
In the empirical analysis in this section, we concentrate our attention on time-invariant characteristics of consortia.While our initial specifications exploited the full panel structure of the data and gave us insight into the important question of the time path of benefits, many of the variables on consortium characteristics that we have at the consortium level (or the firm-consortium level) do not change over time.Including them in a panel regression with a time-series dimension actually creates statistical problems, as has been demonstrated by Brent R. Moulton (1986).For that reason, it makes sense to collapse the time-series dimension of the data.What we do henceforth is measure consortium outcomes as the cumulated sum of patenting in the targeted classes, taken over a 15-year horizon (or as much of this as the data will allow) from the official inception of the consortium.This sum will be regressed on (summed) measures of research inputs, direct and indirect, measures of pre-consortium technological strength, and time-invariant consortium characteristics.

A. Technological Proximity of Participants
If the potential for R&D spillovers is strongest among firms which are pursuing research in the same technological areas, then one needs a quantitative index of the proximity of firms in technology space.25Following Adam B. Jaffe (1986), we employ such a measure, the T coefficient.
Let a firm's R&D program be described by the vector F, where Fi = (f1 fk) and each of the k elements of F represents the firm' s research resources and expertise in the kth technological area.This is measured by the number of patents held by a firm in a narrowly defined technological field.We can measure the "technological proximity" between two firms by measuring the degree of similarity in their patent portfolios, or more precisely, the "distance" in "technology space" between two firms i and j can be approximated by Ti, where T1i is the uncentered correlation coefficient of the F vectors of the two firms.This is calculated by dividing FjF) by the square root of the product of FjF' and FjFj.We calculate a technological proximity measure for each consortium for which we have sufficient data by averaging Ti1 for all pairs of firms in a consortium.A number close to 1 implies a high degree of technological proximity, while a number close to 0 implies a low degree of proximity.For a subset of our firms and consortia for which we have sufficient data, we can calculate a separate technological proximity measure for each firm in each consortium by averaging the Ti term over all firms j not equal to i.This construction gives us a measure we can use in an econometric specification that includes firm and consortium fixed effects.26

B. Product Market Proximity of Participants
Our measure of product market proximity is calculated using data from Market Share in Japan, which is published by the Yano Research Institute (1984,1990).This private Japanese market research firm tracks the market shares of the top Japanese firms in hundreds of narrowly defined product markets.27We use these data to count the number of times a pair of firms in a given consortium "meet" each other in product markets.For each firm, a proximity measure with respect to each other firm can be calculated by dividing the number of product markets in which a meeting takes place by the number of product markets in which firm i is currently active.Two firms which meet one another in a large number of product markets are presumed to be more proximate to one another than firms for whom the set of overlapping products is small or zero.
However, our measure of proximity does not guarantee symmetry.For any pair of firms, i may be closer to j than j is to i if i is in only one product market (and meets j in that market) whereas j is in 100 product markets, in only one of which it meets i.For i, j is a major and close competitor, while for j, i's presence is negligible.This measure captures the asymmetries of product market competition that exist in the real world between multiproduct and single-product firms of very different sizes.As in the case of our technological proximity measures, we calculate product market proximity in two waysaveraging over all firms within a consortium to produce a consortium-specific measure and averaging within firms and consortia to produce a firm-consortium-specific measure.

C. Organizational Characteristics
Basic Technological Orientation.-Fora subset of firms and consortia in our database, we have a rich set of qualitative variables recording Japanese R&D managers' perceptions of various aspects of the consortia.28These variables were obtained from a survey conducted by Sakakibara (1995Sakakibara ( , 1997a, b), b).Among the most interesting and relevant of these survey responses are those pertaining to the nature of the technological goals of the joint research projects undertaken by the consortia.While obviously highly subjective, the respondents' answers to the following questions nevertheless provide quite useful information on the technological "ambitiousness" and technological "focus" of the project.Questions include the following: (1) Rank the project outcomes along a spectrum from basic to highly applied research.(2) How ambitious was the goal of the project?(3) What was the state of development of the subject industry?The respondents' answers to these questions were recorded on a five-point Likert scale.These responses were then averaged for participants in the same project.Because a number of these variables measure similar factors, there are potential problems of multicollinearity and interpretation.We collapse a number of these variables into a simple linear combination of the variables that serves as a univariate summary statistic of the "basicness" of research conducted in the consortium, with a larger index indicating that the consortium targeted more basic research.
When R&D consortia focus on basic R&D, the effective level of ex post product market competition could be quite low, even if the participants are quite proximate, in terms of their current product portfolios.With our data, we can measure the level of product market competition along both of these dimensions.Industry Mix/Wide Participation.-A third set of survey questions dealt with the "partici-pation pattern" of firms in the consortia.This provides useful information on the "inclusiveness" of the consortium, such as the presence of firms from upstream/downstream industries.Questions include the following: (1) How wide was participation in your principal industry?(2) How wide was participation from other industries?(3) Did upstream or downstream industries participate?The aggregated single univariate measure is constructed such that the larger the index, the greater the diversity in terms of the industry mix of participating firms.

D. Results at the Firm-Consortium Level
The impact of consortium characteristics on consortium outcomes could be measured at a number of different levels of aggregation.For instance, we can use the consortium itself as the unit of analysis.29While interesting and potentially informative, the consortium-level regressions are subject to a serious identification problem.We would like to interpret the coefficients on the characteristics of our consortia as giving us the marginal effect of a unit change in a given consortium characteristic on the outcome of a consortium.The problem with this interpretation is that the coefficients on our consortium characteristics could be reflecting differences in the participating firms across consortia as much as they reflect the "ceteris paribus" marginal impact of a unit change in a given consortium characteristic.
In order to address this issue, we utilize our "firm-consortium" cut of the data.We can collapse the time dimension of this data set, summing up measures of patent output and R&D input over a fixed horizon beginning with the inception of the consortium.However, even after collapsing the time dimension, we are left with two other dimensions to our data set-the project, or consortium, dimension, and the firm dimension.This gives us critical leverage around the problem in the preceding paragraph.
Ideally we would like to conduct the following conceptual experiment: to examine how the same firm would perform if, for example, we moved it from a consortium with a low level of average technological proximity to one with a high level of average technological proximity.Since we observe the same firms in a number of different consortia, a regression on our firmconsortium panel data set with firm fixed effects allows us get close to this ideal experiment.Holding the unobserved characteristics of individual firms constant, we can trace out the average marginal impact of differences in consortium characteristics on research outcomes.30 We would get even closer to this ideal experiment if we could also include consortium fixed effects.Our measured consortia characteristics could potentially be correlated with unmeasured characteristics of those same consortia.Employing consortium fixed effects provides us with empirical leverage concerning this issue, at least in principle.The practical problem with including consortium fixed effects is that any characteristic that is the same across all firms in a given consortium "falls out" with the fixed effect.Unfortunately, some of our consortium characteristics are not available at the firm-consortium level, but only at the consortium level, and we are thus constrained from including consortium fixed effects in some specifications.
Given these constraints, there are two ways of measuring the consortium-induced boost to firm patenting in the targeted patent classes in these data.One way is to regress the cumulated sum of patenting in the targeted area on inputs, pre-project patenting, consortium characteristics, firm fixed effects, and, in some cases, consortium fixed effects.Alternatively, we subtract a cumulated sum of pre-project patenting from the output measure, and use this difference (or in our case, the difference of the logs) as the dependent variable.Table 3 takes both approaches to the estimation of the impact of technological and product market proximity on outcomes.
For comparison purposes, column (1) of Table 3 presents results from a Negative Binomial model estimated at the consortium level of aggregation.At this level of aggregation, it is impossible to include either firm or consortium fixed effects.Furthermore, measures of technological proximity and product market proximity are averaged across participating firms within a consortium.Nevertheless, we find that technological proximity and product market proximity have the predicted effects on consortium outcomes.The former is significantly positive in its effects, while the latter is significantly negative.
In column (2), we move our analysis to the firm-consortium level.However, we continue to use measures of technological and product market proximity that are consortium specific rather than firm-consortium specific.The regression is run in logs, incorporating firm (but not consortium) fixed effects.31Taking a slightly different approach, column (3) presents results from the "differences" specification suggested above, where the dependent variable is the difference in the (log) patent output in the targeted area before and after the inception of the consortium.In column (3), U.S. patents in the targeted area, rather than Japanese patents, are used as indicators of innovative output.Columns (2) and (3) illustrate that the estimated effects of our two key consortium characteristics still have the predicted signs and are still statistically significant.
For all the reasons discussed above, an even more stringent test of our hypotheses would be to insert firm-consortium-specific measures of technological and product market proximity into a specification with both firm and consortium effects.When we construct these firm-consortium-specific measures, we lose a number of observations, because we lack sufficient data to compute these measures for all firms in all consortia.Furthermore, including firm effects and consortium effects absorbs a very large portion of the total variance in our outcomes data.Given the loss of observations and variance we incur in taking this approach, it would perhaps not be surprising if our econometric results lost some of their statistical precision.Nevertheless, as can be seen from column (4) of Table 3, when we include firm fixed effects, consortium fixed effects, pre-project patenting, and measures of private and public R&D input as independent variables, it is still the case that our measures of technological proximity are positive and statistically significant.On the other hand, our measures of product market proximity, though negative, are not statistically significant at conventional levels.33We view the fact that our technological proximity measures survive this test and the product market proximity measures do not survive it as being essentially consistent with results re-ported in the next table, in which the impact of product market proximity is systematically less robust than that of technological proximity.Column (4) uses U.S. patents as its measure of innovative output, but we note that results obtained with Japanese patents are qualitatively similar.34Table 4 reports the results of regressions similar to those in Table 3, except that here 33 Since measured direct R&D inputs are the same for all firms within a given consortium, this variable is not included in the regression whose results are reported in column (4). 34Results using Japanese patents as the measure of innovative output in a specification with firm and consortium fixed effects indicate that the impact of technological proximity is positive and statistically significant, whereas the impact of product market proximity is statistically indistinguishable from zero.The fact that the measured impact of product market proximity is systematically less robust than technology proximity could suggest that the former is measured with substantially greater error in our data.For further robustness tests, see Branstetter and Sakakibara (2000).we add measures of consortium organizational characteristics-basic research orientation, "centralization" of the consortium governance structure, and measures of "diversity" in terms of the industry affiliation of participating firms.Unfortunately, it is not possible to obtain firm-consortium-specific measures of these characteristics.Recall that they were obtained from a limited survey of participating firms, and the number of firm responses within a given consortium is too small.For this reason, we use consortium-specific measures of these attributes-and that precludes the use of a consortium fixed effect.The only organizational attribute with robust effects is the one for which the theoretical literature gives us a clear interpretation-the basic research orientation of consortia.The more basic the research conducted within a consortium, the better the outcome.This is entirely consistent with theoretical predictions.Note that we do not possess measures of these organizational characteristics for all firms or all consortia.Our data set here is less than one-third the size of the data set used in Table 3, column (3), for instance.Given this loss of observations, it is not surprising that some of our measured character-istics lose statistical significance.The impact of product market proximity switches sign (relative to what we found in Table 3), but is statistically indistinguishable from zero.
One final observation merits mention here.In a number of specifications in Tables 3 and  4, the regression coefficients on our measures of R&D inputs to consortia are quite small in magnitude and in statistical significance.35One interpretation of this is that the design of a consortium matters much more than the level of resources expended on it.Putting more money into a consortium in which the members have little prospect for technological spillover, little incentive to cooperate given their overlap in the product market, and little preexisting technological strength in the targeted technologies will not help matters much.Likewise, a well-designed consortium may have beneficial effects even if the direct subsidies expended are modest.36

V. Conclusions
In this paper, we find strong evidence that spillover potential, as measured by our technological proximity variable, is positively related to the outcomes of consortia, which is consistent with the predictions of much of the theoretical literature.We also find evidence that research consortia are likely to have a stronger positive impact when they conduct relatively basic, rather than relatively applied, R&D.Our evidence concerning the impact of the degree of product market competition is generally consistent with theoretical predictions, though these results are less statistically robust.Another consistent regularity in our empirical findings is that the design of a consortium seems to be more important than the level of resources expended on it in terms of explaining research outcomes.Taken together, these results suggest to us that the strategic reactions of firms to consortium attributes identified in the theoretical literature are empirically important in practice.This is a lesson that can and should be incorporated into public policy.
Our results suggest what kinds of consortia are likely to yield the highest returns on both public and private investments and how these benefits unfold over time.The empirical framework in the paper is one that could be applied to research consortia in any country.It is our hope that this paper will stimulate such evaluative work.Partly as a result of economists' theoretical arguments, governments and firms around the world have invested billions of dollars in research consortia.Our profession would be remiss if we did not provide some way of evaluating the impact of this investment.Accounting for inputs in consortia.To measure and compare the real "R&D productivity" enhancing effects of consortia, we need to account for the inputs that are invested into these consortia.While we know the official total budget of each consortium and the official duration of each consortium, we do not have detailed, consistent information on how these expenditures were allocated across participating firms and over time.We are generally constrained to assume that expenditures were divided equally across participating firms and across years of the consortium's operation.However, even if we had perfect information on the allocation of expenditure across participating firms and over time, we must acknowledge that this would still not be an accurate measure of the real contribution of firms.This is because we know from firm interviews that, when firms believed the technology being pursued within the research consortium had a high degree of commercial potential, they would often conduct a parallel research program on this technology within the firm, to maximize their own firm's ability to commercialize this technology upon the completion of the consortium.Because we have no way of measuring the distribution of a firms' R&D spending across different technologies at a particular point in time, we have no way of controlling for the existence of these parallel internal research programs.
Finms tended to participate in multiple consortia and consortia tended to target similar classes of technologies.Thus, there was a considerable degree of overlap in the consortia, both in terms of participating firms and targeted classes.The impact of previous consortia is captured by measures of pre-consortium patenting in the targeted technological classes.The impact of concurrent overlapping consortia is controlled for in the following manner.If firm i, currently participating in consortium X (the consortium of our focus) is also participating in consortium Y (a concurrent consortium), then firm i's share of the total budget for consortium Y is multiplied by the degree of technological overlap between X and Y in terms of targeted patent classes, and this product is imputed to firm i as "indirect inputs" to its participation in consortium X.A similar imputation is done for consortia that follow after X and target some of the same firms and classes.
empirical implications of the theoretical work by Katz and others.Some of the recent empirical literature is summarized in Stephen Martin (2001) and Donald Siegel (2001).
FIGURE 1. TIME PATH OF BENEFITSSource: Table2, column (1)'s coefficients of consortium year dummies and the 95-percent confidence bounds.These results are taken from the consortium level of aggregation.
Organization.A separate set of survey questions dealt with the management structure of the joint research project undertaken by the consortia.Questions asked on this topic include the following: . Based on the descriptions of the technological goals of each consortium, officials of the Japanese Patent Office identified the classes of the Japanese patent system which were most closely related to those goals.We then acquired from JAPIO counts of patent applications in the targeted classes (as well as overall patent applications) for nearly 500 participating firms and control firms.U.S. patent data.The data on patents taken out in the United States by Japanese firms were taken from the CASSIS CD-ROM, then matched to the REI database at Case Western Reserve University.Creating counts of U.S. patent grant data in the targeted classes required us to create a mapping from the International Patent Classification system to the classification system used by the U.S. Patent and Trademark Office.R&D data.The overall R&D spending of individual Japanese firms are taken from several consecutive issues of the Kaisha Shiki Ho, published by Toyo Keizai, and the Nikkei Kaisha Joho, published by the Nihon Keizai Shimbunsha.All R&D expenditure data was deflated by the R&D price index constructed by the Japanese Science and Technology Agency and reported in Gijutsu Yoran.Other firm variables.Data on firm industry affiliation and other variables are taken from various issues of the Japan Development Bank Corporate Finance Database.