On 17 October, the Commission published a revised version of its Best Practices for the submission of economic evidence and data collection in competition cases (“BP”). The first version of the BP, published in January 2010, included guidance on issues such as the way relevant questions in economic submissions should be formulated, choice of methodology, reporting the results, conducting robustness checks, and responding to requests for quantitative data. Compared to the initial draft, the new version of the BP contains relatively few changes. However, the changes that have been made are worth noting.
The most important addition to the new version is the inclusion of some guidance on consumer surveys. This guidance mainly deals (at par. 30 and 31) with the way respondents to surveys should be sampled in order to avoid biased survey results. In addition, the BP contains some guidance on how survey results should be disclosed to the Commission (par. 39) and also notes that when conducting large scale surveys to address a case-specific issue, the parties “might want” to involve the Commission in the questionnaire development and design (par. 48).
However, while the BP’s guidance on surveys is of course welcome, it suffers from two major shortcomings.
First, the guidance is largely restricted to sampling issues and is completely silent on the actual design of the survey. Unrepresentative samples can reduce the reliability of survey results, as the BP acknowledges, but the scope for the actual survey questions to give rise to biases is similar or perhaps even greater. For example, biases can occur as a result of the way survey questions are phrased (the question “do you agree that the merger will give rise to higher prices” is a leading question that is highly unlikely to produce objective outcomes), as a result of the order in which the questions appear, as a result of the way response categories are organised (for example, respondents are more likely to select the first response given than the second), etc. The consumer surveys that the Commission occasionally commissions itself (e.g. in phase II merger cases) are frequently subject to such biases and it is therefore perhaps unsurprising that the Commission has chosen not to give any guidance in this area. However, this is obviously unsatisfactory. The recently published joint guidance on survey design in merger inquiries by the OFT and the Competition Commission does a much better job in this area and represents a model for the Commission to follow.
Second, the BP is at pains to point out that its guidance only applies to large-scale consumer surveys, not to the “qualitative” or “documentary” evidence obtained from the Commission’s own market investigations. As the BP points out, “whilst the evidential value of replies to information requests from market participants lies in the substance of the information provided by players with intrinsic industry of market knowledge, the specific purpose of large-scale surveys of final consumers is to obtain statistically significant data”. It is correct that respondents to a market investigation by the Commission may be fewer in number and more diverse than respondents to a large-scale consumer survey. However, this does not excuse the Commission from the need to draw objective inferences from the results of market investigations. For example, in the Oracle/Sun investigation, the Commission initially suggested that the transaction would be anticompetitive because, among other reasons, a total of seven customers appeared to have raised concerns. But the vast majority of consumers did not see any problems and many explicitly stated that the transaction would be pro-competitive. A large-scale consumer survey would in such a case have pointed to the conclusion that the transaction would not be anticompetitive and the Commission should clearly have drawn the same conclusion from its market investigation.
A further key change in the BP reflects an apparent increased willingness by the Commission to accept economic analyses that are subject to limitations. Economic analyses can never be a perfect reflection of underlying market realities and it is therefore appropriate that the Commission has softened its previous stance in certain areas. For example, par. 37 now explicitly recognises that economic analyses can be useful even if they only show approximate results or are subject to a certain degree of inaccuracy or bias. Indeed, techniques exist with which inaccuracy or biases can be contained (for example by providing confidence intervals or upper bound estimates) and the BP helpfully recognises these.
The new version of the BP also adopts a revised stance on statistical significance. In the previous version, the BP required statistical significance in principle to be shown at a pretty strict level (the “5%-level”), with a less strict level (the “10%-level”) only deemed acceptable in specific circumstances. By contrast, the new version quotes both the 5% and 10% levels as possible ways in which statistical significance can be shown without expressing a preference. Moreover, a new statement has been added that “just because some hypothesis cannot be rejected in a statistical sense does not necessarily mean that the empirical analysis has no evidentiary value”. This is in principle a welcome statement. When only limited data are available, it may not be possible to show that particular effects are significant in the statistical sense, even though they may be highly significant from an economic point of view. Moreover, the Commission has in the past sometimes been overly concerned about statistical significance without paying sufficient attention to the economic context of its findings.
However, there is always a risk that the Commission will primarily use such looser requirements to increase its own margin of discretion. Indeed, a further set of changes explicitly appears to go in this direction. In particular:
- An explicit warning has been added (par. 15) that the Commission may give less weight to economic submissions that do not strictly meet the standards set out in the BP, or may discard such submissions altogether.
- In par. 8, which previously stated that the specificity of an individual case may require a deviation from the BP, a sentence has been added that the BP should moreover be interpreted “in light of procedural and resource constraints”.
- A new footnote (footnote 2) has been added that economic models or econometric analysis will rarely, if ever, prove conclusive by themselves, leading into a statement that “the Commission can always take into account different items of evidence”.
Finally, the BP contains a small but significant (and welcome) change relating to the role of economic analysis in cases involving restrictions by object versus restrictions by effect. In its original version, the BP stated (footnote 1) that infringements by object do not require any substantive economic analysis “because empirical evidence has shown that they generally lead to serious anticompetitive effects”. By contrast, the original version stated that infringements by effect often require a complex economic assessment. The revised version still makes the point regarding the need for economic analysis in by effect infringements, but, interestingly, has dropped the point regarding the absence of any need for economic analysis in assessing infringements by object. This is consistent with the Commission’s position in other documents. For example, when discussing information exchanges in the recently published horizontal cooperation guidelines, the Commission has stated that in assessing whether an information exchange constitutes a restriction of competition by object, the Commission “will take into account whether the information exchange, by its very nature, may possibly lead to a restriction of competition”. In other words, the Commission has set itself a condition that requires it to pass an economic test before concluding that information exchanges are restrictions by object. This arguably does not require a full quantification of competitive effects, but does imply that economics has some role to play even in certain “by object” cases.
In short, the revised version of the BP generally represents a further improvement on what already was a very useful document. As before, the key question is to what extent the Commission will also apply the guidance to its own analysis of the data and responses that it receives during the course of its investigations. The key area to look out for in future versions of the BP is guidance on the actual design of consumer surveys.
________________________
To make sure you do not miss out on regular updates from the Kluwer Competition Law Blog, please subscribe here.
