Navigation – Plan du site

AccueilNuméros2The paradox of measuring success ...

The paradox of measuring success of nations in elite sport

Mesurer les performances des nations en sport de haut niveau : un problème paradoxal
Veerle De Bosscher, Bruno Heyndels, Paul De Knop, Maarten van Bottenburg et Simon Shibli
p. 217-234

Résumés

Les performances sportives au niveau international, et particulièrement au niveau olympique, prennent de plus en plus d’importance pour un nombre croissant de nations. Mais il n’est pas évident de définir ni de mesurer le succès. Le nombre de médailles gagnées lors de Jeux olympiques ou autres compétitions internationales représente la mesure la plus objectivable et la plus transparente du succès dans le sport de haut niveau. Dans cet article, nous comparons différentes méthodes permettant de mesurer ce genre de performances. La part de marché a été identifiée comme la meilleure mesure de succès absolu permettant de procéder à une analyse de séries temporelles significative. Une analyse en régression linéaire est utilisée pour présenter le succès relatif comme mesure du succès lorsqu’on maîtrise des facteurs macro-déterminants tels que la population et la richesse. Cette méthode permet de comparer les différents pays sur des bases plus égales, ce qui est indispensable si l’on veut mesurer l’efficacité des politiques en matière de sport d’élite. Nous procédons ensuite à une analyse similaire pour les Jeux olympiques d’été et d’hiver, avant de conclure que nous parvenons à des résultats contradictoires quant aux performances des différents pays. En effet, le succès dépend du but dans lequel on l’utilise ainsi que des priorités de chaque nation.

Haut de page

Texte intégral

Introduction

1Along with the FIFA Football World Cup, the Summer Olympic Games is the most high profile event in the global sporting calendar. In recent Games, the International Olympic Committee (IOC) has sought to make the event truly global. This point can in part be appreciated by looking at the number of nations taking part in the summer Olympic Games: an increase from 14 in 1896, over 59 in 1948 to 201 in 2004. The number of athletes competing in the Games started with 241 in Athens 1896 and has more than doubled in the period 1980 (5,217 competitors) to 2004 (11,099 competitors). It is thus reasonable to conclude that the summer Olympic Games have become increasingly competitive and medals have become relatively harder to win. As the supply of medals (success) will remain essentially fixed in the future (the IOC has indicated that it would like the number of events to be capped at around 300), and the demand for success is increasing (more nations taking part and more nations winning medals), the “market” adjusts by raising the “price of success” (Shibli, 2003). Nations will have to invest even more just to maintain their success. It is implicit that the “production” of successful elite athletes by nations is an output from a strategic planning process (De Bosscher, 2007). It follows that if nations are adopting a strategic approach to the production of elite athletes, then part of that process must be to evaluate the results achieved (outputs).

2The definition of sporting success is a theme which has been widely discussed in both the literature and media with respect to international comparisons. Success is typically expressed in absolute terms, such as the total number of medals that a country wins during the Olympic Games or other championships. Such comparisons of countries, however, give a biased view of how successful countries actually are in using their available resources. These accounts do not take into consideration a number of socio-economic and political variables that play an important part in determining each country’s success. The Olympic Games are still dominated by a small number of capitalist core and (formerly) socialist countries (Green & Houlihan, 2005; Stamm & Lamprecht, 2001). One should hardly be surprised that large and rich countries like the United States tend to win more medals than Zimbabwe or Ecuador. Yet there are exceptions, such as Cuba, Ethiopia and Kenya. The question is how external factors should be taken into account when defining success. The aim of this paper is to examine various methods by which the outputs of an athlete production system can be measured. The definition of absolute national success using the medal tables from the 2004 Athens Games will be taken as our point of departure. We shall then proceed to examine relative success, or rather success controlling for economic, sociological and political determinants. Furthermore we also examine the limitations of the analysis and propose some alternative measures using other events.

Measuring absolute success of nations

3Although the IOC does not recognise the Olympic medal table as an order of merit, it is widely accepted outside of the IOC that the final medal table for each games is an order of merit. This finding is perhaps best demonstrated by the fact that many nations invest heavily in sport precisely to climb the medal ranking. Several authors used different methods of measuring absolute success, i.e. without taking into account socio-economic variables. Some methods will be further explored in the next part.

4In its most simple form, the total number of medals that a country wins serves as the standard measure for absolute success (see for example Hoffmann, Ging & Ramasamy, 2002a; Gärtner, 1989; Grimes, Kelly & Rubin, 1974; Kiviaho & Mäkelä, 1978; Levine, 1974; Novikov & Maximenko, 1972). The argument for using absolute scores is based on the fact that modern competitive sport at an elite level is absolute in nature. Victories and point scores are what counts (Kiviaho & Mäkelä, 1978). A key weakness with this absolute medal table is that it does not account for the “quality” of medals won (gold, silver and bronze) nor other dimensions of “quality” (for example the value of a medal in athletics may be different from a medal in archery). A method which allows for the relative values of gold, silver and bronze medals is a “points” system which makes use of a weighting system to convert a nation’s medal haul into a points equivalent. This methodology is standard practice when analysing Olympic results (see for example Ball, 1972; De Koning & Olieman, 1996; Den Butter & Van der Tak, 1995; Van Bottenburg, 2000). These are usually weighted as follows: a gold medal gets 3 “points” (or 4), silver receives 2 and a bronze just 1. Applying this point system to the Athens 2004 medal table for the top 10 nations means that the United States was the most successful country, followed by Russia and China (see Table 1).

Table 1. Absolute success of the 10 best performing countries in Athens 2004, ranked according to weighted medals.

Table 1. Absolute success of the 10 best performing countries in Athens 2004, ranked according to weighted medals.

Source: IOC, 2004

5Table 1 shows that the ranking positions for the top ten nations is the same for total medals or weighted points. Exceptions can be found if one looks exclusively at the number of gold medals that have been won. For example, ranked on the basis of gold medals, China with 32 gold medals (total medals 63) would be ranked ahead of Russia which won 27 gold medals (total medals 92) (Shibli & Bingham, 2005). There is a difference of 67 medals (103-33) and 155 medal points between the first ranked nations, the United States and the tenth nation (Great-Britain). The United States won 11.6 % of the medal points on a total of 1832. The top ten nations together win more than 50 % of the medals, of which the first three nations allready take 29 %. The remainder 43 % of the medals are divided among 65 nations.

6Whilst medal based measures of performance are easily understood measures of success, they still ignore the totality of achievement of an elite sport programme. As has already been demonstrated, competition for medals is increasing as more nations take part in the Olympic Games. It is quite possible for Performance Directors in individual sports to make considerable progress in developing a sport without this progress being translated into medals in elite competition. Therefore, a number of studies examine the first six (Shaw & Pooley, 1976; Gillis, 1980) or eight places (Condon, Golden & Wasil, 1999; Kiviaho & Mäkelä, 1978; Stamm & Lamprecht, 2001), using a weighting or 6-5-4-3-2-1 or 10-8-6-5-4-3-2-1 respectively. Kuper and Sterken (2003 a & b) also found the number of participants in the Games per country to be an indicator for success.

7When these methods are compared – as a general rule – it does not make much of a difference if and how the number of medals are weighted or not, nor whether the first three or the first eight places are taken into account (see Table 2) (De Bosscher, 2007).

Table 2. Correlations (Pearson) of various indicators of absolute success during the Olympic Games in Athens 2004: total number of medals and gold medals, weighted medals, first 8 places and number of participants (N= 201).

Table 2. Correlations (Pearson) of various indicators of absolute success during the Olympic Games in Athens 2004: total number of medals and gold medals, weighted medals, first 8 places and number of participants (N= 201).

** correlation is significant at 0,01 level

8In Table 2, high (Pearson) correlations can be observed for all methods. This means that, in general, a country that gets a high score using one method will also have a high score using another. Even the number of participants in the Games from each country is significantly correlated to the number of medals won (r>0,8). In other words, the selection for achieving success has already taken place, that is before participation in the Games with norms that are, on the one hand, set by the IOC and, on the other, by each National Olympic Committee (NOC).

9Nevertheless, these correlations are only statistical methods that never give a correlation of 100 %, which means that there will always be exceptions to this generalisation of methods, like in the example of China and Russia given above. Great Britain is more successful than Italy in terms of top eight places whereas Italy won two more medals (of which one was gold).

10Using one Olympic Games may result in coincidental rankings. For example the Netherlands won only 4 gold medals (22 in total) in Athens but they won 12 gold (25 in total) in Sydney 2000. To overcome the problem of success by coincidence, countries must be evaluated over time. Whilst the points system is a more useful measure of performance than position in the medal table or total medals won, it has one major limitation. As the number of events contested at each Games has varied considerably over time and to a lesser extent the number of points per event has also varied (for example two or more nations “tying” for the same medal), the number of points available at each Olympic Games has also varied. In order to convert points won into a standardised measure, SIRC (2002) offers the principle of computing “market share”, that is, points won as a proportion of points available to win. Using market share it is possible to make a more accurate time series diagnosis in standardised terms. Figure 1 illustrates the temporal evolution in the share of medals for a selection of the ten best performing countries (in medal points in Athens 2004) over the period 1988-2004. There is a logic to starting with Seoul in 1988 as this was the first games since 1972 that had not been contaminated by some form of boycott. The market share shows for example that in the 301 events contested in Athens, a total of 1,832 medal points was awarded, of which the United States won 212. This is a market share of 11.6 %, which in turn is slightly more than the 203 points won out of 1,829 (300 events) in Sydney 2000, a market share of 11.1 %.

Figure 1. Market share of ten best performing countries in Athens during the Olympic Games between 1988-2004.

Figure 1. Market share of ten best performing countries in Athens during the Olympic Games between 1988-2004.

11In terms of market share, Figure 1 reveals that only four of the top ten nations (in Athens) have increased their performances from Sydney to Athens: USA (increase of 0.5 %), China (a remarkable increase of 2.1 %), South Korea (0.3 %) and Japan (a remarkable increase of 2.3 %). Over a longer period a steadily increasing trend was noticed in France and Italy until 1996 and in Australia until 2000 with slightly decreasing performances thereafter. A decreasing trend can be observed for Great Britain until 1996 and Korea until 2000; both nations increased their performances thereafter. Although the UK won fewer medals in Sydney (28 medals) than they did in Athens (30 medals), their market share in Sydney was higher (3.28 %). After the unification of East and West Germany in 1989 and the break up of the former Union of Soviet Republics (USSR), the United States is constantly outperforming other nations with a market share above 11 %. Another point that can be observed from Figure 1 is the narrowing gap in market share between the ten best performing nations in Athens 2004 compared to the ten best performing nations 1988 (of which 4 were not in the top ten list in Athens). While the difference in market share between the first and tenth nation was 16,8 % in Seoul (not in figure 1), it is only 8.5 % in Athens. This finding demonstrates how “traditional market leaders” like x, y and z are confronted with an increased competition from new entrants in the Olympic market. This may reflect the more general tendency of increased competitiveness in sports.

12A key question arising from the data presented in Figure 1 and Table 2, is how might nations be expected to perform given the resources at their disposal? In the previous methods, success continues to be defined in “absolute terms”; external (socio-economic) influences are not taken into account. The medal success (output) is not presented in relation to its determinants (input). Bernard and Busse (2004) raise the question of how many medals a country would need in order to be classified as successful. In other words, what can a country expect in terms of medal outputs, given its inputs (resources). This is the definition of relative success (De Bosscher, 2007). We contend that when establishing indicators for national sporting success, one must take the socio-economic and political differences between countries into account.

Measuring relative success of nations

13The aim of this part is to present a method whereby some of the differences between countries on the macro-level, which are of major significance in international success and cannot be influenced by policies, are controlled. This allows us to construct indicators for relative success, i.e. success-indicators that control for exogenous macro-influences. This approach looks at the efficiency of sports systems when the national characteristics of Olympic success are isolated.

14Numerous empirical studies show that population and wealth are the most important socio-economic determinants of success (see, for example, Bernard & Busse, 2004; De Bosscher, De Knop & Heyndels, 2003 a & b; Jokl, 1964; Johnson & Ali, 2002; Kiviaho & Mäkelä, 1978; Levine, 1974; Morton, 2002; Novikov & Maxi­menko, 1972; Suen, 1992; Van Botten­burg, 2000). These two variables frequently explain over 50 % of total medals or medal points. For this reason, success has also sometimes been expressed in terms of medals per head of population or in terms of per capita GDP, as a measurement for wealth.

15It is obvious that the size of a country’s population will be a determining factor for sporting success (De Bosscher, 2007). The bigger the population, the larger the pool from which talent may be recruited and the greater the opportunities to organise training and competitions. Divided by population the Bahamas, with two medals was the most successful country in Athens.

16There are reasonable explanations for the fact that wealthy countries perform better. Richer countries can invest more in sport and elite sport, individuals may participate in a broader number of sports and a higher living standard may improve their general fitness and ability to perform at top level. Den Butter and Van der Tak (1995) have found that the number of medals won correlates strongly with income (GDP) as well as with more general welfare indicators, such as the human development index or the quality of life index. When we divide the number of medal points in Athens by the per Gross Domestic Product per head of the population China becomes the most successful nation.

17Taking into account just one determinant, population size or wealth, is rudimentary in two respects. Firstly, it disregards other potentially important determinants. Secondly, it assumes an implicit linear relationship between these two factors and success. By dividing medal points by population or wealth (criteria that are sometimes used by the media) the degree to which these factors can influence success is not taken into account. This creates a potentially biased view. After all, a country that has twice as many inhabitants cannot win twice as many Olympic medals. This is the principle of decreasing returns to scale (Glejser 2002). This derives from, among others, institutional characteristics mentioned above such as the fact that countries are allowed to send to the Games only a limited number of athletes who have met the IOC criteria.

18To assess whether a particular country or group of countries does “well” at the Olympic Games, the literature offers a number of methods. These take both of the aforementioned criticisms into account: (a) several determinants of success at the same time and (b) possible non-linear effects. Several studies explored the relationship between international sporting success and the (macro) economic, sociological and political context within which sporting talent thrives. Most studies used simple correlations and regression analysis. During the last decade, some authors have tried to improve the methodology of these studies (see for example Baimbridge, 1998; Bernard & Busse, 2004; De Bosscher, De Knop & Heyndels, 2003 a & b; De Koning & Olieman, 1996; Den Butter & Van der Tak, 1995; Johnson & Ali, 2002; Tcha & Perchin, 2003). These studies revealed that aside from the “key” environmental factors for success (wealth and population), which have already been addressed above, the main explanatory factors for success are: area, degree of urbanisation, religion and political system. These variables are the inputs in the production of sporting success that cannot be controlled by sports policies.

19The starting point for our empirical work is a simple OLS (Ordinary Least Squares) estimation of a reduced form model that captures the main macro-determinants of absolute Olympic success. The “outputs” are the weighted number of Olympic medals that a country has won.

Linear regression analysis

20With the Athens 2004 Olympic Games as a point of departure a linear regression analysis (OLS) of socio-economic factors that influence (absolute) sporting success was carried out using the Statistical Package for the Social Sciences (SPSS). Here the logarithm of the weighted medals (gold=3, silver=2, bronze=1) for the Olympic Games in Athens in 2004 are taken as the absolute success measurement.

21Taking into account the aforementioned factors, the functional form to be estimated is as follows:

  • 1 As a point of departure, a correlation-matrix was taken for the selection of these variables and to (...)
  • 2 The source of our data is the World Factbook 2004 (http://www.cIa.gov/cia/ publications/factbook/in (...)

22In the above1, Ln (POP) is the (logarithm of the) number of inhabitants, that were recorded in millions. Ln (GDPCAP) is the (logarithm of the) Gross Domestic Product per head (recorded in US dollars). Ln (DENS) corresponds with (logarithm of the) population density (population/ area). Given that we control for the number of inhabitants, this variable takes the possible influence of the area of a country into account. These three independent variables and the dependent variable have been transformed into logarithms, to account for non-linear effects in the specification. MUSL and PROT are related to the “religious” structure of a country and includes the percentage of Muslims, and Protestants respectively. COMM is a dummy for (former) communist countries. This dummy is equal to 1 for (former) communist countries and 0 for other countries. β1 to β6 are the regression coefficients to be estimated. is the error term for the regression model, which is the unknown variation (the vertical deviation from the unknown true regression line)2.

  • 3 A boxplot with the Mahalanobis distance indicated that Jamaica was an extreme value. After diagnost (...)

23Initial analysis examined the fitting of the model described by the equation above to the entire data set. After diagnostic tests to see if there were any outlying values (with respect to their Y-values and/or their X values) that could influence the appropriateness of the fitted regression function (Kutner, Nachtsheim, Neter et al., 2005), Jamaica was omitted and 74 cases remained in the sample3. In the final model three independent variables were significant: (Ln (population), Ln (GDPCAP) and communism (dummy). The results are shown in table 3.

Table 3. Stepwise Ordinary Least Squares for weighted number of medals (gold= 3, silver= 2, bronze= 1) in Athens 2004.

Table 3. Stepwise Ordinary Least Squares for weighted number of medals (gold= 3, silver= 2, bronze= 1) in Athens 2004.

24

Note  The stepwise estimation showed exactly the same results as a normal “enter” method.

25A stepwise regression (whereby significant explanatory variables are added one by one and deleted one by one), as shown in Table 3 indicates that the population size is responsible for 19.3 % of the international success. Wealth adds another 19.5 % and together with the political system for (former) communist countries, we end up with a model where 52.4 % of the international success is explained.

26This model was used for further analysis of the residuals. The residuals are normally distributed and constant and no systematic pattern is present (homoscedasticity). This was confirmed with a Kolmogorov Smirnov Test for normality, which was not significant (sign.= 0.200) and a Shapiro Wilk (sign.= 0.488). The following section covers in greater depth this residual analysis as a way to determine success of nations controlling for macro-level determinants.

Figure 2. Graphic representation of a linear regression and the policy as part of the residual.

Figure 2. Graphic representation of a linear regression and the policy as part of the residual.

Identifying relative success of nations

27A regression analysis serves two purposes. First it identifies the determinants for international success on the macro-level. Second, under ceteris paribus conditions, an analysis of residuals allows the comparison of countries and thus also the determination of their relative success. The analysis of residuals compares this “prediction” with the weighted number of medals, actually won. Using a “case-by-case” analysis, we can answer the question: “which countries are successful, if one accounts for socio-economic variables ?” Figure 3 illustrates, in a two-dimensional space, (that is for a situation in which we only analyse one explanatory variable), the regression line that is the best fitting line of a topographical point system (Ottoy, Van Vooren & Hughe, 1993). The points in the figure show the positions of the respective countries. The regression line divides the points (countries) into two groups. A successful country is one (above the regression line) that performs better than one would expect on the basis of macro-economic determinants. The degree to which the country performs “better” is reflected in the size of the residual.

Figure 3. Market share in Winter Olympics 1992-2006 of the ten best performing nations (in Turin 2006).

Figure 3. Market share in Winter Olympics 1992-2006 of the ten best performing nations (in Turin 2006).

28Applying this statistical technique to our study, the starting point of our analysis is now that the residual represents (partly) the effects of elite sport policies. In other words, the positive residual of nations is among others the result of effective elite sport policies, which is part of the unexplained variance. It should be noted that some other factors will, to an unknown extent, also influence the size of the residual. Examples are the elite sport culture, the tradition of sport and sporting success in a nation and maybe just “coincidence”. However, these factors cannot be fashioned by policies. Although it is not known to what extent the residual can be explained by elite sport policies, these are the only factors that can be fashioned.

29Table 4 offers an overview of the ten countries with the greatest relative success in Athens, taking the differences in population size, wealth and, where relevant, the (former) communist character of the country into consideration. This ranking reflects the regression residuals [residual = absolute success - predicted success] in the multidimensional model with only the significant variables.

Table 4. Ten best performing nations during the Olympic Games in Athens according to OLS method (relative success), with Log (points of medals) as the dependent variable and Log (pop), Log (GDP/head and communism as independent variables.

Table 4. Ten best performing nations during the Olympic Games in Athens according to OLS method (relative success), with Log (points of medals) as the dependent variable and Log (pop), Log (GDP/head and communism as independent variables.

30Table 4 shows that Cuba was the most successful country in Athens in terms of relative success, followed by Australia, Kenya, Ethiopia and Greece. Kenya’s third and Bahamas’ seventh position is noteworthy. On the basis of the socio-economic context it was anticipated that these nations would win respectively 1.70 and 1.03 times more medals than expected. Moreover, the results showed that the United States is ranked 10th, still with a positive residual (0.93).

31Some rich Western European countries perform less than predicted, for example: Belgium (65th), Sweden (44th), Austria (52nd), Finland (59th), Norway (64th), Switzerland (68th).

32When Table 4 is compared to the top ten of nations in terms of absolute success (Table 1), it can be seen that only three nations are mentioned in both tables: the USA (ranked first in absolute success), Russia (ranked second) and Australia (ranked fourth). Hence, these nations are highly successful when both absolute and relative measurements are used.

Other measurements of Performance: the Olympic Winter Games

33The preceding analysis has focused primarily on the Summer Olympic Games, illustrating different methods using the Athens Games as a case study. The Olympic Games are a truly global event containing a portfolio of different sports that are popular and recognised in a number of nations. Moreover, Olympic performances are seen as the ultimate performance in many sports. In this regard the Olympic Summer Games were indicated to be the best measure of the overall sporting performances of nations. However, some nations may prefer to invest in Winter sports because their geographic environment makes this likely. This was not included in the preceding analysis. To illustrate the point, Canada is particularly successful in speed skating and in Short track whilst Norway is successful in cross country skiing and alpine skiing. These nations do not belong to the top 20 in summer Olympic sports. Thus there is the danger that the success of nations who particularly value success in the Winter Olympic Games is underestimated when only summer Olympic sports are taken into account. We will therefore apply the same analysis for Winter Games as that presented earlier in this paper, using market share as a measurement of success in absolute terms (i.e. without controlling for socio- and macro-economic determinants) and residual analysis as a measurement of relative success. In terms of efficiency of elite sport policies an analysis for winter sports, comparable with the preceding analysis, may help to understand a broader interpretation of success with regard to nations’ elite sport investments. However, the Winter Games are less globalized and have different characteristics compared to the Summer Games and their size is a little less than 20 percent of the Summer Games in terms of participants and events (Kuper & Sterken, 2003b). Only 77 National Olympic Committees participated in the Games in Salt Lake City and of these 25 won medals.

Market share during the Winter Olympic Games

34For the last five Winter Olympic Games, we have replicated the market share figures for the ten best performing nations in Turin 2006 and these are shown in Figure 3.

35From Figure 3 three general conclusions can be drawn. First, the market share of Winter Games for the top ten nations is higher than for Summer Games. This finding may indicate that competition is less in Winter Games and for participating nations it is easier to win medals compared to Summer Games. Second, in Turin the medals have been divided more equally among the top ten nations than in Salt Lake City, resulting in a range of 8.3 % between the first and tenth nation in Turin versus 15.3 % among the top ten nations in Salt Lake City (of which 2 nations differ from the top ten in Turin; not represented in figure). The market share figures for Korea (5.2 %), Switzerland (5.6 %), Sweden (6.0 %) and Norway (6.2 %) are very close to each other in Turin. Third, five nations in figure 3 are not in the top ten list of summer sports: Norway, Sweden, Switzerland, Austria and Canada.

36Figure 3 also shows that Germany has always played a leading role in winter sports, with the exception of Lillehammer 1994 where the Russian federation heads the other nations, despite the break up of the former USSR. Russian performances decreased until Salt Lake City 2002 with a remarkable increase in 2006 (Turin). Germany’s market share decreased with 3.5 % in Turin. Austria begins the time series with a third place in Albertville 1992 and then has a remarkable decrease of 7.3 % in the run up to Lillehammer (1994). Afterwards a steady increasing trend can be noticed with in Turin an equal score to Canada that also improved its market share from 4.1 % in Albertville (1992) to 9.5 % in Turin. The USA was the host nation in Salt Lake City where it improved its market share from a fourth in Albertville to a second in the last two Olympic Games, despite falling market share in Turin. Norway also gains its highest market share in Lillehammer when it hosted the Olympic Games.

Relative success in Winter Olympic Games

37Contrary to the Summer Games only a few authors have estimated success on Winter Games (see Balmer, Nevill & Williams, 2001; Kuper & Sterken, 2003b).

  • 4 We thank Mr Kuper and Mr Sterken for securing this data, obtained from the Encarta encyclopaedia. M (...)

38Making an OLS estimation on one edition of the Winter Games is less plausible than for Summer Games because of the small data set. Without going into as much detail as with Summer Games, an OLS estimation was made for the 25 medal winning nations in Salt Lake City 2002. The same independent parameters as for the Summer Games were entered with one additional variable: mountain elevation for Alpine Skiing (MOUNT). The variable was inserted as a dummy equal to “1” when mountains with a minimum height are available in the country and “0” when they were not4.

39Interestingly, the results revealed that only wealth and communism are significant variables in this estimation for Winter Games. Population is not significant (t= 1.134), nor is mountain elevation (t= 1.094). The latter may partly be explained by the fact that our analysis is confined to medal winning nations. Nevertheless, this finding corresponds with earlier findings of Kuper and Sterken (2003b) who estimated time series models for each Olympic Games from 1924-1998. Apparently, contrary to the Summer Games, a large basis for recruitment of young talent is less important in Winter Games. A logical explanation for the wealth of nations as the main precondition for success may be found in the fact that Winter Sports are generally more expensive sports and require more expensive equipment. Poor nations can send fewer athletes to the Games. The poorest medal winning nations in Salt Lake City, in terms of GDP/CAP (<10.000 $), are former communist nations (Russia, Croatia, Belarus, Bulgaria, Poland and Estonia), which may explain the significant output of communism in the regression.

  • 5 Via Diagnostic tests two outlying cases in X-direction were identified: Germany and the USA, with a (...)

40The estimated function, with only significant variables, resulted in a model with a determination coefficient of 54.6 %5:

  • 6 Shapiro-Wilk : sig. = .475

41The residual analysis was normally distributed6 and collinearity statistics were satisfactory (VIF= 3.184). The position of the ten best performing nations ranked according to their residuals is shown in Table 5.

Table 5. Ten best performing nations during the Olympic Winter Games in Salt Lake City according to OLS method (relative success), with “points of medals” as the dependent variable and “GDP/head and communism” as independent variables.

Table 5. Ten best performing nations during the Olympic Winter Games in Salt Lake City according to OLS method (relative success), with “points of medals” as the dependent variable and “GDP/head and communism” as independent variables.

42The residual analysis shows that Norway was the most successful nation in Salt Lake City. Germany, first in absolute success measurements has become sixth and the USA fourth. Three nations appear in Table 5 that did not belong to the top ten in terms of market share: China (3rd), Croatia (9th) and Spain (10th).

43The analysis thus far has focused on performance in the Olympic Games, which although a truly global event, is not a measure of success in all events. The Olympic medal table is but a snapshot of global sporting prowess at a given point in time and does not necessarily represent global sporting achievement for the duration of each Olympiad. In some sports, success in the Olympic Games is not recognised as the pinnacle of achievement, notably tennis where Grand Slam tournaments are widely regarded as being the pinnacle of achievement; and in football the FIFA World Cup. Therefore, for nations which view themselves as “sporting” nations, there needs to be another measure to contextualise overall sporting achievement. This measure needs to be able to change more frequently than once every four years and also needs to rate recent success more highly than historical success.

Conclusion

44The key purpose of this article has been to demonstrate a methodology by which an objective assessment of performance can be made using techniques that have both an absolute and relative nature. Each of these methods – in their own specific way – allows countries to be classified with respect to their absolute and relative sporting success. One key point noted from this paper is that defining success of nations is a tremendously difficult exercise and from this perspective it can be assumed that ranking nations in terms of their success may raise more questions than answers when it comes to the relation with effectiveness of policies. An objective performance ranking of nations, in order to identify a link between performance and policy, has proven to be an ambitious task because the sporting priorities of individual nations vary along winter or summer sports and because the ranking differs along the purpose of the success measurements.

45If we wish to identify which country is the most efficient (i.e. which country succeeds in achieving sporting success using the sources (at macro-level) available), then differences in population, wealth and other economic, political and sociological circumstances must be considered. Policy determinants are part of the residual and countries that perform better than predicted may thus have more efficient policies in producing medal winning capability. It is self-evident that establishing such criteria is impossible albeit for the simple reason that we are not even aware of all the factors that influence sporting success. Analogous to findings in literature it can be concluded from our methodology that international sporting success, both in summer and winter sports is determined for over 50 % by these determinants. These factors cannot be controlled by politics and this may be an argument to use relative success measurements. On the other hand there are some arguments to support absolute measurement. One is that top level sport is absolute by definition (Van Bottenburg, 2000) and this makes relative success measurements irrelevant. What finally counts is the absolute number of medals won, no matter how rich or large nations are. The best performing nations are then the nations with the highest number of medals. Traditional measures such as medal tables have a function in terms of being easily understood rankings of actual performance. However, as outlined in this paper, it is possible for medal based measures of performance to give conflicting results concerning a nation’s performance. To overcome this problem, market share is identified as being a standardised measure of absolute success which enables meaningful time series analysis to be conducted.

46The key point noted from an application of these two methods (absolute and relative success) to Olympic Summer and Winter Games in this paper is that all these success measurements give different results in terms of ranking nations on the basis of performances. Determining success therefore depends on the purpose for which the ranking is made. We therefore state in this paper that a broad perspective of success measurements must be considered in order to draw any meaningful conclusions regarding the relation between elite sport policies (input and throughput) and international sporting success (output).

47In addition to raising the issues linked to quantifying and ranking the performance of nations, other measurements of success can be discussed. The analysis has focused on performance in only Olympic sports. Some 28 sports and 35 disciplines are contested at the Olympic Summer Games and in many nations over 50 sports are formally recognised by the national agencies of sport. Therefore, the Olympics are but a subset of all sports and inevitably for some nations culturally important sports do not figure in the Olympic programme. Furthermore other success measurements could be introduced, such as the number of top eight places or the number of world level athletes.

  • 7 This was the subject of an international comparative study on elite sport policies that has recentl (...)

48Finally this paper also highlighted an issue that may have an impact on how policies towards elite sport development systems are monitored and evaluated in the future. Competition for success in elite sport is increasing. More nations are adopting strategic approaches towards the development of elite athletes and as a result an increasing number of nations have developed genuine medal winning capability. Therefore a further measure of performance is the efficiency of the production process in terms of input-output analysis and the processes that lead to a certain output7.

Haut de page

Bibliographie

Baimbridge M. (1998), “Outcome uncertainty in sporting competition: The Olympic Games 1896-1996”, Applied Economics Letters, 5, pp. 161-164.

Ball D.W. (1972), “Olympic games competition. Structural correlates of national success”, International Journal of Comparative sociology, 13, pp. 186-200.

Balmer N.J., Nevill A.M. & Williams A.M. (2001), “Home advantage in the Winter Olympics (1908-1998)”, Journal of Sports Sciences, 19, pp. 129-139.

Bernard A. & Busse M. (2004), “Who wins the Olympic Games ? Economic resources and medal totals”, Review of Economics and Statistics, 86, pp. 413-417.

Condon E.M., Golden B.L. & Wasil E.A. (1999), “Predicting the success of nations at the summer Olympics using neural networks”, Computers & operations Research, 26, pp. 1243-1265.

De Bosscher V., De Knop P. & Heyndels B. (2003a), “Comparing relative sporting success among countries : Create equal opportunities in sport”, Journal of Comparative Physical Education and Sport, 3, 3, pp. 109-120.

De Bosscher V., De Knop P. & Heyndels B. (2003b), “Comparing tennis success among countries”, International sport studies, 1, pp. 49-69.

De Bosscher V. (2007), Sports Policy Factors Leading to International Sporting success, Published Doctoral thesis, Vrije Universiteit Brussel, Brussel, VUBPRESS.

De Bosscher V., Bingham J., Shibli S., van Bottenburg M., De Knop P. (2008), A Global sporting arms race. An international comparative study on sports policy factors Leading to international sporting success, Aachen (Germany), Meyer & Meyer.

De Koning J. & Olieman R. (1996), “Twijfel over medaille-euforie [Doubts about medal euphoria] ”, Economische Statistische Berichten, 2, pp. 813 - 815.

Den Butter F.A.G. & Van der Tak C.M. (1995), “Olympic medals as an indicator of social welfare”, Social Indicators Research, 35, pp. 27-37.

Gärtner M. (1989), “Socialist countries’ sporting success before perestroika-and after ? ”, International Review for the Sociology of Sport, 24, 4, pp. 283-297.

Gillis J. (1980), “Olympic success and national religious orientation”, Review of Sport and Leisure, 5, pp. 1-20.

Glejser H. (2002), “Verdere bewijzen over het comparatief nadeel van kleine landen op het gebied van sport [further prove on the comparative disadvantage in sport of small countries] ”, Nieuw Tijdschrift van de Vrije Universiteit Brussel, 15, 1, pp. 82-91.

Green M. & Houlihan B. (2005), Elite sport development. Policy learning and political priorities, London and New York, Routledge.

Grimes A., Kelly W. & Rubin P. (1974), “A socio-economic model of national Olympic performance”, Social science quarterly, 55, pp. 777-783.

Hoffmann R., Ging L.C. & Ramasamy B. (2002a), “Public policy and Olympic success”, Applied Economic Letters, 9, pp. 545-548.

Johnson K.N., & Ali A. (2002), A tale of two seasons : participation and medal counts at the summer and winter Olympic Games, Retrieved February 15, 2003, from Wellesley college, Massachusetts website : http://www.wellesley.edu/economics/wkpapers/wellwp_0010.pdf.

Kiviaho P. & Mäkelä P (1978), “Olympic Success : A sum of non-material and material factors”, International Review of Sport sociology, 2, pp. 5-17.

Kuper G.H. & Sterken E. (2003a), Olympic Participation and Performance Since 1896, Graduate School/Research Institute Systems, Organisations and Management, University of Groningen Report, No. 03C19, Retrieved September 3, 2003, from http://som.eldoc.ub.rug.nl/ reports/themeC/2003/03C19.

Kuper G. & Sterken E. (2003b), The Olympic Winter Games : Participation and Performance, Retrieved April 20, 2006 from Departement economy, University of Groningen : http://www.eco.rug.nl/medewerk/kuper/download/olwinter.pdf.

Kutner M.H., Nachtsheim C.J., Neter J. & Li W. (2005), Applied Linear Statistical Models (5th edition), New York, McGrawHill.

Levine N. (1974), “Why do countries win olympic medals. Some structural correlates of Olympic Games success”, Sociology and Social Research, 58, pp. 353-360.

Morton R.H. (2002), “Who won the Sydney 2000 Olympics ? An allometric approach”, The Statistician, 51, pp. 147-155.

Novikov A.D. & Maximenko A.M. (1972), “The influence of selected socio-economic factors on the level of sports achievements in the various countries”, International Review of Sport sociology, 7, pp. 22-44.

Ottoy L., Van Vooren M. & Hughe V. (1993), Inleiding tot SPSS voor Windows, Ghent, University Press.

Rogge J. (2002), The challenges of the third millennium. Opening speech made at the International Conference on Sports Events and Economic Impact, Copenhagen, Denmark.

Shibli S. (2003), Analysing performance at the Olympic Games : Beyond the final medal table. Paper presented at the 11th Congress of the European Association for Sport Management, Stockholm, Sweden.

Shibli S. & Bingham J. (2005), “An evaluation of medal-based measures of performance in the Summer Olympic Games [Abstract] ”, Proceedings of the 13th Congress of the European Association for Sport Management, Power in sport, Newcastle, England, pp. 246-247.

SIRC (2002), European sporting success. A study of the development of medal winning elites in five European countries. Final Report, Sheffield, Sport Industry Research Centre.

Shaw S. & Pooley J. (1976), “National success at the Olympics : An explanation”, in Lessard C., Massi­cotte J.P. & Leduc E. (Eds.), Proceedings of the 6th international Seminar : History of Physical Education and Sport, Québec, Canada, Trois Rivières, pp. 1-27.

Stamm H. & Lamprecht M. (2001), Sydney 2000, the best games ever ? World Sport and Relationships of Structural Dependency. Paper presented at the 1st World Congress of the Sociology of Sport, Seoul, Korea, Retrieved March, 2002, from http://www.lssfb.ch/download/ISSA_Seoul.pdf.

Suen W. (1992), Men, money and medals : an econometric analysis of the Olympic Games, Discussion Paper from the University of Hong Kong.

Tcha M. & Perchin V. (2003), “Reconsidering performance at the summer Olympics and revealed comparative advantage”, Journal of Sports economics, 4, pp. 216-239.

Van Bottenburg M. (2000), Het topsportklimaat in Nederland [The elite sports climate in the Netherlands], ’s Hertogenbosch, The Netherlands, Diopter-Janssens and van Bottenburg bv.

Haut de page

Notes

1 As a point of departure, a correlation-matrix was taken for the selection of these variables and to avoid problems of multicollinearity. In this respect the percentage of Catholic inhabitants was omitted, due to the high correlation with percentage of Muslims (r = -0.435 ; sign. 0.000) and moreover only a low correlation with medal points. Urbanisation, which was highly correlated with GDP/head (r = 0.605 ; sign. 0.000), was left out for the same reason ; area, which correlates highly with population (r = 0.558 ; sign. 0.000) was replaced by density ( = population/ area) ; the number of protestant inhabitants in a nation correlated significant at the 0.05 level with GDP/capita (r = 0,281) but was included because of the significant correlation with medal points (r = 0,317 ; sign. 0.000).

2 The source of our data is the World Factbook 2004 (http://www.cIa.gov/cia/ publications/factbook/index.html) for the variables of population, wealth, religion and density. The data relating to the former communist countries derives from Encarta (http://encarta.msn.com/related_761572241/Communism.html). The information on the medals comes from the IOC official report 2004, available at: http://www.olympic.org/uk/games/index_uk.asp

3 A boxplot with the Mahalanobis distance indicated that Jamaica was an extreme value. After diagnostic tests, to ascertain how influential this case is in the fitting of the regression function, Jamaica was eliminated. A major reason for discarding an outlier is that "under the least squares method, a fitted line may be pulled disproportionately toward an outlying observation because the sum of squared deviations is minimized" (Kutner et al., 2005, p. 23). Further analysis showed that this was the case with Jamaica, mainly because of its Protestant nature. Indeed, when Jamaica was omitted from the regression model, Protestantism was not a significant variable anymore. Refined measures, to ascertain whether nations are influential, showed that for Jamaica the "DfFITS" (= 0.475) (which measures its influence on the fitted value) was too high and so was the DFBETA value (= 0.4) for the influence on the intercept. Although these measures are not problematic it was decided to omit Jamaica from the sample mainly because of its influence on the regression parameters (Protestant­ism) and thus on the other residuals. Cook’s distance was only 0.117 for Jamaica, the largest but one (after India) but within the required critical values.

4 We thank Mr Kuper and Mr Sterken for securing this data, obtained from the Encarta encyclopaedia. Medal data are obtained from the official IOC report available at: http://www.olympic.org/uk/games/ index_uk.asp

5 Via Diagnostic tests two outlying cases in X-direction were identified: Germany and the USA, with a Mahalanobis distance of 92,000 and 66,000 respectively, far above the other nations. These nations are also the most successful nations in absolute number of medals and repeating the OLS without either, or both, of these two nations lead to a serious decrease of the determination coefficients : 43.0 % (excluding Germany), 37.8 % (excluding USA) and 15.7 % (excluding both). Further analysis revealed that these nations influenced the regression coefficients (DFBeta) and their single fitted Value (DFFits was 8.28 for GER and 7.92 for USA). Cook’s distances, which measures how much the residual of all cases would change if a particular case were excluded, are all right (0.288 for Germany and 0.352 for the USA). It was therefore decided to keep Germany and the USA in the sample. Nonetheless prudence is needed for the interpretation of the coefficients and residuals of these two nations.

6 Shapiro-Wilk : sig. = .475

7 This was the subject of an international comparative study on elite sport policies that has recently been finished and where the methods presented in this paper were used as measurement of outputs in six nations. For more information about this study : see De Bosscher, 2007 and De Bosscher, Bingham, Shibli et al., 2008).

Haut de page

Table des illustrations

Titre Table 1. Absolute success of the 10 best performing countries in Athens 2004, ranked according to weighted medals.
Crédits Source: IOC, 2004
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-1.jpg
Fichier image/jpeg, 88k
Titre Table 2. Correlations (Pearson) of various indicators of absolute success during the Olympic Games in Athens 2004: total number of medals and gold medals, weighted medals, first 8 places and number of participants (N= 201).
Légende ** correlation is significant at 0,01 level
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-2.jpg
Fichier image/jpeg, 100k
Titre Figure 1. Market share of ten best performing countries in Athens during the Olympic Games between 1988-2004.
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-3.jpg
Fichier image/jpeg, 228k
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-4.jpg
Fichier image/jpeg, 228k
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-5.jpg
Fichier image/jpeg, 40k
Titre Table 3. Stepwise Ordinary Least Squares for weighted number of medals (gold= 3, silver= 2, bronze= 1) in Athens 2004.
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-6.jpg
Fichier image/jpeg, 88k
Titre Figure 2. Graphic representation of a linear regression and the policy as part of the residual.
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-7.jpg
Fichier image/jpeg, 52k
Titre Figure 3. Market share in Winter Olympics 1992-2006 of the ten best performing nations (in Turin 2006).
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-8.jpg
Fichier image/jpeg, 212k
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-9.jpg
Fichier image/jpeg, 216k
Titre Table 4. Ten best performing nations during the Olympic Games in Athens according to OLS method (relative success), with Log (points of medals) as the dependent variable and Log (pop), Log (GDP/head and communism as independent variables.
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-10.jpg
Fichier image/jpeg, 76k
Titre Table 5. Ten best performing nations during the Olympic Winter Games in Salt Lake City according to OLS method (relative success), with “points of medals” as the dependent variable and “GDP/head and communism” as independent variables.
URL http://journals.openedition.org/belgeo/docannexe/image/10303/img-11.jpg
Fichier image/jpeg, 76k
Haut de page

Pour citer cet article

Référence papier

Veerle De Bosscher, Bruno Heyndels, Paul De Knop, Maarten van Bottenburg et Simon Shibli, « The paradox of measuring success of nations in elite sport »Belgeo, 2 | 2008, 217-234.

Référence électronique

Veerle De Bosscher, Bruno Heyndels, Paul De Knop, Maarten van Bottenburg et Simon Shibli, « The paradox of measuring success of nations in elite sport »Belgeo [En ligne], 2 | 2008, mis en ligne le 20 octobre 2013, consulté le 29 mars 2024. URL : http://journals.openedition.org/belgeo/10303 ; DOI : https://doi.org/10.4000/belgeo.10303

Haut de page

Auteurs

Veerle De Bosscher

Vrije Universiteit Brussel, Faculty of Physical Education and Physiotherapy, Veerle.de.bosscher@vub.ac.be

Bruno Heyndels

Vrije Universiteit Brussel, Bruno.Heyndels@vub.ac.be

Paul De Knop

Vrije Universiteit Brussel, pdknop@vub.ac.be

Maarten van Bottenburg

Utrecht University, M.vanbottenburg@uu.nl

Simon Shibli

Sheffield Hallam University, s.shibli@shu.ac.uk

Haut de page

Droits d’auteur

CC-BY-4.0

Le texte seul est utilisable sous licence CC BY 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search