UvA-DARE (Digital Academic Repository) Which are the "best" cities for psychology research worldwide?

We present scientometric results about worldwide centers of excellence in psychology. Based on Web of Science data, scientific excellence can be identified for cities from where highly-cited papers originate. Data refer to all psychology articles published in 2007 which are documented in the Social Science Citation Index and to their citation frequencies from 2007 to May 2011. 218 cities are visualized with an article output of at least 50 in 2007. Statistical z tests are used for the evaluation of the degree to which an observed number of top-cited papers (top-10%) for a city differs from the number expected on the basis of randomness in the selection of papers. The map points at excellence centers in cities at the East and West Coast of the USA as well as in Great Britain, Germany, the Netherlands, Ireland, Belgium, Sweden, Finland, Australia, and Taiwan. These results indicate that highly-cited psychological research articles come from the Anglo-American countries and some of the non-English European countries in which the number of English-language publications has increased during the last decades. Implications of the results for the publication strategies of psychologists in non-English speaking countries are discussed as well as the neccessity to care for qualitative criteria in evaluations in addition to quantitative, scientometric criteria.


Introduction
Recently, Loet Leydesdorff, Lutz Bornmann and co-authors have developed new approaches for the spatial visualization of concentrations of highly-cited papers using overlays to Google Maps. Bornmann, Leydesdorff, Walch-Solimena, and Ettl (2011) provided methods to map field-specific centers of excellence around the world using bibliometric data. These methods identify and agglomerate excellence in cities where highly-cited papers were published. Using colors and sizes for the marks, differences among cities in terms of performance rates can be visualized on maps. Bornmann and Waltman (2011) extended these methods and used a new approach with density maps for a spatial examination revealing regions of excellence. In contrast to  who focus on cities, this approach is intended to visualize broader regions where highly-cited papers were published.
A further step in the development of these spatial visualization methods is described in . This most recent method does not only consider the quantitative numbers of highly-cited papers, but the observed number of highly-cited papers for a city is tested statistically against the expected number of highly-cited Europe's Journal of Psychology ejop.psychopen.eu | 1841-0413 papers. For example, if authors located in a single city have published 1,000 papers, one would expect for statistical reasons that approximately 100 (that is, 10%) would also belong to the top-10% most-highly cited papers. An observed number of 70 highly-cited papers for this city may seem as a large number compared to other cities, but the specification of the expectation above changes this appreciation. This approach has drawn considerable attention in science journalism. 1 The application of scientometric methods outside the physical and life sciences has sometimes been considered questionable (Moed, 2005). As Abramo and D'Angelo (2011) formulated: "The use of such measures is still limited to the natural and formal sciences, where publications in international journals and conference proceedings are the most accepted form for the diffusion of research outputs, and where the publications therefore represent a trustworthy proxy of research outputs ... In the arts and humanities and most of the social sciences, bibliometric indicators are considered not yet sufficiently robust to inform peer-review" (p. 500). Although psychology does not belong to the "hard-science" disciplines scientometric methods have increasingly been applied for evaluative purposes within the discipline (see, e.g., Endler, Rushton, & Roediger, 1978;Krampen, 2008;Nosek et al., 2010). According to the results of Mahdi, D'Este, and Neely (2008) "citation counts are a reasonable proxy for peer assessment in some subjects, such as … Psychology" (p. 3). Publication and citation behavior in psychology converges at least in some of its sub-disciplines (e.g., experimental psychology, bio-and neuropsychology, clinical psychology, social psychology) with the physical and life sciences. This resulted during the last decades in increased publishing in English-language and internationally peer-reviewed journals (see, e.g., Krampen, Huckert, & Schui, 2012;Krampen & Schui, 2011). We return to this topic in the results section below.
In this study, we provide a map for cities using all 2007 articles contained in the relevant so-called subject categories of the Social Science Citation Index (Web of Science, WoS, Thomson Reuters). Although  used psychology data as an example the present study is more systematic, because these authors based their psychology map only on a random sample of all publications from one year, while in the present study all publications (with the document type "article") are included. Furthermore, one of us as an expert for psychology documentation and information checked the plausibility of the scientometric results.

Procedure for Data Collection
The procedure to map the cities of the authors having published the top-10% most-highly-cited papers in a certain field is described in detail in . In the following, we give a short overview of this method.
The share of top-10% most highly cited publications in their subject category (publication year, document type) has become a de facto standard for the assessment of excellence at the institutional level (Bornmann, de Moya Anegón, & Leydesdorff, 2012;Bornmann, Mutz, Marx, Schier, & Daniel, 2011;Leydesdorff, Bornmann, Mutz, & Opthof, 2011;Tijssen & van Leeuwen, 2006;Tijssen, Visser, & van Leeuwen, 2002;Waltman et al., 2012). In this study we follow this classification and focus on the top-10% of papers published In a first step, all papers (n = 21,528) with the document type "article" published in 2007 and belonging to the subject categories "psychology," "psychology, applied," "psychology, biological," "psychology, clinical," "psychology, developmental," "psychology, educational," "psychology, experimental," "psychology, mathematical," "psychology, multidisciplinary," "psychology, psychoanalysis," and "psychology, social" were downloaded from the WoS (Social Science Citation Index). We restricted the search to articles (as document types) since (1) the method proposed here is intended to identify excellence at the research front and (2) different document types have different expected citation rates, possibly resulting in non-comparable datasets.
After running in subsequent steps various programs which can be copied freely, 2 the output file can be uploaded to the webpage of the GPS Visualizer 3 in order to visualize the content. The file contains the city entries from the WoS data. The counting of occurrences in this study (so-called "integer counting") follows the procedure of how author addresses on publications are gathered by Thomson Reuters for inclusion in the WoS. If co-authors are provided with an identical city-address on a publication, this leads to a single city occurrence in the output. If the co-authors are affiliated with different cities, the different city names are all used equally in the programs. In a further file, the city entries from the WoS data are organized so that aggregated city occurrences can be visualized on a map, that is, provided with latitudes and longitudes (source of the coordinates: Google).
For the maps presented below we zoomed in on Europe and North America. Other regional foci can be studied using the Google Map at http://www.leydesdorff.net/psychology/cities.htm. One can inspect the ratio between observed and expected numbers of excellent papers for a specific city by clicking on the respective city node.
Since the underlying data of a map from WoS (bibliographic data) and Google (geocoding coordinates) 4 are error-prone , we decided to visualize only cities (n = 218) with an article output in 2007 of at least 50. There is a danger for cities in the data with a small number of papers that they result from private addresses of researchers or addresses of hospitals (but not from psychology departments) and that they receive wrong coordinates from Google. Furthermore, the use of a threshold of 50 provides us with a minimum of five papers expected in the top-10% and thus confirms with a requirement of using the statistical procedure described in the following.
For these 218 cities we could check the validity of the geocoding coordinates. In the case of (systemic) errors still on the maps, however, we appreciate and will respond to feedback.

Statistical Procedure
The z test for two independent proportions (Sheskin, 2007, pp. 637-643) is used for evaluating the degree to which an observed number of top-cited papers for a city differs from the value that would be expected on the basis of randomness in the selection of papers . z has a positive value (sign) if the observed number of top papers is larger than the expected number and the z-values are negative in the reverse case. An absolute value of z larger than 1.96 indicates statistical significance at the five percent level (p < .05) for the difference between observed and expected numbers of top-cited papers (marked with an asterisk *). In other words, the authors located in that city are outperformers with respect to scientific excellence in terms of this statistics. Due to the large number of city tests being conducted (n = 218), especially highly significant p values (p < .01) should be considered as significant and interpreted (marked with at least two asterisks **; *** will indicate p < .001).
Europe's Journal of Psychology 2012, Vol. 8(4), 535-546 doi:10.5964/ejop.v8i4.482 Using this statistical test, we designed the city circles which are visualized on the map using different colours and sizes. The radii of the circles are calculated by using: | observed value -expected value | + 1 The "+1" must prevent the circles from disappearing if the observed ratio is equal to the expected one. Furthermore, the circles are coloured green if the observed values are larger than the expected values. We use dark green if z is statistically significant; light green indicates a positive, but statistically non-significant result. In the reverse case that the observed values are smaller than the expected values the circles are red or orange, respectively.
They are red if the observed value is significantly smaller than the expected value and orange-red if the difference is statistically non-significant. If the expected value equals the observed value a city node is coloured grey. If one clicks on a circle, 5 a frame opens showing the number of observed versus expected values for the respective city, as well as an asterisk indicating whether the difference between the values is statistically significant or not.

Locations of Authors Having Published Highly-Cited Papers
In Figure 1, for example, London is indicated by a very large dark green circle-the largest green circle in Europe-because of an observed value much larger than expected. In the description in a frame, the large and statistically highly significant difference between the observed (n o = 194) and the expected value (n e = 100.9) can be retrieved. Further large green circles on the map with a statistically highly significant difference (p < .01) between The reverse case that the observed value is significantly smaller than the expected citation value is identified for no European city. However, there are many more cities which must be described as highly productive (i.e.,   in the USA is supported further by the large number of light green circles, only a very few orange-red circles, and no red circles in Figure 2. citations.

The Anglicization of Psychology
The results show impressively that most research results are published and cited within the Anglo-American countries: Most of the best cities for psychology research worldwide are located in the USA and Great Britain. This is completed only by some cities in the Netherlands, Israel, Australia, the Scandinavian, and the German-speaking countries. This can be considered as an Anglo-American agglomeration effect which we would like to discuss in more detail.
Anglicization of the sciences occurred rapidly in the natural sciences, particularly in areas of discovering facts and natural laws, which are not or-at least-only weakly and indirectly dependent on culture and socialization, i.e., language, nationality, politics, etc. The anglicization of the arts and humanities occurs more gradually and less extensively because of its direct dependency and relationships to culture and socialization, e.g., national and cultural specifics of educational, legal and health systems, social norms, traditions, etc. Somewhat in-between the faster moving natural sciences and the slower arts and humanities are the social sciences and, especially, psychology, because its research topics (i.e., behavior and experience or-more contemporary-action and cognition) must be analyzed from both a natural science and a humanities perspective (see, e.g., Krampen et al., 2012). For example, the psychobiological and neuropsychological determinants and correlates of behavior and experience are under psychological study as well as the social, cultural and psychodynamic aspects of action and cognition. Thus, psychological research can incorporate the natural science methodology as well as the (often more qualitative) methodology of the arts and humanities which includes the methods (e.g., statistics) of the social sciences.
Within the methodological and theoretical developments in the history of psychology, the American Psychological Association (APA) is the major worldwide player since the 1940s, which propagates its research orientation and methodology as well as standardization of "Psychological Index Terms" (Gallagher Tuleya, 2007) and referencing.
As O'Connell and Kowal (2009) formulated: "What is abundantly clear is that the APA has become an empire.
Whether it and psychology throughout the modern world -developmentally not entirely independent of the APA -have become more scientific over the decades since that famous, arbitrary data, 1879, remains an open question" (p. 73; remark: 1879 is the year in which Wilhelm Wundt established the first experimental psychology laboratory at the University of Leipzig, Germany). This is confirmed by the author affiliations of the psychology publications documented in the APA-database PsycINFO (Krampen, 2009): About 85% of the authors' affiliations refer to Anglo-American countries (about 65% USA, 12% Great Britain, 8% Australia and New Zealand), from "the rest of the world" are only 15% (5% German-speaking countries, 3% Netherlands, about 2% each France, Spain, Italy, and 1.5% China). These numbers reflect the very strong "Publish or Perish" culture in U.S. academia. To advance on any level, or to simply retain a job, an academic in the U.S. must publish and publish prolifically -this is to a lesser extent also the case in Great Britain. So, while some researchers may spend more time researching, analyzing, and writing an important piece of work, it can be argued that a U.S.-based researcher has to focus on quality but also on quantity.

stated in the leading American
Psychologist that "psychological research published in APA journals focuses too narrowly on Americans, who comprise less than 5% of the world population. The result is an understanding of psychology that is incomplete and does not adequately represent humanity" (p. 602). He concludes that "the narrowness of American psychological research … focus on a philosophy of science that emphasizes fundamental processes and ignores or strips away cultural context." Arnett's suggestions for broadening the scope of American psychology are positively discussed in some comments (LoSchiavo & Shatz, 2009;Webster, Nichols, & Schember, 2009). Arnett (2009) explains in his reply that the neglected 95% are a challenge not only to psychology's empirical research, but -much more far reaching -to its philosophy of science, epistemology, and methodology too. This is a plea for an (re-)opening For the sake of optimizing the international visibility, reception, and impact of the results of psychological research in the non-English-speaking countries (and cities), fierce discussions in the research communities have taken place. One conclusion was that researchers should attempt to publish their empirical findings more frequently in international English-language journals and less frequently in national, non-English journals. In the recent past, similar discussions are taking place in many larger non-English psychology research communities, such as France, Germany, Italy, and Spain. This is confirmed by bibliometric analyses of the publication languages of the documents in PsycINFO (see Krampen, 2009): The huge majority of 95% is published in English (or -more exactly -in an Anglo-American science idiom), only 1.2% in German, 1.1% in French, 0.7 in Spanish, 0.6% in Chinese, 0.5% in Portuguese, 0.4% in Italian, 0.3% in Japanese, and 0.1% in Russian. All other languages of the world are less than 0.01%. Results on the anglicization of psychology publications from the German-speaking countries (see Krampen & Schui, 2011) point out that publishing in English is highest for basic research in experimental psychology and bio-/neuropsychology (about 85%), medium in social, developmental and differential psychology (about 20-30%), and lowest in applied psychology (about 10-15% each in clinical, industrial and educational psychology).
Science and research are a priori neither mono-cultural (i.e. Western) nor mono-national (i.e., US-American) undertakings. Science and research are multi-cultural and multi-national undertakings, admitting cultural and national specifics as well as cultural and international universals. Therefore, psychologists in the non-English countries (and these in the Anglo-American countries too) should be motivated to do research within their culture and country in search for the psychological specifics of culture and language as well as in search for the psychological universals and cross-cultural differences. Both of which are of intra-cultural and inter-cultural interest, because cultural specifics (emics) and cross-cultural differences and universals (etics) are significant topics of psychology and anthropology (Berry, Poortinga, Segall, & Dasen, 2002;Pike, 1967).
The results of multiple (many one-cultural and multi-cultural, respectively; "emic") research and the results of (cross-cultural; "etic") research should be published in national and international science communities in various languages (but all with abstracts, keywords, phrases, titles in English translation) and in English as well. Under the assumption that the culture of psychology is an open scientific culture, which is interested in advances of scientific knowledge, this publication strategy attracts more and more international attention. This publication strategy is supported by European undertakings promoting high standard digital open access journals and publication platforms (see, e.g., Krampen, 2009 than can be statistically expected. Since these cities do not necessarily have a high output of highly-cited papers, our approach normalizes for size. Despite the advantages of our approach of mapping observed versus expected numbers, we recognize the limitations inherent to these bibliometric exercises (see  (1) It is not always the case that the addresses listed on the publication reflect the locations where the reported research was conducted.
(2) Our method does not consider the different positions of authors on a paper. A paper is counted for a city if at least one co-author is located there.
(3) If the same author has a double affiliation then both cities will get credit for the same piece of work of the author. This might lead to inflating the citation impact of certain cities. (4) Citation counts (and thus the categorization of publications as highly cited) are a function of many variables in addition to scientific quality (Bornmann & Daniel, 2008). (5) We present a spotlight on all psychology papers published in 2007 which are documented in the Social Science Citation Index covering many, but not all psychology journals while maintaining its focus on English-language publications.
(6) Indicators of scientific output and impact -as used in this study -do not tell the whole story in research evaluation (Bornmann, 2011(Bornmann, , 2012: Scientometrics is not the attempt "to measure the un-measurable, but a necessary but not at all sufficient quantitative tool for the difficult task of gathering data on scientific performances" (Krampen, 2011). Advantages of scientometrics are its transparency and the reduction of subjective evaluation biases in peer reviews; its disadvantages are, e.g., the incompleteness of and mistakes (e.g., misspellings) in databases. The whole story (named as an informed peer review) should include qualitative criteria (e.g., peer evaluation and review, significance of results for society and application, etc.) and quantitative criteria (besides bibliometric indicators, e.g., funding of research, number of academic vocations, invited lectures). However, the whole story about all cities worldwide which are relevant in psychological research cannot be presented on a map -this is only possible with bibliometric indicators.