Maja Breznik, Rastko Močnik
Research Policies in Practice: the Social Sciences and the Humanities
(“Themes-oriented” application call, Slovenian Research Agency, 2005)
Accelerated transformations of scientific practices do not offer much occasion for serious reflection. Educational and research priorities are defined in a process to which people from the field have little access.1 The demand for economic impact of science and commercial success of scientific practices, moreover, limits the operational scope of sciences and block reflection on scientific practices and their effects upon society. In social sciences and the humanities that may have little impact on commercial performance, the reconsideration of scientific practices is, roughly speaking, held back. This seems particularly true about the situation of social sciences and the humanities in the South-East European region. There are at least two main reasons for this.
The first reason is general and banal. The humanities and social sciences remain in the shadow of natural and technical sciences. Societies that identify economic growth as the most important impetus for social development, appreciate the scientific endeavour mostly as far as it produces innovations that can swiftly be transformed into profitable market products. Consequently, it is the type of local economy that decides which kind of science will mostly be needed and, accordingly, appreciated. Social sciences and the humanities will find fields of economically profitable application in economies that are intensely concerned with the conception of new products and their launching, i.e., with the two extremes of the production chain. Such are the economies in the centre of the world-system.2 However, peripheral economies, such as those of the S-E Europe, mostly engage in the intermediate stages of the production chain, i.e., in the material production. There, social sciences and the humanities do not have much application. To the best, they can contribute to better management and to the governance techniques: however, this knowledge is now massively imported from the core-countries in the pre-packed form of know-how that does not need additional research, or simply imposed by the patterns of organisation required by the transnational partners and/or owners. Local social and human science is in this way largely considered as not being useful, and is excluded from the fields where it could eventually find specific applications. The competitiveness of the humanities and social sciences in the local S-E European “research markets” is considerably limited, and their reputation is proportionally low. Consequently, the humanities and social sciences remain “silent” in proportion to their “fruitlessness”.
The second reason is even more particularly valid for the South-Eastern European region. These societies were politically defeated along with their hegemonic social engineering after the fall of communism. Social scientists in these countries are, consciously or not, placed in the position of the “vanquished” and are compelled to give the “conquerors” a due recognition. Consequently, social scientists from the region promptly assume hegemonic Western theoretical (or "theoretical") models, fashionable jargons, methodologies, and scientific practices to transplant them into the local social contexts without due reconsideration. Uncritical westernization of their research fields does not really equip them for self-reflective criticism.3
The question of "defeat" certainly gets more complicated if we consider the high quality of theoretical production in the region before the change of socio-economic systems.4 There has been no inherent theoretical reason for the demise of this production during the nineties. Quite the contrary: theory and research were strong and productive during the period from the sixties to, and through, the eighties, they were critically intervening into the local social and political life and often produced important political effects5, they were entertaining a sustained dialogue with the contemporary production in the world, and developed a scholarly dialectic of their own (internal debates, generational conflicts, "schools" and deviations, splits and mergers, heresies and orthodoxies etc.). However, these trends in theoretical production and research never succeeded to gain much institutional support. They were tolerated at universities and institutes, often reduced towards the margins of their institutional constructions, and from time to time exposed to purges.6 Their presence in other cultural apparatuses (publishing, journals, the media) was weak; however, public impact of theoretical production in social sciences and the humanities, and its general importance was increasing during the eighties, especially with the affirmation of the alternative cultures and the alternative politics (the "new social movements"). With the change of the system, institutional weakness of theoretical production became fatal: it was the institutionally established parts of research and scholarship, the mainstream that easily survived the changes and adapted to the new context – basically carrying on its old social role, albeit under the new conditions. The development in the field of theoretical production was somewhat parallel to the development in the political field: in the political sphere as well, it was the "mainstream" that successfully survived the transformation of the system, re-articulated itself to follow the transformation of the mono-party to the multi-party organisation of the field, changed the ideological discourse, but basically carried on the old tasks of political monopoly, domination and control. In the political sphere, too, it was the new social movements and other alternative trends that were the victims to the "transition": marginalised and institutionally weak under the previous arrangements, they were successfully liquidated by the mainstream actors, who could use their well-established position under the previous system in order to secure the monopoly of political power within the new context.7 These drastic discontinuities in theoretical production in social sciences and the humanities that had no "immanent" cause in theoretical work itself, but were the effect of institutional arrangements, power-relations and other "extraneous" conditions in a wider context of deep social transformation, remind us how decisive is the impact of seemingly "heteronymous" factors upon the development of science and research.
This paper is going to investigate contemporary research policies and practices in Slovenia. The study will take under scrutiny legal documents, administrative regulations, project-financed programmes, general rules for the application call in 2005 and all research proposals submitted to this call. We will here focus upon the social sciences only, since the field of the humanities will be analysed in a separate study. Both studies will hopefully provide some material for serious future reflection. Our investigation has been limited by its financial frame and the time schedule. Hopefully, our studies will at least point to some general trends and will encourage future more profound examination of the nature of changes in social sciences and the humanities and of their effects upon society.
* * *
Within the general frame shortly outlined above, we investigated the application call for the social sciences and the humanities in 2005 organised by the Slovenian Research Agency8. This call was the first themes-oriented basic and applied research contest in Slovenia. Additionally, we analysed Slovene and European legal and administrative documents in order to reconstruct the inherent logic of the policies and the argumentation strategies of the involved actors. We wanted to find out the general conditions of research as they were designed for the application call in 2005. We also wanted to include the response of the scientific community to the new research conditions. We accordingly analysed all the project proposals submitted, those selected and those rejected. We wanted to know, on one side, what were the selection criteria that were used in evaluation of research proposals. On the other side, we wanted to asses the epistemological approaches developed in the projects, and also the eventual “adaptation” strategies that were adopted by research teams and communities.
We took under closer examination the application call for themes-oriented basic, applied and post-doctoral research in 2005.9 Slovenian Research Agency kindly delivered us the essential content-defining parts of all the applications in the social and human sciences.10 For each proposal, we had access to texts related to the fractions “General starting-points” and “Summary”. We had on our disposal anonymous descriptions of the projects without the names of the applying researchers and without the indication of their institutional affiliation. We organised the material by ranking the submitted proposals according to the sum-total of evaluation points they had been awarded, and ascribed them ordinary numbers that indicated their rank.
The creation of a thematic-oriented research programme was a novelty introduced that year; the evaluation procedure was also designed in a new way that distinguished this call from previous evaluation and selection arrangements. The “Resolution on national research and development programme” that was adopted later, is presenting this evaluation procedure as a novelty: however, it was already used in 2005, precisely in the competition that was the object of our analysis. The document claims that the evaluation system shall be designed after “the model of leading European countries, and will guarantee the highest possible objectivity of evaluation”11. The evaluation process was in fact in a great part designed after the European Commission’s “Guidelines on proposal evaluation and selection procedures” for the Sixth Framework Programme.
Evaluators examined proposals individually and awarded them evaluation points from 1 to 5, according to the particular criteria defined in advance. Additionally, there were certain thresholds in the procedure. The evaluation was successively performed by two independent groups of evaluators, the first one international and the second one domestic. After the first round of evaluation (performed by international experts), the twenty percent of proposals that had obtained the lowest number of points were removed from further evaluation; the projects that, in each of the categories of the call, had obtained the highest number of points, were automatically selected (without being submitted to further evaluation). In the next phase, the evaluation was conducted by Slovene Council of Science. The Council appointed one domestic expert for each field to evaluate the rest of the projects. When the reviews were completed, members of the Council, each for her or his own field, made their lists of project proposals that they recommended for financing. Besides the evaluation points, they had to take into consideration other criteria: quantitative indicators of researchers (like citation indexes, publications, patents…), the estimation of research group’s capability to carry out the proposed project. They could rely on their own estimation as well. When the lists were ready, the Council of Science discussed the proposals and took final decisions about the financing of projects.12
The evaluation procedure in 2005 was therefore already designed after the model of “leading European countries” (i.e., the European Commission). The novelty was also that it involved foreign experts in the first phase of evaluation. The following analysis is going to provide a view into the results of such an evaluation and selection of research projects.
In the field of social sciences, the result of this complicated procedure was rather surprising: it promoted an expertise to the top of the best graded applications. Among all the various research project proposals (basic, applied and post-doctoral), it was an expertise (see our definitions in the grey square) that was awarded the highest number of points. Evaluators graded best a research project that proposed to survey the efficiency of a governmental educational reform. The proposal openly admitted that the team intends to use as main research data the already available international and national surveys in some of which the applicant had participated. Consequently, the proposal was most likely a by-product of the applicant’s regular activities.
EXPERTISE: Study intended to monitor the effects of the state (governmental) measures or of the state policies, using either empirical methods or juridical comparative analysis or some other approach. It maintains the appearance of scientific distance and methodological strictness. Such studies are usually commissioned by the state (government) administration in order to support their operations.
POLICY STUDY: Research paper that, like an expertise, monitors the efficiency of a particular state policy. Contrary to the expertise, it openly admits its bias and its participation to the practices like lobbying, endeavours to introduce new practices or of new strategies into the operations of the state administration, and the like. On the organisational level, it corresponds to the output of “think tanks” (e.g., Institute for Policy Study, Washington). In European context, it corresponds to the research scheme EQUAL.
PURE BASIC RESEARCH: Research that introduces a new conceptual and theoretical paradigm or produces new knowledge without predetermined results. (Not all of the projects classified as “basic” in the 2005 competition satisfy this strict definition. We were charitable and included also projects producing “new knowledge” or providing “new information”.)
APPLICATION: Problem solving research with intention to produce a concrete product or service for industry or service sector (including state administration) using experimental methods or some other approach.
STRATEGIC HYBRID: Research that combines several approaches in order to maximize the chance to win the competition as, for instance, combining basic research with policy study.
But the privilege accorded to a mere evaluation expertise is only one of the problems opened by the top-graded project. The applicant justified the proposal with two main arguments. The first introduces an international survey as an infallible source of information. The data processed in such a research are presented as “mere facts”, as the outer reality itself.
The second argument is built upon the first one. If one assumes that a particular international ranking of national achievements is trustworthy (as to the definition of an "achievement" and as to the reliability of its measuring), than one can proceed to evaluate whether the position of Slovenia on such a list is "good" or "bad". However, in order correctly to perform such an evaluation, one has to develop a measure of the relative "success" on this particular list. The application does not enter into these details, but bluntly states that the position of Slovenia below the highest ranks means that “Slovenians lag behind the best countries”. The main focus of research proposal actually is the supposedly bad ranging of Slovenia. The application interprets the ranking as a threat that “Slovenians lag behind the developed countries” and that “Slovenian pupils are backward in comparison to others”. The definition of the “good performance” in education is not discussed. Were the proposal really trying to identify what a “good performance” might be, it would need to deconstruct the premises, the methodology and the “object of research” of the international research to which it wants to refer. Without such an effort, it takes the results of the international survey as direct indicators of mere objectivity. With this argumentation procedure, the applicants first created a “problem” and then proposed their project as a step towards its “solution”.
The very top rank of such a proposal suggests that the evaluators (and perhaps the organisers of the competition) did not consider new knowledge and questioning of theoretical paradigms as the most appreciated achievements. Their attitudes towards research and scientific practices obviously emerged from other premises. From the example above we can surmise that it was “problem solving” that interested them the most. They understood the call as giving a chance to the applicants to identify, within the frame of the proposed "themes", particular (disturbing) social problems and to propose ways how to solve them. Put into such a context, research risks to be drawn out of the field of scientific practices and to confront "problems" that have not been theoretically, i.e., conceptually defined, but have been intuitively, i.e., ideologically determined. Problems so defined belong to practices other than theoretical practices. Theoretical concepts are then applied to them from the "outside", instrumentally – and not as procedures of genuine analysis within the field of theory. In other words, in this way, research is drawn towards technological practices, i. e., towards the techno-sciences.13
While scientific practices have to keep in their horizon past scientific achievements and the insertion of a particular problem into larger structures (of science, society, civilisation…), in techno-sciences, these concerns are not only unnecessary, but troubling and undesirable. Scientific practices have to sustain a critical approach towards the past scientific achievements and towards their present presuppositions. Contrary to these theoretical attitudes, techno-sciences are meant to grasp an isolated particular and immediate problem only, and to develop instruments how to solve it, control it or predict its impact and effects. If we transfer the questions that are usual and expected within the sciences into the preoccupations of the techno-sciences, then we risk invalidating their instrumental efforts. For this reason, applied research actually has to avoid properly scientific approaches: they may reveal that applied research is something impossible to perform. In the above example, how we can reach a “good performance” in education, if we do not know what a “good performance” should be; or if we do not ask the radical question about what is the "education" in contemporary society?
Moreover, in scientific work, it is a necessity that scientists position their endeavours into a more general horizon of “knowledge”, “progress”, “truth” or the like, since such a general horizon validates their work, puts limits to it and identifies the objectives to be reached. The little that a particular scientist can contribute to science (with regard to its past and its future), can only get a meaning if this overall context is identified.14 It gives a sense to the work of the particular scientist and determines the conditions that mould scientific communities. It determines what goals they serve, how they are organised, how scientists communicate, co-operate and eventually compete.
For this reason, and since science actually has no master, scientists have to overview and to control the conditions of the production of knowledge. Techno-scientists, to the contrary, have to take their conditions of production for granted and to accept them as their administrative counterparts have programmed them. They offer skills how to solve a certain problem and, after completing their task, the result of their work is taken away from them. It is not a part of their practice any more. They might continue to research the same subject-matter or, as a flexible and skilled person, shift to the solving of some new problem. They are not supposed to entertain any “intimate” relationship with the object of their research; and even when they feel certain affection, it is in their interest, in the interest of techno-scientist, to hide it. In the technocratic world any other consideration besides doing the “job” is taken as a passion that endangers good performance and objectivity. “Objectivity” actually is the key word in the technocratic research.
Given that the difference between scientific practice and technocratic practice is so fundamental and definite, we would assume that scientists will struggle for the basic research. The investigated application call of 2005 was the only call that year that offered opportunity for basic research. Accordingly, one would expect that most of applications asked for basic research. In fact, this was not the case; according to our classification above (see the grey square), we identified:
Only one fifth of all applications proposed to do basic research, while the others proposed applications, expertises, policy studies and so on. The outcome is somewhat understandable given that only few basic research projects were offered and, additionally, they had to correspond to a certain “theme”. At the end, 18 research projects were selected from among 62 applications submitted. Projects were selected proportionally according to the classification of the call: basic research, applied research and post-doctoral research. Six projects in applied research were finally selected, four basic projects and eight post-doctoral projects. According to our classification, there were six basic research projects, six expertises, four policy studies, and two applications.
Classification of selected projects according to Research Agency of science and to our classification: