Dorys Y. Rodríguez-Castro, Mg.Sc., Predoctoral researcher, Deusto Business School, University of Deusto, Unibertsitate Etorb., 24, 48007 Bilbo, Bizkaia, Spain, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
Juan Aparicio, Ph.D., Director of the Center of Operations Research (CIO in Spanish), University Miguel Hernandez of Elche, Center of Operations Research, Center of Operations Research, Avda. de la Universidad, s/n, 03202 Elche, Spain, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Purpose: The paper introduces a functional framework that synthesizes the functions and capabilities that currently guide the empirical evaluations identified in the literature. Methodology: In this paper, a systematic review of the literature is carried out, which sheds light on the relationship between the modeling of the production of higher education institutions and the objectives of higher education policies. Findings: Our results evidence that four input-output relationships predominate in the production models used to measure the performance of higher education institutions. However, our results point to the existence of certain imbalances in measuring the three university missions. Implications for theory and practice: The functional framework presented here shows that there are several mismatches between the production that is examined in the assessment of HEIs' performance and the goals of higher education policies. This has important implications, both for academia and for the policy practice of HEIs and HESs, if we are to achieve a fair and equitable representation of the activities performed by HEIs and their multiple contributions to HESs. Originality and value: This review emphasizes the need to address broader analytical frameworks that help to avoid possible systemic failures that may arise due to the absence or excessive importance given to concrete functions and capabilities.

Keywords: higher education institutions, functions, performance, capabilities.literature review, integrative, efficiency, functional framework.

INTRODUCTION

Higher Education Institutions (HEIs) participate in at least fifteen different types of policies and objectives related to higher education (OECD, 2017a). The term Higher Education Institutions (HEIs) is used to refer to different types of tertiary education training institutions including universities, colleges, polytechnics, technical institutions, vocational training centres, etc. They contribute to the training of human capital, as well as to the social, cultural, economic, and environmental development of territories (OECD, 2017a). Proving whether HEIs ‘work well’ requires a comprehensive understanding of the activities these undertake (Benneworth, Pinheiro, & Sánchez-Barrioluengo, 2016), of the synergies produced among these activities (Johnes, 2015), and of the social objectives and expectations set for HEIs by higher education policy (OECD, 2017b). The current (dominant) model used to assess the performance of HEIs streamlines their operation in university missions (called basic functions) representing teaching, research and extension - commonly referred to as the third mission - (Martin & Etzkowitz, 2000). However, this model falls short in comprehensively characterizing the role of HEIs and their contribution to society, and hence, it may be unsatisfactory to detect problems in the operation of the Higher Education System (HES) (Sánchez-Barrioluengo, 2014). To contribute to this endeavour, the article reviews and synthesizes the evidence related to the measurement of HEIs’ performance. With it, we aim to identify the functions and capabilities that are modeled by the empirical literature and integrate them into a functional framework that facilitates the design and implementation of systemic evaluation. A systemic perspective is a holistic approach that puts the study of wholes before that of parts (Jackson, 2009).

Policy makers are increasingly interested in functional frameworks as novel means to align the functions and capabilities required by (health, innovation, education, etc.) systems with their respective policy objectives (Smits & Kuhlmann, 2004). These increasing demands make them become a suitable methodological choice to promote the sustainability of systems, and particularly, of those with profound structural and functional transformations (Weber & Rohracher, 2012) as it is the case of HESs (Benneworth et al., 2016). Functional frameworks thus emerge to highlight the processes that are required to guarantee the good operation of a system, aiming to clarify how such system works (Wieczorek & Hekkert, 2012). Following Parson (1974), the term process is here understood as an action of transforming inputs into results. However, few examples of functional frameworks can be found in the literature on higher education, and even fewer examples of its use in empirical research (Molas-Gallart & Castro-Martínez, 2007). For an exemption, see Molas-Gallart & Castro-Martínez (2007) in relation to the third mission. The paper thus aims to contribute to reducing this gap by integrating into an analytical framework, the functions and capabilities that are examined in the literature to measure the performance of HEIs. Specifically, the paper focuses on reviewing the extant evidence at the Nation-State level.

Throughout this review, the term ‘university function’ will be used to refer to the contribution of HEIs to achieve the central objectives of higher education policies. In turn, ‘capabilities’ correspond to the set of internal processes that HEIs execute autonomously to develop their missions. This implies that HEIs have multiple responsibilities (Salmi, 2017; Zwaan, 2017), emphasizing their multi-dimensional and multi-product character (Cheng & Wu, 2008; Cohn, Rhine, & Santos, 1989). Adding to this, the global change in university models (e.g., entrepreneurial university, research university) is leading HEIs to specialize in certain functions, so as to differentiate from the rest (Benneworth et al., 2016). Consequently, these changes have influenced the relative importance of the functions and capabilities to be chosen and assessed by HEIs when defining their strategies (Martin & Etzkowitz, 2000). In spite of the efforts made in previous literature reviews (Berbegal Mirabent & Solé Parellada, 2012; De Witte & López-Torres, 2017; Gralka, 2018; Rhaiem, 2017), little is yet known as to the functions and capabilities that currently guide the evaluations of HEIs, and whether these vary depending on the perspective of analysis. By introducing a functional approach, which considers functions and capabilities as processes, we offer an alternative way to conduct empirical evaluations of HEIs’ performance (Wieczorek & Hekkert, 2012).

The way in which HEIs fulfill their functions can be qualified from different perspectives. From an economic approach, one could argue that a HEI works well if it is effective in achieving the expected results, if it achieves these results efficiently, and at a minimum cost (Sarrico et al., 2010). For this reason, most studies addressing the performance of HEIs have focused on their efficiency (De Witte & López-Torres, 2017; Gralka, 2018). Methodologically, it is a matter of modeling the relationship (R) between the inputs (I) dedicated to a particular HEI, and the multiple outputs (O) achieved by this in all or in some of its missions, which we have labeled here as input-output relationships (IO-Rs). However, the generalization of the IO-Rs identified in the literature with respect to university missions has led to a lack of awareness about the functions and capabilities required, exploited, and combined by HEIs on the one hand, and about how the empirical models used in the assessment of HEIs’ performance reflect the contribution of HEIs to the operation and performance of the HES in which they are embedded. To contribute to closing this gap, we characterize the combinations of IO-Rs in terms of functions and capabilities, and analyze their relevance in the assessment of the functioning and performance of HESs. To reach the previous overarching goals, the following research objectives are developed in the paper: 1) to characterize the processes (IO-Rs) according to their missionary nature and output mixes; and 2) to examine the adjustment between the production modeled in the IO-Rs and the goals of higher education policies. These research goals also become instrumental for policy makers when defining educational policies, while being relevant for organizations developing university rankings.

Based on the literature that addresses the measurement of HEIs, it is thus worth asking the following research question:

RQ1: How is the production of HEIs modeled to evaluate their performance?

RQ2: What is the adjustment between the production of HEIs and the

objectives set for them?

To answer these research questions, a systematic literature review is conducted, applying the methodological approach of the Integrative Review of the Literature (IRL) (Torraco, 2016). Due to the nature of the above research questions, the IRL was chosen because it allows the extraction and classification of quantitative evidence (Whittemore & Knafl, 2005) using techniques that can be used to compare several groups, which in our case correspond to the IO-Rs. In addition, the final purpose of an IRL is to integrate its results into a framework, which in our case corresponds to a functional framework that integrates the capabilities and functions that we want to make available for researchers and public managers in higher education. It is worth noting that the functional framework presented in the paper is based on the evidence gathered from the review of the literature on the assessment of the performance of HEIs from an efficiency perspective.

The functional framework introduced in the article adopts a systemic perspective because it allows a thorough analysis of the relationships among the multiple production processes and the policy objectives defined for a given system (Smits & Kuhlmann, 2004). The benefit of applying a functional framework is that it allows examining the connection between functions and capabilities (Wieczorek & Hekkert, 2012), offering an alternative way to assess the sustainability of a system. This analytical approach considers that a system is sustainable as long as there is a balance between the contributions that emerge from it and the existence of the resources required to continue producing them (Edquist et al., 2018), which is becoming increasingly attractive for public policy makers (Jackson, 2009).

The contribution of the paper to the literature is twofold. First, it identifies the functions and capabilities that guide the empirical evaluations of HEIs. In doing so, it provides a critical analysis of the generalization of the IO-Rs (i.e., production functions) with respect to the three university missions. The second contribution is the analysis of the different ways in which the literature combines functions and capabilities with respect to the three university missions. As a result, the paper introduces a functional framework that draws attention to the possible systemic failures that may arise from the absence or excessive importance given to certain functions or capabilities (over others) in the evaluations of HEIs.

The paper is structured as follows. Section 2 discussess the rationale of functional approaches in higher education, and how IO-Rs are modelled in the literature. In Section 3 we present the methodology followed in the IRL. The results of the review are evidenced in Section 4. In particular, emphasis is made on the structure of the models used in the literature to assess HEIs’ performance, and the functions and capabilities required by HEIs. Section 5 introduces the functional framework that helps assess HEIs’ performance from a systemic perspective. Finally, Section 6 concludes the paper.

THEORETICAL FRAMEWORK

The literature on educational efficiency has been the subject of numerous reviews3. Most of them have addressed the methodological (and mathematical) approaches used to assess the education sector as a whole (i.e., De Witte and López-Torres, 2017; Johnes, 2015; Thanassoulis et al., 2016). The two most recent reviews are those by Rhaiem (2017), focused on research efficiency, and Gralka (2018), which applies Statistical Frontier Analysis to assess the efficiency in the higher education sector. Despite both studies address efficiency in higher education, they analyze the methodological approaches that are applied to the analysis of all types of units (higher education systems, universities, departments, etc.), geographic scales (between countries, regions, localities, etc.) and missions, without deepening into how the empirical literature relates the inputs and the outputs of the previous units of analysis, and without discussing the functions and capabilities of HEIs.

The functionalist approach

To gain a better understanding of the rationale of this review, we will start by discussing the concept of ‘university function’ from a systemic perspective. In higher education it is common to use the term ‘university mission’ as a synonym for the functions of HEIs, to refer to their operation in terms of teaching, research and extension, the latter being associated with the links and interactions of HEIs with the society (Benneworth et al., 2016; Martin & Etzkowitz, 2000; Sánchez-Barrioluengo, 2014). Parsons’ functionalist theory, which applies to social systems, refers to the term function as the contribution made by the components of a system, so the latter operates continuously (Cuff et al., 2006). From this perspective, social systems work well if they meet two conditions: (i) they carry out the processes that are indispensable to meet the objectives assigned to them; and (ii) they produce results that contribute to the overall achievement of these objectives (Jackson, 2009). Castells (1993) refers to the term function as the role played by the university in society. In this sense, the OECD (2017a) defines the production of results that meet the needs and objectives of students and their families, employers, economy and society as the main functions of HEIs. In the context of innovation systems, the expected contributions of HEIs are clearly defined for the innovation system to achieve the objectives assigned to it (Giuri, Munari, Scandura, & Toschi, 2019; Hekkert et al., 2007; Laredo, 2007a; Molas-Gallart et al., 2002; Wieczorek & Hekkert, 2012). However, to the best of our knowledge, there is no analytical framework that classifies all the functions and capabilities of HEIs. Table 1 summarizes the different classification schemes identified in the literature, together with the objectives posed to higher education.

From this systemic perspective, the term function encompasses the set of activities, processes and interactions that a HEI needs to execute to fulfill the purpose(s) for which it was created. This implies that the notion of function simultaneously involves the capability to produce (outputs) as well as the ability to contribute to the system. One of the conceptual frameworks that analyzes the functions of a system and discriminates among the capabilities of its components is the cascade model (Potschin & Haines-young, 2013). According to the conceptual assumptions behind this model, the production of a component (in our case the HEI) that is part of a system (in our case the HES) aims to fulfill two purposes, namely, supplying the services or goods that the system is expected to deliver, and providing the support that is required to produce the said services. Following, we define university functions as all those processes the results of which directly contribute to the achievement of the central objectives of higher education policies. Likewise, we define the capabilities of HEIs as the determinants of the processes that are required for HEIs to achieve the results that are expected from them within the general framework of higher education policies.

The main advantage of analyzing functions from this systemic perspective is associated with the possibility of assessing the performance of HEIs from the adjustment between social expectations, the results achieved and the capabilities of HEIs to achieve these results (Smits & Kuhlmann, 2004; Wieczorek & Hekkert, 2012). Other advantages are related to the opportunity to identify the specialization strategies of HEIs (Kitagawa & Oba, 2010), offering a methodological alternative to overcome the classical “one-size-fits-all” measurement models (Benneworth et al., 2016; Sánchez-Barrioluengo, 2014). These advantages, together with the well-established arguments about the transformation of HESs (Carpentier, 2018), could provide a broad framework to identify potential failures or operational problems in these systems (Weber & Rohracher, 2012; Brennan et al., 2014). It is also worth noting that the application of the functional framework to social systems has some limitations. As discussed by Jackson (2009), the main limitation is the lack of a linear relationship between the production of a social institution and the accomplishment of the political objectives that are intended with it.

Economic, political, and social circumstances have promoted substantial changes in the role of HEIs and their organizational models (Martin & Etzkowitz, 2000). This paradigm shift has led to the emergence of various university models: teaching universities, research universities, innovative universities, business universities (Etzkowitz, 2017; Etzkowitz et al., 2000), adult education universities, open (remote) universities, and the so-called world-class universities (Salmi, 2009). The diversification among the previous university models has increased, partly due to the growing global competition for reputational success, or as a strategy of adaptation to the dynamics of global markets (Olivares & Wetzely, 2014). Consequently, depending on the model a HEI may have chosen, its activities are expanded or specialized to comply with some of the essential functions of HESs (Benneworth et al., 2016), which has direct implications in the difficulty to assess the functioning of HEIs and thereby of HESs (Martin & Etzkowitz, 2000).

The functions and capabilities of HEIs

Education economists use production functions to assess how HEIs convert inputs into outputs to achieve the goals assigned to them. This implies the use of a mathematical function that models the relationship between the available resources (Inputs) and the results (Outputs), in an Input-Output Relationship (IO-R). As a result, IO-Rs aimed at estimating efficiency of teaching, research or extension missions can be found, together with IO-Rs in a combined model, in which the inputs and outputs of different missions are mixed (Berbegal Mirabent & Solé Parellada, 2012). IO-Rs have been conceptualized as educational production functions (Toutkoushian & Paulsen, 2016), and specifically, as multiproduct function (Cohn et al., 1989). One of the critical elements of the educational production function is the ability to combine and juggle the different interests of the policies underlying higher education, since they can become contradictory (Castells, 2001).

Theoretically, the inputs of the educational production function represent those variables that are indicative of the human (students, lecturers, administrative staff) and capital resources (financial resources, facilities, consumables). Input indicators are usually defined in terms of quantities or investments that represent the availability of certain resources that are needed to achieve certain results. In the educational production function, these results are measured through such measures as the number of students, research income or research scholarships (see Berbegal Mirabent & Solé Parellada, 2012). This opens a debate about whether HEIs are efficient for what they contribute to the system, or because they have greater and better possibilities of increasing their results for a given amount of resources (i.e., in public HEIs). According to Benneworth et al. (2016) the existing measures reinforce the inequalities among the HEIs that are well equipped in research and technology transfer, and those that are oriented towards the quality of teaching, which can have consequences on the sustainability of certain institutions if public policies award a larger relative weight to a particular mission as compared to another (Duan, 2019).

To capture the overall production of HEIs, scholarly work has strived to combine outputs from the three university missions and model a multiproduct type of educational production function. In general terms, the outputs related to the teaching function are measured through indicators of academic achievement (graduation rates, grades, enrolled students) and employability. The production of research is captured through such indicators as scientific publications, research contracts, scholarships and income for research activities, or doctoral theses (Berbegal Mirabent & Solé Parellada, 2012; De Witte & López-Torres, 2017; Sánchez -Barrioluengo, 2014). Additionally, the studies that focus on the third mission especially deal with issues related to knowledge transfer (Laredo, 2007a; Molas-Gallart et al., 2002).

In recent years, institutional dynamics have assigned new and greater responsibilities for internationalization, entrepreneurship and regional development to HEIs, stressing their multidimensional status even further (Martin & Etzkowitz, 2000). Operationally, this has led to the need to further decompose the educational production function into different outputs of a mission (Toutkoushian & Paulsen, 2016). For example, in the teaching mission, the number of new graduates needs to be broken down either into the number of graduates for undergraduate, postgraduate and doctoral studies, or in the number of new graduates by disciplines, among others (Thanassoulis et al., 2016). As long as the research mission is concerned, it is necessary to expand the IO-R to include outputs that reflect the effort of HEIs towards knowledge transfer, such as licensing, or the income from the sales of research services, to name a few (Liang, Li, Cook & Zhu, 2011; An et al., 2017; Yang, Fukuyama & Song, 2018). In line with Laredo (2007b) there is a general consensus in the orientation of the third mission towards the transfer of knowledge, being the outputs associated with patenting, the creation of spin-off firms, contracts with industry and the public sector, participation in policy definition, inclusion in cultural and social life, as well as involvement in initiatives to socialize and promote science. In turn, the OECD considers that the third mission should focus on innovation, with outputs associated with the creation of companies and the generation of royalty income (OECD, 2017a).

Despite these efforts, scholarly work has warned about the limitations of data availability and their quality, which has led to an excessive use of some indicators as compared to others (Bonaccorsi & Daraio, 2008), and in many cases, to combine in the same educational production function indicators that represent each mission, but that do not discriminate between functions and capabilities of HEIs.

METHODOLOGY

This article makes an IRL on the performance measurement of HEIs, following the methodological guidelines provided by Whittemore & Knafl (2005). In particular, we focus on those contributions that have approached such measurement from an efficiency perspective (see Section 1). In order to guarantee the robustness of our results, these are compared with previous reviews dealing with the performance and efficiency of the education sector, the quality of higher education, and DEA (i.e., Data Envelopment Analysis) methods applied to the education sector.

The IRL is developed in three stages: 1) strategy followed for the literature search, 2) development and application of taxonomies, 3) data analysis and integration of the results in a functional framework. The following sections provide the details of each stage, so the systematic method followed in the IRL can be double-checked, and thus, ensure the replication of our research.

Literature search strategy

The documents included in the review had to meet the following eligibility criteria: 1) their objective is to assess the efficiency, productivity, and performance of HEIs; 2) they provide an empirical or methodological study with application and implications in higher education; 3) the unit of observation is the HEI, which operates in a national context; and 4) the documents are academic articles, written in English, published between 1978 and the cut-off date of 12/02/2018, in a peer-reviewed journal and indexed in the bibliographic databases contained in the Web of Science, Scopus, Proquest or EBCHOST (see Table 2).

For the literature search, the entire set of bibliographic databases used in other reviews was checked. The search condition was constructed using a set of keywords frequently used in studies of HEIs’ performance, efficiency and productivity, which was validated with the keywords used in the search conditions of other systematic reviews. Boolean operators nested in three terms were used, which resulted in the following search condition: (efficiency OR productivity OR performance measurement) AND (higher education OR college OR university OR “post secondary” OR postsecondary OR universities) AND (“data envelopment analysis” OR “DEA” OR “SFA” OR “stochastic frontier”).

Figure 1 describes the flow of activities followed to get to the final sample of articles reviewed. The titles and abstracts of the 1604 remaining articles were reviewed and classified by sector according to the methodology described by Liu et al. (2013a). We excluded 1054 studies that referred to other sectors beyond higher education, and 52 studies focused on non-academic units of HEIs. 90 methodological studies without empirical application, as well as narrative, theoretical or methodological review studies were also excluded (see Tables 2 and 3). The remaining 391 documents were included in the IRL for full text reading and their additional classification according to the observation unit (i.e., Higher Educaction System-HESs, HEI, department, center, etc.) and geographical scale (global, national, or local), relevant aspects on systemic perspective. As a result of this classification, 205 articles that measure HEIs’ performance at a national level were kept for the quantitative and qualitative synthesis of the IRL.

Figure 1. Flow chart on the search and selection process of the Integrative Review of the Literature

The articles that constitute our sample are published in a large number of journals (140), although in most of these (61%) only one article has been published. The journals with the largest number of publications are Education Economics (9), Socio-Economic Planning Sciences (8), Scientometrics (8), Omega (6), Annals of Operations Research (6), Economics of Education Review (6), and Research in Higher Education (5). The full list of references that have been reviewed are available in the supplementary material available online (see Table 5). Most of them are empirical (86%), and the remaining 13% develop methodological solutions with a direct application to higher education.

Figure 2 shows how studies on HEIs’ performance and efficiency have increased significantly in the last decade. Most of them (93.7%) measure the performance of traditional HEIs (i.e., universities, colleges), which fulfill the three missions and offer long-term programs (UNESCO Institute for Statistics, 2014). This implies that only 13 out of the 205 studies analyze other types of HEIs, such as business schools or vocational training centers, which offer short-term post-secondary courses (less than two years) and do not necessarily fulfill the three university missions (Porto Gómez et al., 2018). It is also important to note that, despite the rise of the universities that provide distance education, no specific study was found about this in our review. Regarding the sectoral nature of HEIs, 34.9% of the studies focus on public HEIs, 11.1% compare between public and private HEIs, and 4.4% deal with private HEIs. It is noteworthy that the remaining 49.5% do not discriminate among HEIs according to their sectoral nature.

Figure 2. Number of articles on the evaluation of the efficiency of universities (1988-2017)

Note: articles published in 2018 are excluded because the coverage is unrepresentative.

Overall, the evidence gathered encompasses 30 countries. However, 70% of the studies reviewed focused on HEIs in 8 countries: United States (42 articles), United Kingdom (33), Italy (18), China (15), Australia (12), Spain (12), Taiwan (10) and Germany (8). As of 2011, the year in which the literature on efficiency in higher education increases exponentially (see Figure 2), there is a boom of this type of studies in the United States, China, and Spain, while in the United Kingdom, they decrease significantly. There is evidence on the evaluation of National Higher Education Systems in a total of 30 countries. However, the majority of these studies (comprising 19 countries) were published recently. In this regard, it is important to note that only articles published in English were considered in the IRL, and therefore, it is likely that the figures are underestimated for non-English speaking countries (Figure 3).

Figure 3. Geographical distribution of the articles on the assessment of HEI efficiency (N=205)

Taxonomies and classification of the evidence

The second stage extracted, classified and systematized the evidence in the sample using the taxonomy technique (Doty & Glick, 1994). In particular, two taxonomies were applied. The first taxonomy classifies input and output variables as well as external factors according to their mission and functional orientation. This taxonomy was constructed taking as reference the previous revisions (Tables 4a, b, and c; Figure 1). The second taxonomy classifies the structure of the model according to its dimensionality (production, structure and temporary nature). Production is classified as one-dimensional, when the model is only associated with a single mission, or multi-dimensional, when outputs are associated with two or more missions. Following Cook, Liang & Zhu (2010) and Kao (2014), the structure of the production system is categorized as basic (black box) and network-based (not black box). Finally, depending on the temporary nature considered by the production process, four types of models are distinguished: static models of a single period, static of several periods (time-invariant), dynamic without intertemporal dependence, and dynamic with temporal dependence (time-variant) (Emrouznejad & Thanassoulis, 2005; Gralka, 2018). For every taxonomic category and sub-category a code was assigned, with the purpose of synthesizing, normalizing and representing in a standardized way all alternatives in such a way that the empirical evidence obtained per article corresponds to the frequency of a particular code.

DATA ANALYSIS

The third stage consists of establishing the analysis framework to identify the university functions and discriminate them from the capabilities of HEIs. Initially, the different IO-Rs were classified and characterized by an exploratory analysis of the input and output variables grouped by the missional orientation of the output. Then, a step-by-step discriminant analysis was performed. The discriminant analysis was used because it allows building a predictive/descriptive model that discriminats a group based on observed predictive variables (e.g., IO-Rs) (Calvo & Rodríguez, 2003). The discriminant analysis allows us to achieve three purposes. First, to reduce the variability of the 82 variables included in our analysis through discriminant functions. To validate their statistical significance, we use the canonical correlation coefficient and p-value of the Lambda Wilks statistic. Second, to examine whether the form of IO-Rs classifies studies. To classify the studies, we used the Mahalanobis distance, which is calculated from the covariance matrix grouped within the group. Finally, to establish which of the identified inputs or outputs are predictors of the IO-Rs defined to prior. The assignment of a study to a given group has been validated using the cross-validation procedure.

Once the dominant IO-Rs were identified, the functions and capabilities were analyzed and classified. The first condition was to define what would be considered as a university function. As discussed, we follow the concept of function given by Jackson (2009). The frame of reference for university functions and capabilities was built from the harmonization of policy objectives in higher education (OECD, 2017a) and from the different schemes of the functions of a HEIs used in the literature (Brennan et al., 2014; Laredo, 2007a; Molas-Gallart et al., 2002) (see Table 1). To integrate functions and capabilities into a functional framework, we apply Minimal Spanning Tree analysis, since this technique makes the visualization of the results easier. To build the resulting tress, the similarity among the studies included in the analysis was explored using Jaccard’s measure of similarity.

Table 1. Correspondence of the integrated classification of functions and capabilities of HEIs proposed in this review, as compared to previous classifications

	Functions (F) And Capabilities (C) Present review	Functions from the HES benchmarking perspective (OECD, 2017b)	Functions from the innovation perspective (Brennan et al., 2014)	Functions from the third mission perspective (Laredo, 2007a; Molas-Gallart et al., 2002)	Guidance on higher education policy themes (OECD, 2017a)
Teaching
F1	Attainment and graduation rates	Attainment and completion Skills (score)	Teaching and learning Student assessment	-	Participation in higher education
C1 C2	Attraction students Reduce student dropouts	Access and participation. Student experience of teaching and learning	-	-	Equity of access Participation in higher education
C3	Academic production and quality assurance	Use of technology	Curriculum development	Alignment of curriculum to societal needs	Diversification of study provision
F2	Employability	Labour market outcomes	Progression rates	Employment relationship and Learning activities	Links to the labour market
F3	Continued education and lifelong learning	-	-	-	Life-long learning Continued education
C4	Internationalization	International education and mobility	Student mobility	-	Internationalization
C5	Resources mobilization-Teaching	-	-	-	Funding for teaching (including grants and loans)
Research
F4	Doctoral Education	Doctoral and postdoctoral research	-	PhD students and graduates research	Doctoral education
F5	Generation of new knowledge	Publications. Scientific impact. Joint publications with non-academic authors	New knowledge creation. Testing and measurements. Experimentation. Validation and Dissemination of results	Knowledge creation. Guidance of the search (prospective, and surveillance technologies)	Research and innovation
C6	Resource mobilization-Research	Earned income for research projects	-	Contract research academic	Funding for research (competitive)
Extension mission
F6	Generation of Intellectual assets	Patents Licensing of patents	Protection of intellectual property	Intellectual property	Technology transfer and innovation
F7	Entrepreneurial activities	Spin-off companies. - Start-up companies	Creation of spin-offs	Entrepreneurial activities	Policies on entrepreneurship education
C7	Revenue from knowledge transfer	Consultancy contracts. - Total earned royalty income	Contracts with industry. - Contracts with public bodies	Commercialization of facilities. - Resources mobilization (advisory work, contracts with industry and public institutions)	Policies on the commercialization of research
C8	Create techno-economic networks	-	-	Non-academic collaboration in academic research. -Market formation. -Knowledge diffusion through networking	Policies to develop collaboration University-Industry-State
F8	Social engagement	Social and economic outcomes	Participation in policymaking. - Public understanding of science. - Involvement in social and cultural life	Social networking. - Non-academic dissemination of knowledge	Social engagement. - Development of civic competences. - Collaboration between HEIs and their communities
F9	Regional development	-	-	-	Regional development and Regional integration
Administrative management
C9	Attraction of academic staff	Engagement active staff, researcher, and staff support. - Staff development	-	Academic staff, scientists and technicians’ mobilization	Academic career
C10	Improvement in administrative management	Student support	-	-	Research capacity of the HEI

RESULTS

Processes examined in the literature

This section aims to illustrate the processes (IO-Rs) that are discussed by the sampled empirical literature. The 205 articles reviewed encompassed 1690 variables, which after applying the taxonomies, were classified and coded into 30 inputs (I), 38 outputs (O), 14 intermediate outputs and 16 external factors. Table 2 presents a summary of the variables with the highest frequencies4.

Most of the reviewed studies (approximately 90%) model the educational production function classifying the variables as inputs and outputs (i.e., black box models), although sometimes intermediate outputs (Z) are used in the internal structure models (i.e., non black box). Additionally, some of these studies (52.2%) also include variables that represent the environmental conditions in which HEIs operate, and which are known as external factors or determinants of efficiency.

Regarding the set of variables used as proxies for the outputs (O) of HEIs, research has become the most attractive mission to measure (see Table 2). The financing of research and scientific production, especially publications and their quality, are the indicators that represent the accomplishments of HEIs in this mission (Bonaccorsi & Daraio, 2009; Rhaiem, 2017), as evidenced by the emphasis put by 147 out of 205 studies. In the case of the indicators related to research income from donations or grants, they are frequently used as input variables, despite some models consider them as outputs. However, it is important to note that the use of this variable either as input or as output does not necessarily indicate the leverage of resources. On the contrary, in models oriented to inputs, these types of outputs can be indicative of an IO-R related to IESs’ capability to operational improvement, as in the studies by Castano & Cabanda (2007), de Guzman & Cabanda (2009), and Kudła & Stachowiak-Kudła (2016).

The graduation rate and the number of enrolled students are the main proxies for assessing the efficiency of HEIs. It is common to find that outputs related to human capital formation are analyzed from global graduation rates (61 out of 205 articles), although some studies discriminate between undergraduate (28 articles), postgraduate (24 articles), and doctorate studies (19 articles) (Table 2). However, as Sánchez-Barrioluengo (2014) warns, postgraduate or doctoral graduation rates are not always considered as indicators of the teaching mission. In fact, a large share of the sampled studies considers them as proxies of the research mission.

Table 2. Variables of input, output, intermediate outputs and external factors of the IO-R in the literature

Type of variable	Cod.	No. art.	%	Type of variable	Cod.	No. art.	%
a) Input (30 variable types)				b) Output (38 variable types)
Academic staff	I11.1	132	64.4	Research income	O14.2	78	38.0
Total operation (total expense)	I15.1	93	45.4	Research outcomes	O23.2	69	33.7
Non-academic staff	I11.3	51	24.9	Graduation (rates)	O22.1	61	29.8
Fixed assets value	I13.3	38	18.5	Graduate students (enrolment)	O12.3	35	17.1
Total students (enrolment)	I12.1a	35	17.1	Undergraduate students (enrolment)	O12.2 a	34	16.6
Financial revenues	I14.4	28	13.7	Research quality	O23.3	33	16.1
Academic expenditures	I15.2	28	13.7	Total students (enrolment)	O12.1 a	32	15.6
Research staff	I11.2	27	13.2	Undergraduate students’ graduation	O22.2	28	13.7
Scores of national entrance exam	I12.6	25	12.2	Graduate students’ graduation	O22.3	24	11.7
Undergraduate students (enrolment)	I12.2 a	23	11.2	Income technology transfer	O14.3	21	10.2
c) Intermediate output (14 variable types)				d) External factors (16 variable types)
Research outcomes	OZ23.2	5	2.4	Region’s economic development	F32.4	34	16.4
Research income	OZ14.2	4	2.0	Funding availability	F32.5	17	8.2
Total students (enrolment)	OZ12.1	3	1.5	Socioeconomics conditions	F32.1	15	7.2
Total operation (Total expense)	OZ15.1	3	1.5	Area of knowledge	F31.4	26	12.6
Research quality	OZ23.3	3	1.5	Sectoral or governance nature	F31.1	23	11.1
Academic staff	OZ11.1	2	1.0	Presence of medical schools	F314.1	18	8.7
Intellectual assets	OZ24.1	2	1.0	Student population	F31.7	17	8.2
Reputation	OZ27.5	2	1.0	Size of the HEI	F31.2	15	7.2
				*Do not use external factors	F0	101	48.8

The attraction of human capital is related to the ability of HEIs to attract students, which is estimated by the number of graduate students (17.1%), undergraduate students (16.6%), or by the total number of enrolled students (15.6%). It is worth noting that, despite these indicators have traditionally been considered as inputs (De Witte & López-Torres, 2017), our results show that all of them are rather used as outputs in the literature, especially when the purpose of the study is to analyze the competitiveness of HEIs (Abbott & Doucouliagos, 2009; Agasisti & Dal Bianco, 2009).

Regarding the third mission, few studies refer to the outputs of this mission (21 articles), and in all cases, the indicator used is the income generated by knowledge transfer activities (Table 2). Despite the potential diversity of outputs associated with this mission (Laredo, 2007b), our results confirm that HEI’s third mission is assessed solely from knowledge transfer activities. This occurs, among other reasons, due to the lack of consensus about what is the expected output of the third mission (de La Torre et al., 2018), the autonomy of HEIs to choose their approach towards this mission (Giuri et al., 2019), and the lack of data availability (Molas-Gallart & Castro-Martínez, 2007; Molas-Gallart et al., 2002).

Input-Output Relationships (IO-Rs)

In this section, the sampled studies are classified into IO-Rs according to the combinations of inputs and outputs used, applying a discriminant analysis. Initially, 11 possible IO-Rs were found between the production of the university missions. Therefore, it was necessary to verify whether these IO-Rs helped to classify the studies. To this end, a discriminant analysis was applied to the matrix of variables of the different IO-Rs, excluding those with a single article. The IO-Rs were classified by the first two discriminant functions resulting from grouping the studies into four groups of the IO-Rs (see Figure 4). These functions explain 84% of the total variance and have a significant high value in the canonical coefficient (0.96 and 0.82, respectively). This analysis showed that the median among the variables is statistically different among the analyzed IO-Rs (λ = 0.002; P < 0.005). Therefore, we consider that they provide a satisfactory solution to discriminate the revised studies. According to the p-value of Lambda Wilks, 21% of the variables are relevant to discriminate between these IO-Rs. Most of them correspond to outputs associated with knowledge transfer (O14.3; O24.1; O24.2; O15.4), research quality (O23.3), and total revenue (O14.4). Overall, 161 of the 205 studies (79%) were accurately classified by IO-Rs groups. The teaching and teaching/research IO-Rs appeared to have more misclassifications, implying that these two IO-Rs were harder to characterize based on the groups that were concluded from the extant literature. The examination of the discriminant function score plot (Figure 4) confirmed that knowledge transfer and research/knowledge transfer IO-Rs were fairly well separated from the others, while there was some notable overlap between teaching/research and teaching IO-Rs, as earlier discussed.

Figure 4. Discriminant analysis of the IO-Rs in the literature

Notes: Centroids correspond to the IO-Rs: Teaching (T), Research (R), Knowledge Transfer (KT) and Administrative Management (AM).

The four IO-Rs identified have different perspectives for analysis (Figure 5). A first group includes studies from the research, teaching/research, and teaching/research/administrative management IO-Rs (i.e., labeled as teaching/research IO-R). In 122 articles (59.5%), modeling this IO-R implies representing the key essential characteristic of HEIs, which is the integration of teaching and research (Martin & Etzkowitz, 2000). From this perspective, the educational production function combines research proxies that are mainly indicative of the generation of resources (research grants, project financing) or scientific production (research results), with global graduation rates or with the number of students enrolled in postgraduate studies (Figure 5). The basic argument for the studies that model teaching/research IO-R is that there are mutual benefits between teaching and research, and hence make HEIs efficient, for example, by linking scholars who are at the forefront in their field of knowledge with the generation of knowledge to achieve better results in teaching. However, there is large empirical evidence that supports the opposite (e.g., Duan, 2019). There is also evidence of the dangers of not combining these two missions (Martin & Etzkowitz, 2000).

The second group includes studies that belong to the IO-Rs associated with the teaching mission, administrative management, and the combination of outputs from these two missions (i.e., labeled as Teaching IO-R). The studies that model this IO-R leave aside the multidimensional nature of HEIs, to delve into teaching processes (one-dimensional models). They propose an educational production function where the efficiency of these institutions is not only represented in granting degrees, but also in attracting students, forming quality human capital, guaranteeing the employability of graduates, and generating an academic offer of quality (Figure 5). In other studies, the outputs of the Teaching IO-R are decomposed to specify the production by levels of training (undergraduate, postgraduate, and doctorate), or by disciplines (Thanassoulis et al., 2016), which provides some advantages in benchmarking studies. Although there are divided positions about the need to focus on individual missions (Sánchez-Barrioluengo, 2014), our results support the idea that studies using the Teaching IO-R emphasize the quality and excellence in teaching over other outputs. Accordingly, they can reward HEIs specialized in teaching, and hence discourage them to engage in research development and extension (Benneworth et al., 2016).

Figure 5. Frequency of variables by IO-Rs groups. n=205 articles

Notes: Funtion (F), Capabilites (C), Absolut frecuency (F.A) and Relative Frecuency (F.R).

The third group only includes studies that belong to the extension mission IO-R, or those whose IO-R is exclusively modeling the knowledge transfer production function (i.e., labeled as Knowledge Transfer IO-R). Conceptually, this IO-R is closer to a knowledge production function (Azagra Caro, 2003), than to an educational production function. Thursby & Kemp (2002), use the Knowledge Transfer IO-R to examine the different outputs that characterize university licensing activities. They include five outputs as representative of this IO-R: sponsored research agreements between HEIs and industry; license agreements that allow the exploitation of HEIs’ intellectual property; fees received by HEIs in exchange for the use of their intellectual property; academic dissemination of potentially marketable innovations; and university patent applications. In the sampled literature, few studies model this IO-R (9 studies), and all of them use one or more of the indicators described by Thursby & Kemp (2002). In particular, these studies have quantified the income received by HEIs as a result of the research agreements sponsored by industry (Figure 5). Recently, some indicators related to entrepreneurship have also been considered by Ho, Liu, Lu & Huang (2014).

Finally, the fourth group includes studies from the research/extension and research/extension/teaching IO-Rs (i.e., labeled as research/knowledge transfer IO-R). This group includes studies that link research to knowledge transfer, and those (less common) that deal with modeling the three university missions. The studies that use research/knowledge transfer IO-R, unlike the knowledge transfer IO-R, jointly analyze the results in research and knowledge transfer, and its relationship with the availability and capabilities of academic staff (Figure 5). The studies that use this IO-R have in common the concern about whether the expansion of functions in HEIs to fulfill a third mission affects their performance (i.e., de La Torre et al., 2017).

Structure of the models

Because of the multi-dimensionality and multi-production of HEIs, it is thus expected that: 1) synergies occur between the production processes assigned to each mission; 2) there is an orderly sequence of outputs that accumulate in each dimension of the system; and 3) there is heterogeneity in the forms of production (Cheng & Wu, 2008; Cohn et al., 1989; Johnes & Johnes, 2009). In the production models discussed in the literature, the synergies that characterize the production of HEIs are managed in three ways. First, there are production models that assume that synergies do not occur, and where the production of each mission is independent from the others (one-dimensional). These production models approach multi-product through proxies that reflect several outputs of the same mission (Johnes, 1998). Second, there are production models – interdependent – that assume that synergies occur either between missions (Cherchye et al., 2017). Finally, there are some production models – interactive –- that assume a flow of intermediate outputs between missions (An et al., 2017), which may sometimes be generated from shared or related inputs (Cherchye et al., 2013). It is important to note that examining the processes of each mission independently can generate flaws in assessment and its interpretation, due to the synergies that exist among production functions (Johnes, 2015).

Most studies use frontier methods to estimate efficiency, particularly Data Envelopment Analysis (DEA) (64.8%), Stochastic Frontier Analysis (SFA) (26.2%), or a combination of both (8.1%), which confirms the findings of previous reviews (De Witte & López-Torres, 2017; Rhaeim, 2017; Gralka, 2018). In the last decade, several researchers have achieved significant progress to model the production of HEIs. For example, the advances in DEA techniques have allowed to ‘open’ the black box and model the causal relationships within HEIs (Cherchye et al., 2017; Kao, 2014; Liu et al., 2013; Tone & Tsutsui, 2014). With regard to SFA there are important advances in the models that analyze time-invariant efficiency, time-variant efficiency, environmental variables, heterogeneity, persistent inefficiency, or distributional assumptions (see Gralka, 2018).

It is common (89.1%) in the literature to use ‘traditional’ models in which production processes are interpreted as black boxes, in which the transformation of inputs into outputs is not considered (Aparicio et al., 2017), nor is the variation over time (Emrouznejad & Thanassoulis, 2005). This way of modeling the IO-R assumes that all HEIs have the same capabilities to fulfill their functions, which is certainly far from the observable heterogeneity (Benneworth et al., 2016). As evidenced in Table 3, three models dominate the extant literature. The first model is black box, time-invariant and multi-dimensional (50.7%). This model assumes that synergies occur between outputs of different missions, but it does not specify the transformation from inputs to outputs. The second model is also traditional, but assumes synergies between outputs of a single mission (one-dimensional), and is specifically applied in the IO-R Teaching (38 articles). The third type of model is multi-dimensional, black box, and dynamic (12.7%). Despite their internal structures are time-invariant, they intend to measure changes in productivity over time, assuming there is no intertemporal dependency between inputs and outputs (Chang et al., 2015; Emrouznejad & Thanassoulis, 2005; Gralka, 2018). To reach this goal, these models use the Malmquist productivity index, and do rarely use others such as the Hicks-Moorsteen productivity factor or the Luenberger indicator.

There are additional models that are recently applied to model production systems in higher education (see Table 3). These models are based on network structures and reflect the multilateral relationships between the different phases of a mission, and between different missions (Lee & Worthington, 2016; Cook et al., 2010; Schalk et al., 2010). Specifically, in higher education, the most used network DEA models are those of two-stage (Kao, 2014; Liu et al., 2013). These models assume that all inputs are devoted to the first process to produce certain intermediate products, which are, in turn, the inputs of the second process to produce the final outputs. Besides opening the black box, other scholars have resorted to evaluate the adjustment between the production of HEIs and the results expected from them (i.e., effectiveness), such as those of Clermont (2016), Powell et al. (2012), and Thanassoulis et al. (2018). Other network structures, although less common, are the parallel structure (7 articles) and the general structure in two stages (2 articles). Unlike the basic structure, these models allow inputs to be supplied to both stages of the process, which can also produce final outputs (Kao, 2014). Other forms of modeling correspond to dynamic network-based models that evaluate several periods, assuming that the production process follows a time sequence (i.e., there is intertemporal dependence between inputs and outputs). Examples of the latter models would be those by Färe & Grosskopf (1997), who have pioneered dynamic DEA models, the Tone & Tsutsui model (2014), or the intertemporal DEA model proposed by Chang et al. (2015).

Table 3. Distribution of models used in the literature

Dimensionality	Structure			IO-Rs groups				Total
			T/R		T	KT	R/KT		No. Papers	%
Multi-dimensional	Black box - time-invariant		93		-	-	11		104	50.7
One-dimensional	Black box - time-invariant		-		38	7	-		45	21.9
Multi-dimensional	Black box - Dynamic		19		-	-	7		26	12.7
Multi-dimensional	Non Black box - time-invariant		7		-	-	4		11	5.3
One-dimensional	Black box - Dynamic		-		6	1	-		7	3.4
One-dimensional	Non Black box - time-invariant		-		5	1	-		6	2.9
Multi-dimensional	Non Black box - Dynamic		1		-	-	3		4	1.9
One-dimensional	Non Black box - Dynamic		-		2	-	-		2	0.9
	Total		120		51	9	25		205	100

Functions and capabilities of HEIs

To discriminate between functions and capabilities, we followed the cascade model used by Potschin & Haines-young (2013). From the outputs modeled in the different IO-Rs, 7 functions and 12 capabilities were identified (Table 4). The comparison of these results with the capability/function classification scheme (see Table 1) reveals the absence of indicators representing social engagement (F8) and regional development (F9) in efficiency evaluations. By contrast, attainment and graduation rates are the most discussed functions in efficiency evaluations (50.7%), and the capability of HEIs to mobilize resources for research (40.6%). The multi-dimensional perspective dominates evaluations, as most studies examine two or three different functions (65%), and in some other cases (17%), more than four functions. Only 18% of the studies analyze a single function, and most of them evaluate the efficiency of teaching. However, these results should not be interpreted too simplistically, because the choice of functions or capabilities may depend to a large extent on the availability of reliable data. For this reason, we have analyzed which functions and capabilities are chosen to analyze the primarly objectives of higher education.

Table 4. Functions and capabilities by IO-Rs groups

C/F	Description	IO-Rs groups				Total
		T/R	T	KT	R/KT
Teaching mission
F1	Attainment and graduation rates	45.9	73.5	-	52.0	50.7
C1	Attract students	41.0	38.8	-	20.0	35.7
C3	Academic production and quality assurance	9.8	12.2	-	-	8.7
F2	Employability	4.9	12.2	-	-	5.8
C5	Resources mobilization-Teaching	4.9	8.2	-	12.0	6.3
C2	Reduce student dropouts	5.7	10.2	-	-	5.8
C10-T	Improvement in academic operations	4.1	4.1	-	8.0	4.3
C4	Internationalization	4.1		-	-	2.4
F3	Continued education and lifelong learning	0.8	2.0	-	-	1.0
Research mission
C6	Resources mobilization-Research	54.9	6.1	-	56.0	40.6
F5	Generation of new knowledge	54.1	-	-	68.0	40.1
F4	Doctoral and postdoctoral education	16.4	12.2	-	28.0	15.9
C10-R	Improvement in research activities	10.7	4.1	-	8.0	8.2
Extension (knowledge transfer)
C7	Revenue from knowledge transfer	-	2.0	88.9	48.0	10.1
F6	Generation of Intellectual assets	-	-	77.8	48.0	9.2
F7	Entrepreneurial activities	-	-	22.2	16.0	2.9
C10-KT	Improvement in transfer activities	0.8	-	11.1	-	1.0
Administrative Management
C10	Improvement in administrative management	8.2	14.3	-	-	8.2
C9	Attraction of academic staff	6.6	2.0	-	8.0	5.3

The central purpose of HEIs is to contribute to the formation of human capital in a territory (OECD, 2017a). Over the years, the literature has examined that HEIs fulfill this policy goal, particularly through the measurement of attainment and graduation rates (50.7%). Recently, measuring the training at the doctoral and postdoctoral levels has gained importance (15.9%). However, this function is closer to research than to teaching (Sánchez-Barrioluengo, 2014). Very few studies have targeted the employability (5.8%) or the provision of continued education and lifelong learning (1.0%). In terms of capabilities, it is common and recurrent to examine the capability of HEIs to attract students (35.7%), being mainly representative of the teaching mission (ibid). Other key capabilities that support the training of HEIs are less considered in efficiency evaluations, such as producing academic programs and accrediting their quality (8.7%), resource mobilization by teaching, particularity generating income by tuition (6.3%), or reducing student dropouts (5.8%) (see Table 4). In terms of capabilities, our findings indicate that the internationalization of HEIs remains the biggest challenge, as only 5 articles include variables related to international production (international students, international mobility, etc.).

Another central intent of HEIs is to contribute to innovation (OECD, 2017b), being this understood as a process that generates added value from research results (Azagra Caro, 2003; Molas-Gallart & Castro-Martínez, 2007; Philpott, Dooley, Oreilly & Lupton, 2011). Our results show that the literature focuses on whether HEIs generate new knowledge (40.1%), activity levels in doctoral and postdoctoral education (15.9%), and in the conversion of research results into intellectual assets (9.2%) or entrepreneurial activities (2.9%). Likewise, our results highlight the capability of HEIs to mobilize resources through research and knowledge transfer activities. In particular, the literature points to the relevance of generating new income sources from competitive funds supporting the development of research (C6 = 40.6%) and consultancy activities, either for the provision of research services or for the licensing of intellectual assets (C7=10.1%).

The third driver of HEIs is related to the role they play in social, cultural, and environmental development (Giuri, Munari, Scandura, & Toschi, 2019; Laredo, 2007b). Our results show, however, that this university mission is only evaluated from the perspective of knowledge transfer. From our point of view, the contribution of HEIs to regional development and integration, the dissemination of research results to societal actors, the engagement with policy makers, and their inclusion in social and cultural life, is something that should be tackled by future research.

Introducing a functional framework to assess HEIs’ performance

This section introduces the functional framework that integrates the functions and capabilities considered by the empirical literature, applying a Minimal Spanning Tree analysis. This functional framework represents a function/capability (each node), and the length of the line segments indicates the percentage of studies in which the output variables are shared by two nodes (Figure 6).

Empirical studies have different approaches to measure the missions of HEIs, although they evaluate similar activities and processes. The functional framework reveals that there are four key competences that characterize HEIs’ evaluations, which are the functions of teaching and research-knowledge transfer, and the capabilities of resource mobilization and improvement of teaching activities.

The generalization of IO-Rs concerning university missions highlight the functions and capabilities related to innovation, and it masks other them that are relevant in the operation of higher education. Although the set of the functions and capabilities turns out to be indispensable for the functioning of HESs, most of these are also key in science and technology systems, and innovation systems (i.e., Laredo, 2007a; Molas-Gallart et al., 2002). Therefore, we consider that this functional framework offers information that could be useful for future research not only on the contributions of HEIs to HESs, but also on their contribution to innovation systems, and to science and technology systems.

Figure 6. An integrative framework of functions and capabilities of HEIs based on the literature about performance assessment (1988-2018)

One of the key capabilities for assessing the performance of HEIs is their capability to mobilize economic resources, and specifically, their capability to obtain resources from competitive funds that finance research, and to sell or provide specialized services. The main objective of efficient HEIs in implementing an income generation strategy is to maximize income (see Figure 5), and to maximize the amount of research and services generated to support industry or government (Philpott et al., 2011). At the level of HESs, the measurement of this capability tends to reinforce the gap between specialized and highly prestigious HEIs and generalist HEIs that meet the demands of local knowledge (Giuri et al., 2019), which can lead to failures in the operation of these systems. Our review has evidenced that it is common in the literature to examine whether the efficiency in the development of this capability is affected by local environmental conditions, such as the presence of funding opportunities and the area of knowledge in which it operates. However, funding instruments also reward HEIs that implement income generation strategies, which means that generalist and low-prestige HEIs must support research with the income generated from teaching (de La Torre, Casani & Sagarra, 2018). It is important to note that the capability of HEIs to leverage resources is considered an essential capability for the development of research functions and knowledge transfer (Philpott et al., 2011) and a success factor in world-class universities (Salmi, 2009).

Generating new knowledge and obtaining research results is the most attractive function in the performance measurement of HEIs. The fulfillment of this function emphasizes the creation, dissemination and practical application of knowledge in academic and non-academic fields (see Figure 5). In the last decade, we have observed how the functions and capabilities related to knowledge generation and its transfer to society have started to dominate performance evaluations. These findings could be interpreted as a bias in the literature when judging the performance of HEIs, mainly from the elements that characterize research universities (Abramo & D’Angelo, 2014), or from those universities that are in a transition towards the adoption of entrepreneurial models (Etzkowitz, 2017), with potential unintended consequences on the structure of HESs and on their sustainability. The relevance given to generating new knowledge function has also contributed to widening the gap between prestigious HEIs and those specialized in areas of knowledge that have better publication rates, as compared to HEIs with low prestige (Giuri et al., 2019) and those dedicated to teaching (Benneworth et al., 2016). It is worth noting that the fulfillment of this function by HEIs is considered a prerequisite for innovation, becoming an essential function in their operation (Azagra Caro, 2003; Hekkert et al., 2007). However, one of these potential consequences is related to the hiring, retention and dedication to teaching of lecturers, a HEIs capability that has been poorly assessed by the literature (see Table 4).

Teaching is evaluated specifically in terms of the ability of HEIs to graduate (function) and attract (capability) students. Despite universities are currently required to be more committed to the employability of their graduates, it is notable that the effort to evaluate the compliance with this function is recent and much less frequent (5.8%) than the effort to evaluate educational achievements (50.7%). According to our results, few studies are concerned with the performance of HEIs from their contribution to the training of the workforce or from their internationalization capability, the latter being referred by Salmi (2009) as a key success factor of world-class universities. With regard to the capability of HEIs to attract and concentrate talent, the ability to attract students (35.7%) dominates the evaluation of HEIs’ efficiency. In turn, the number of studies that measure the ability of HEIs to retain students (5.8%), attract academic and research staff (5.3%), and to internationalize their activities (2.4%) is much lower, highlighting the need to develop metrics and to expand evaluations to cover these capabilities. It should also be noted that there is a greater interest in the literature to assess the performance of HEIs from their ability to attract students (35.7%), rather than from their ability to retain them or decrease their dropout (5.8%). Another pending aspect to consider in future research is thus related to the academic quality and the satisfaction of students on academic services.

The ability to improve operationally HEIs focuses primarily on examining research-related processes, as well as to produce academic programs and certify their quality. Specifically, the ability to improve academic quality is indicated by the decrease in spending and the increase in the production of academic credits. As shown in Figure 6, operational improvement is related to the capability of attracting students. On the other hand, the operational improvement of research is related to the attraction of human resources and to decrease the expenses in research. Furthermore, internationalization is considered a capacity, and not a function. According to the literature, the efficiency in fulfilling these capabilities is particularly determined by the size of the HEI.

Operational improvements in HEIs are essential for the sustainability of higher education systems (Sav, 2016). However, in the current competitive environment, the measurement of these capabilities tends to reinforce the gaps between HEIs due to rigidities in inputs (An et al., 2017; Duan, 2019). Examples can be found in the limitations in the access to qualified academic staff faced by regional HEIs, or the scarcity of financial resources to create adequate learning and research facilities (e.g., for modern laboratories in response to changes in the educational environment) (Benneworth et al., 2016).

CONCLUSIONS

The results of this review evidence the limitations of the generalization of the I-ORs with respect to the three university missions. Based on functions-capabilities framework, we conclude that the empirical literature has considered it relevant to evaluate four competences in HEIs, shedding new light on an alternative methodological approach to the traditional models based on the three missions. From a functional perspective, this alternative leads us to highlight two basic functions, teaching and research-knowledge transfer, and two basic capabilities, the mobilization of resources and the improvement of activities. The most recurrent is the HEIs ability to mobilize and generate economic resources, followed by their ability the generate research results and convert them into (tangible or intangible) assets. Consequently, one of the most notable results of this paper is the fact that more than 60% of the studies are inclined towards measuring functions and capabilities related to the fulfillment of objectives of research and knowledge transfer missions.

The functional framework presented here shows that there are several mismatches between the production that is examined in the assessment of HEIs’ performance and the goals of higher education policies. This has important implications, both for academia and for the policy practice of HEIs and HESs, if we are to achieve a fair and equitable representation of the activities performed by HEIs and their multiple contributions to HESs. Our framework shows that the functions and capabilities related to extension (i.e., social approach) and internationalization are clearly underrepresented in the literature. Due to, in many cases, the absence of data and information on social commitment, contribution to local development, regional integration, mobilization, and international insertion remains a bottleneck that could impede the future of systemic evaluations of HEIs. It should be noted that this representation of the operation of HEIs has some limitations, including: 1) the difficulty of measuring quality (Agasisti & Dal Bianco, 2009; Sánchez-Barrioluengo, 2014); 2) the interactions of the variables in the production of the activities carried out by HEIs; and 3) the lack of measures for many outputs. However, one of the main advantages of this approach is the adoption of the concepts of functions and capabilities expressed as input-output relationships.

The main trends observed in the review are described below, and based on the identified gaps, future research opportunities are identified. Our findings suggest that the different IO-Rs are common for the measurement of all types of HEIs, but our intuition is that despite measures may coincide, the existence of one type of HEI or another may have a direct impact not only on the educational performance of countries, but also in their economic and social development. From this perspective, further research could dig into the motivations for falling into institutional isomorphism, and the extent to which this isomorphism is also accompanied by an economic convergence, or whether an institutional isomorphism actually leads to an economic anisomorphism.

This review demonstrates to policymakers that the efficiency assessment is focused on a few functions and capabilities of HEIs. In particular, the review emphasizes the need to address broader analytical frameworks that help to avoid possible systemic failures that may arise due to the absence or excessive importance given to concrete functions and capabilities. Nevertheless, we acknowledge several limitations that leave room for further improvements in the approach. The functional framework has been developed inductively from the main lines of reasoning of the objectives of higher education policies and the perspectives of analysis of the efficiency of HEIs. To further consolidate the theoretical and practical foundations of this policy-oriented framework, deeper integration of the efficiency perspective and the functionalist approach would be necessary. In this regard, further research could provide methodological alternatives that integrate into the same analytical framework the analysis of the structural transformations of HESs and the changes in their operation.

Acknowledgments

The authors are grateful to the editor and two anonymous referees for the comments provided on an earlier draft of this paper. Dorys Y. Rodríguez-Castro is grateful to the Cátedra Unesco Banco Santander for the funding provided to develop her Ph.D. studies at the University of Deusto. Juan Aparicio thanks the financial support from the Spanish Ministry for Economy and Competitiveness (Ministerio de Economía, Industria y Competitividad), the State Research Agency (Agencia Estatal de Investigacion) and the European Regional Development Fund (Fondo Europeo de Desarrollo Regional) under grant MTM2016-79765-P (AEI/FEDER, UE).

References

Abramo, G., & D’Angelo, C. A. (2014). How do you define and measure research productivity? Scientometrics, 101(2), 1129–1144. http://dx.doi.org/10.1007/s11192-014-1269-8

Agasisti, T., & Dal Bianco, A. (2009). Measuring efficiency of higher education institutions. International Journal of Management and Decision Making, 10(5–6), 443–465. http://dx.doi.org/10.1504/IJMDM.2009.026687

An, Q., Yang, M., Chu, J., Wu, J., & Zhu, Q. (2017). Efficiency evaluation of an interactive system by data envelopment analysis approach. Computers and Industrial Engineering, 103, 17–25. http://dx.doi.org/10.1016/j.cie.2016.10.010

Aparicio, J., Pastor, J. T., Vidal, F., & Zofío, J. L. (2017). Evaluating productive performance: A new approach based on the product-mix problem consistent with Data Envelopment Analysis. Omega, 67, 134–144. http://dx.doi.org/10.1016/j.omega.2016.04.007

Azagra Caro, J. M. (2003). La contribución de las universidades a la innovación: Efectos del fomento de la interacción universidad-empresa y las patentes universitarias. Universitat de Valencia. PhD dissertation.

Berbegal Mirabent, J., & Solé Parellada, F. (2012). What are we measuring when evaluating universities’ efficiency? Regional and Sectoral Economic Studies, 12(3), 31–46. Retrieved from https://www.usc.gal/economet/reviews/eers1233.pdf

Benneworth, P., Pinheiro, R., & Sánchez-Barrioluengo, M. (2016). One size does not fit all! New perspectives on the university in the social knowledge economy. Science and Public Policy, 43(6), 731–735. http://dx.doi.org/10.1093/scipol/scw018

Brennan, J., Broek, S., Durazzi, N., Kamphuis, B., Ranga, M., & Ryan, S. (2014). Study on Innovation in Higher Education: Final report. European Commission Directorate for Education and Training Study on Innovation in Higher Education, Publications Office of the European Union, Luxembourg. ISBN 9789279350818. Retrieved from http://eprints.lse.ac.uk/55819/

Cuff, E. C., Sharrock, W. W., & Francis, D. W. (2006). Perspectives in Sociology (5th ed.). New York: Routledge.

Calvo, A., & Rodríguez, M. (2003). Análisis discriminante múltiple. In J.-P. Lévy & J. Varela (Eds.), Análisis multivariable para las ciencias sociales (pp. 251–276). Madrid, España: Pearson, Prentice Hall.

Castano, M. C. N., & Cabanda, E. C. (2007). Performance evaluation of the efficiency of Philippine Private Higher Educational Institutions: Application of frontier approaches. International Transactions in Operational Research, 14, 431–444. http://dx.doi.org/10.1111/j.1475-3995.2007.00599.x

Castells, M. (1993). The university system: Engine of development in the new world economy. In A. Ranson, S. Khoo, & V. Selvaratnam (Eds.), Improving Higher Education in Developing Countries (pp. 65–94). Washington D.C.: The World Bank.

Castells, M. (2001). Universities as dynamic systems of contradictory functions. In J. Muller, N. Cloete, & F. van Schalkwyk (Eds.), Castells in Africa: Universities & Development (pp. 35–55). Cape Town, South Africa: African Minds.

Chang, T. S., Tone, K., & Wu, C. H. (2015). Past-present-future Intertemporal DEA models. Journal of the Operational Research Society, 66(1), 16–32. http://dx.doi.org/10.1057/jors.2013.139

Cheng, G., & Wu, K. (2008). The internal efficiency in higher education: An analysis based on economies of scope. Frontiers of Education in China, 3(1), 79–96. http://dx.doi.org/10.1007/s11516-008-0005-7

Cherchye, L., De Rock, B., Dierynck, B., Roodhooft, F., & Sabbe, J. (2013). Opening the ‘Black Box’ of efficiency measurement – Input allocation in multi-output settings. Operations Research, 61(5), 1148–1165. https://doi.org/10.1287/opre.2013.1185

Cherchye, L., De Rock, B., & Hennebel, V. (2017). Coordination efficiency in multi-output settings: A DEA approach. Annals of Operations Research, 250(1), 205–233. http://dx.doi.org/10.1007/s10479-015-1892-7

Clermont, M. (2016). Effectiveness and efficiency of research in Germany over time: An analysis of German business schools between 2001 and 2009. Scientometrics, 108(3), 1347–1381. http://dx.doi.org/10.1007/s11192-016-2013-3

Cook, W. D., Liang, L., & Zhu, J. (2010). Measuring performance of two-stage network structures by DEA: A review and future perspective. Omega, 38(6), 423–430. http://dx.doi.org/10.1016/j.omega.2009.12.001

Cohn, E., Rhine, S. L. W., & Santos, M. C. (1989). Institutions of higher education as multi-product firms: Economies of scale and scope. The Review of Economics and Statistics, 71(2), 284. http://dx.doi.org/10.2307/1926974

De Witte, K., & López-Torres, L. (2017). Efficiency in education: A review of literature and a way forward. Journal of the Operational Research Society, 68(4), 339-363. http://dx.doi.org/10.1057/jors.2015.92

de Guzman, M. C. G. N., & Cabanda, E. (2009). Selected private higher educational institutions in Metro Manila: A DEA efficiency measurement. American Journal of Business Education, 2(6), 97-108. https://doi.org/10.19030/ajbe.v2i6.4092

Doty, D. H., & Glick, W. H. (1994). Typologies as a unique form of theory building: Toward improved understanding and modeling. The Academy of Management Review, 19(2), 230–251. http://dx.doi.org/10.2307/258704

Edquist, C., Zabala-Iturriagagoitia, J.M., Barbero, J., & Zofío, J.L. (2018). On the meaning of innovation performance: Is the synthetic indicator of the Innovation Union Scoreboard flawed? Research Evaluation, 27(3), 196-211. http://dx.doi.org/10.1093/reseval/rvy011

Emrouznejad, A., & Thanassoulis, E. (2005). A mathematical model for dynamic efficiency using data envelopment analysis. Applied Mathematics and Computation, 160(2), 363–378. http://dx.doi.org/10.1016/j.amc.2003.09.026

Etzkowitz, H. (2017). The Entrepreneurial University. In J. C. Shin & P. Teixeira (Eds.), Encyclopedia of International Higher Education Systems and Institutions (pp. 91-95). Netherlands: Springer. https://doi.org/10.1007/978-94-017-9553-1_17-1

Etzkowitz, H., Webster, A., Gebhardt, C., & Terra, B. R. C. (2000). The future of the university and the university of the future: Evolution of ivory tower to entrepreneurial paradigm. Research Policy, 29(2), 313–330. http://dx.doi.org/10.1016/S0048-7333(99)00069-4

Färe, R., & Grosskopf, S. (1997). Intertemporal production frontiers: With dynamic DEA. The Journal of the Operational Research Society, 48(6), 656–659. http://dx.doi.org/10.1038/sj.jors.2600779

Gralka, S. (2018). Stochastic frontier analysis in higher education: A systematic review. CEPIE Working Paper, No. 05/18. Retrieved from http://hdl.handle.net/10419/189968

Giuri, P., Munari, F., Scandura, A., & Toschi, L. (2019). The strategic orientation of universities in knowledge transfer activities. Technological Forecasting and Social Change, 138, 261–278. http://dx.doi.org/10.1016/j.techfore.2018.09.030

Hekkert, M. P., Suurs, R. A. A., Negro, S. O., Kuhlmann, S., & Smits, R. E. H. M. (2007). Functions of innovation systems: A new approach for analysing technological change. Technological Forecasting and Social Change, 74(4), 413–432. http://dx.doi.org/10.1016/j.techfore.2006.03.002

Ho, M., Liu, J., Lu, W., & Huang, C. (2014). A new perspective to explore the technology transfer efficiencies in US universities. Journal of Technology Transfer, 39(2), 247–275. http://dx.doi.org/10.1007/s10961-013-9298-7

Jackson, M. C. (2009). Fifty years of systems thinking for management. Journal of the Operational Research Society, 60(1), 24-32. https://doi.org/10.1057/jors.2008.176.

Johnes, G. (1998). The costs of multi-product organizations and the heuristic evaluation of industrial structure. Socio-Economic Planning Sciences, 32(3), 199–209. http://dx.doi.org/10.1016/S0038-0121(97)00035-9

Johnes, G., & Johnes, J. (2009). Higher education institutions’ costs and efficiency: Taking the decomposition a further step. Economics of Education Review, 28(1), 107–113. http://dx.doi.org/10.1016/j.econedurev.2008.02.001

Johnes, J. (2015). Operational research in education. European Journal of Operational Research, 243(3), 683–696. http://dx.doi.org/10.1016/j.ejor.2014.10.043

Kao, C. (2014). Network data envelopment analysis: A review. European Journal of Operational Research, 239(1), 1–16. http://dx.doi.org/10.1016/j.ejor.2014.02.039

Kitagawa, F., & Oba, J. (2010). Managing differentiation of higher education system in Japan: Connecting excellence and diversity. Higher Education, 59(4), 507–524. http://dx.doi.org/10.1007/s10734-009-9262-5

Kudła, J., & Stachowiak-Kudła, M. (2016). Quality of teaching and research in public higher education in Poland: Relationship with financial indicators and efficiency. Journal of Management and Business Administration. Central Europe, 24(4), 88–108. http://dx.doi.org/10.7206/jmba.ce.2450-7814.184

Laredo, P. (2007a). Toward a third mission for universities. Regional Seminar “Globalizing Knowledge: European and North American Regions and Policies addressing the Priority Issues of other UNESCO Regions”. 5-6 March 2007, UNESCO, Paris.

Laredo, P. (2007b). Revisiting the third mission of universities: Toward a renewed categorization of university activities? Higher Education Policy, 20(4), 441–456. http://dx.doi.org/10.1057/palgrave.hep.8300169

Lee, B. L., & Worthington, A. C. (2016). A network DEA quantity and quality-orientated production model: An application to Australian university research services. Omega, 60, 26–33. http://dx.doi.org/10.1016/j.omega.2015.05.014

Liu, J. S., Lu, L. Y. Y., Lu, W. M., & Lin, B. J. Y. (2013). A survey of DEA applications. Omega, 41(5), 893–902. http://dx.doi.org/10.1016/j.omega.2012.11.004

Martin, B. R., & Etzkowitz, H. (2000). The origin and evolution of the university species. Science and Technology Policy Research (SPRU), Electronic Working Paper Series, Paper No. 59. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.599.5719&rep=rep1&type=pdf

Molas-Gallart, J., & Castro-Martínez, E. (2007). Ambiguity and conflict in the development of “Third Mission” indicators. Research Evaluation, 16(4), 321–330. http://dx.doi.org/10.3152/095820207X263592

Molas-Gallart, J., Salter, A., Patel, P., Scott, A., & Duran, X. (2002). Measuring third stream activities: Final Report to the Russell Group of Universities. SPRU-Science and Technoly Policy Research (April), 85.

OECD (2017a). Benchmarking Higher Education System Performance: Conceptual framework and data. Paris: OECD Publishing. Retrieved from https://www.oecd.org/education/skillsbeyondschool/Benchmarking%20Report.pdf

OECD (2017b). Systems Approaches to Public Sector Challenges. Paris: OECD Publishing. Retrieved from https://www.oecd.org/publications/systems-approaches-to-public-sector-challenges-9789264279865-en.htm

Olivares, M., & Wetzely, H. (2014). Competing in the higher education market: Empirical evidence for economies of scale and scope in German higher education institutions. CESifo Economic Studies, 60(4), 653–680. http://dx.doi.org/10.1093/cesifo/ifu001

Philpott, K., Dooley, L., Oreilly, C., & Lupton, G. (2011). The entrepreneurial university: Examining the underlying academic tensions. Technovation, 31(4), 161–170. http://dx.doi.org/10.1016/j.technovation.2010.12.003

Porto Gómez, I. Zabala-Iturriagagoitia, J.M., & Aguirre Larrakoetxea, U. (2018). Old wine in old bottles: The neglected role of vocational training centres in innovation. Vocations and Learning, 11(12), 205–221. http://dx.doi.org/10.1007/s12186-017-9187-6

Potschin, M., & Haines-young, R. (2013). Conceptual frameworks and the cascade model. In M. Potschin & K. Jax (Eds.), OpenNESS Ecosystem Services Reference Book (pp. 1-6). Retrieved from: http://www.openness-project.eu/library/reference-book

Powell, B., Suitt, D., & Pearson, L. C. (2012). Expenditures, efficiency, and effectiveness in U.S. undergraduate higher education: A national benchmark model. The Journal of Higher Education, 83(1), 102–127. http://dx.doi.org/10.1353/jhe.2012.0005

Rhaiem, M. (2017). Measurement and determinants of academic research efficiency: A systematic review of the evidence. Scientometrics, 110(2), 581–615. http://dx.doi.org/10.1007/s11192-016-2173-1

Salmi, J. (2009). El desafío de crear universidades de rango mundial. Washington D.C. The World Bank. Retrieved from http://hdl.handle.net/123456789/1435

Salmi, J. (2017). The Tertiary Education Imperative Knowledge, Skills and Values for Development. Global Perspectives on Higher Education. [EPub] Rotterdam: Sense Publishers.

Sánchez-Barrioluengo, M. (2014). Articulating the “three-missions” in Spanish universities. Research Policy, 43(10), 1760–1773. http://dx.doi.org/10.1016/j.respol.2014.06.001

Sarrico, C. S., Rosa, M. J., Teixeira, P. N., & Cardoso, M. F. (2010). Assessing quality and evaluating performance in higher education: Worlds apart or complementary views? Minerva, 48(1), 35–54. http://dx.doi.org/10.1007/s11024-010-9142-2

Sav, G. T. (2016). Declining state funding and efficiency effects on public higher education: Government really does matter. International Advances in Economic Research, 22(4), 397–408. http://dx.doi.org/10.1007/s11294-016-9602-z

Schalk, J., Torenvlied, R., & Allen, J. (2010). Network embeddedness and public agency performance: The strength of strong ties in Dutch higher education. Journal of Public Administration Research and Theory, 20(3), 629–653. http://dx.doi.org/10.1093/jopart/mup018

Smits, R., & Kuhlmann, S. (2004). The rise of systemic instruments in innovation policy. International Journal of Foresight and Innovation Policy, 1(1–2), 4–32. http://dx.doi.org/10.1504/IJFIP.2004.004621

Thanassoulis, E., Sotiros, D., Koronakos, G., & Despotis, D. (2018). Assessing the cost-effectiveness of university academic recruitment and promotion policies. European Journal of Operational Research, 264(2), 742–755. http://dx.doi.org/10.1016/j.ejor.2017.06.046

Thanassoulis, E., Witte, K. De, Johnes, J., & Johnes, G. (2016). Applications of data envelopment analysis in education. In J. Zhu (Ed.), Data Envelopment Analysis. International Series in Operations Research & Management Science (pp. 367-438). Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7684-0_12

Tone, K., & Tsutsui, M. (2014). Dynamic DEA with network structure: A slacks-based measure approach. Omega, 42(1), 124–131. http://dx.doi.org/10.1016/j.omega.2013.04.002

Torraco, R. J. (2016). Writing integrative literature reviews: Using the past and present to explore the future. Human Resource Development Review, 15(4), 404–428. http://dx.doi.org/10.1177/1534484316671606

UNESCO Institute for Statistics. (2014). ISCED Fields of Education and Training 2013, Montreal: UNESCO Institute for Statistics. http://dx.doi.org/10.15220/978-92-9189-150-4-en

Weber, K. M., & Rohracher, H. (2012). Legitimizing research, technology and innovation policies for transformative change: Combining insights from innovation systems and multi-level perspective in a comprehensive “failures” framework. Research Policy, 41(6), 1037–1047. http://dx.doi.org/10.1016/j.respol.2011.10.015

Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52(5), 546–553. http://dx.doi.org/10.1111/j.1365-2648.2005.03621.x

Wieczorek, A. J., & Hekkert, M. P. (2012). Corrigendum to “Systemic instruments for systemic innovation problems: A framework for policy makers and innovation scholars.” Science and Public Policy, 39(6), 842. http://dx.doi.org/10.1093/scipol/scs094

Yang, G.L., Fukuyama, H., & Song, Y.Y. (2018). Measuring the inefficiency of Chinese research universities based on a two-stage network DEA model. Journal of Informetrics, 12(1), 10–30. http://dx.doi.org/10.1016/j.joi.2017.11.002

Zwaan, B. van Der. (2017). Higher Education in 2040. A Global Approach. (1th Ed.). [EPub], Amsterdam: University of Chicago Press. Retrieved from https://library.oapen.org/bitstream/handle/20.500.12657/31675/625978.pdf?sequence=1

Abstrakt

Cel: Artykuł przedstawia ramy funkcjonalne, które syntetyzują funkcje i możliwości, które obecnie kierują ocenami empirycznymi zidentyfikowanymi w literaturze. Metodyka: W artykule dokonano systematycznego przeglądu literatury przedmiotu, który rzuca światło na związek między modelowaniem produkcji uczelni a celami polityki szkolnictwa wyższego. Wyniki: Nasze wyniki dowodzą, że w modelach produkcyjnych stosowanych do pomiaru wyników instytucji szkolnictwa wyższego dominują cztery zależności między nakładami a wynikami. Jednak nasze wyniki wskazują na istnienie pewnych nierówności w pomiarze trzech misji uniwersyteckich. Implikacje dla teorii i praktyki: Przedstawione tutaj ramy funkcjonalne pokazują, że istnieje kilka rozbieżności między produkcją, która jest badana w ramach oceny wyników uczelni, a celami polityki szkolnictwa wyższego. Ma to istotne implikacje, zarówno dla środowiska akademickiego, jak i dla praktyki politycznej uczelni i HES, jeśli mamy osiągnąć uczciwą i sprawiedliwą reprezentację działań prowadzonych przez uczelnie i ich wielokrotny wkład w HES. Oryginalność i wartość: Przegląd ten podkreśla potrzebę zajęcia się szerszymi ramami analitycznymi, które pomogą uniknąć potencjalnych błędów systemowych, które mogą powstać z powodu braku lub nadmiernego znaczenia przypisywanego konkretnym funkcjom i możliwościom.

Słowa kluczowe: uczelnie, funkcje, wydajność, możliwości, przegląd literatury, integracja, efektywność, ramy funkcjonalne.

Biographical notes

Dorys Y. Rodriguez-Castro is a manager of science, technology and innovation at the public and university sectors. She received her doctorate in Business and Territorial Competitiveness, Innovation and Sustainability (2020) at the University of Deusto (Spain). For her predoctoral training, she obtained a scholarship from the UNESCO Chair financed by Banco Santander (Spain), and she was a visiting researcher at the Center of Operations Research (CIO) of the Miguel Hernández University of Elche (Spain). She is a system thinking practitioner, and her line of doctoral research is related to the systemic approach and its application through efficiency models. Her systemic models bring into focus the complexity and interrelated of the private sector of higher education and their sustainability.

Juan Aparicio is a Full Professor in Statistics and Operations Research at the University Miguel Hernandez (UMH), Elche (Alicante), Spain. He is the director of the Center of Operations Research. He earned his Ph.D. in Statistics in 2007 at the UMH. He is Co-Chair (with Knox Lovell) of the Santander Chair on Efficiency and Productivity. His research interests mainly include Efficiency and Productivity Analysis. He has published and co-edited several books focusing on performance evaluation and benchmarking using Data Envelopment Analysis and he has published more than 80 contributions in different international journals.

Conflicts of interest

The authors declare no conflict of interest.

Citation (APA Style)

Rodríguez-Castro, D.Y., & Aparicio, J. (2021). Introducing a functional framework for integrating the empirical evidence about higher education institutions’ functions and capabilities: A literature review. Journal of Entrepreneurship, Management and Innovation, 17(1), 231-267. https://doi.org/10.7341/20211718

---