Key research challenges we are facing in this project are associated with the development of the catalogue of indicators as well as metrics and tools, which could provide decision makers with more relevant, accurate, and multi-faceted analytical information for qualified and quantified preparation, implementation and evaluation of e-government policies (ex-ante or ex-post). Research is based on the hypothesis that existing methodologies for the evaluation of e-government policies do not provide enough valuable information to e-government policy makers in conducting quality decision-making. Past experience in the e-government field and public finance trends evidently require the development of the instruments which could assist better policy making in the e-government field and enable comprehensive evaluation of e-government policies and projects – be it before or after their implementation. The existing methodologies are undoubtedly applicable to the certain extent and can improve the quality of decision-making processes in the field; however, according to several limitations and deficiencies, they fail to facilitate the evaluation of e-government policies in an all-encompassing manner.
Up to date studies as well as available basic and applied research of e-government area facilitate extraction and synthesis of the key evaluation aspects and evaluation levels covered by the existing methodologies. Below are described accredited evaluation aspects and the research characteristics concerning each particular aspect:
- Evaluation of transformational effects: changes in back-office, the reduction of hierarchical levels, business process reengineering, outsourcing, reduction of administrative barriers, costs and burdens etc. (organizational aspect);
- Evaluation of infrastructure investments: costs of ICT infrastructure, data infrastructure, human resources, legal framework etc. (infrastructural aspect);
- Evaluation of political and sociological effects: transparency, openness, corruption, user satisfaction, democratization, participation etc. (political-sociological aspect);
- Evaluation of economic and sustainability impacts: costs, public benefits, effects on GDP, competitiveness index, economic growth, sustainable development etc. (economic and sustainability aspect).
Next to the categorization of the methodologies regarding different evaluation aspects, the methodologies can be classified according to the evaluation levels as well. During the research five evaluation levels have been extracted and outlined:
- Territorial-administrative unit level of the evaluation refers to the territorial / administrative unit being observed (e.g. particular level within the country; region, federal state, particular level between countries; cross-border regions and series of countries or country members of transnational organizations such as EU, OECD and UN).
- Sector policy level of the evaluation encompasses various policies which are the subject of the evaluation (e.g. tax policy, environmental policy, research policy etc.).
- Program level of the evaluation focuses on programs which usually present groups of program-related projects (e.g. e-health and e-learning program).
- Organization level refers to the different levels of organizations (e.g. the level of state agency, ministry or its organizational unit/s, the level of municipality or any other public administration body).
- Project level of the evaluation covers projects, and services resulting from the projects (e.g. informatization of the process/service for establishment of a business).
Comprehensive evaluation of e-government policies requires taking into account the complex structure of e-government and the intricacy of policy process itself. Proposed conceptual model for the construction of the catalogue of indicators for the evaluation of e-government policies attempts to capture a wide range of relevant factors, which is reflected in broadly defined aspects of evaluation, while it should also allow the evaluation of e-government policies at different levels. The conceptual model lays the theoretical and practical foundations for the development of the comprehensive catalogue of indicators for the evaluation of e-government policies (Figure 1).
Figure 1: The conceptual model for the construction of the catalogue of indicators.
The presented conceptual model facilitates a comprehensive as well as targeted (focusing on individual evaluation aspect or level) evaluation, and enables more transparent and reliable evaluation process. The basic scheme for the development of the catalogue of indicators for the evaluation of e-government policies is derived from the presented conceptual model (Figure 1). Quality catalogue of indicators for the evaluation of e-government policies should comprise the indicators which are clearly defined, complementary to each other, measurable in practice and upon which it is possible to conduct a transparent and consistent evaluation process. Several sets of indicators for the evaluation of e-government policies were identified during the research; however, in order to provide a reliable support in the evaluation process, the indicators have to be adapted to the evaluators’ needs, institutional ecosystem and other e-government conditions.
The developed catalogue of indicators for the evaluation of e-government policies is now available to the expert and general public. It identifies all the relevant indicators, their object and unit of measurement, history, structure, context etc. The structure of the presented catalogue of indicators implies that designated categories are not definitive and that their number and contents could be changed and adapted to the specific needs of the individual evaluators and environmental circumstances in which the evaluation of e-government policies takes place. A catalogue of indicators allows alteration of the primary sets of indicators, optional selection of the operational indicators and a different layout and definition of the evaluation aspects and levels within its structure.