A FRAMEWORK FOR EVALUATING PROJECT MANAGERS ' PERFORMANCE-IDENTIFICATION AND ANALYSIS OF KPIS IN SUBWAY CONSTRUCTION PROJECTS IN TEHRAN

Background/Objectives: One of the major issues for project management in a project environment such as subway construction project in Tehran, is the poor project management performance analysis. PM’s performance plays a major role in the success of construction projects and the identification of proper Key Performance Indicators (KPIs) can lead the project to success. Methods/Statistical analysis: In spite of advances in project management, these indicators have not yet been identified for subway construction projects. This paper reports a research study that identifies the KPIs in Tehran subway construction projects and investigates their use as a tool for evaluating the project manager’s performance. Findings: Statistical analysis of the success factors and use of key performance indicators has revealed that 3 of the indicators have the greatest impact on PM success. These indicators include: human resource management, communication management and procurement management. This suggests that PMs who work in Tehran Metro construction projects, and have been successful in managing projects have taken more account of these performance indicators than other factors. Applications/Improvements: It can be stated that the subway construction PMs studied can improve their performance by focusing more on human resource, communication, and procurement management and increase the chances of project success. These three factors can be very helpful making success in subway construction projects.


INTRODUCTION
An increasing number of organizations are implementing their business operations through projects 1 .Subway construction projects in Tehran are so essential in view of the city's traffic congestion and the success of this projects is vital for sustaining economic growth and social well-being.The ability to travel inside the city, connecting the suburb to the center of Tehran with proper speed, accurate travel and high-tech safety, reduction of environmental pollution, creation of a calm and relaxed social atmosphere, optimization of urban transportation, decreasing citizens time wastes and reducing accidents, are some of quality advantages of using subways 2 .
Although performance of the project managers (PMs) plays a major role in this, the current project management practices of construction industry sector organizations do not always ensure project success.The success of a construction project greatly depends on how the project has been managed and controlled.The main problem associated with project management practices have always been identified as planning, project implementation, cost and time overruns and non-achievement of quality 3 .Several studies have shown that a set of key performance indicators (KPIs) can be used to assess the performance of the projects, companies, and project managers and, so the purpose of this paper is to identify the KPIs which can lead subway construction projects to success and use them to evaluate PM performance.

LITERATURE SURVEY PROJECT MANAGEMENT PERFORMANCE
According to A Guide to the Project Management Body of Knowledge 4, 5 , Project management is the application of knowledge, skills, tools, and techniques to project activities to meet the project requirements.Traditional PM systems which exclusively pursue the success criteria of cost, time, quality and meeting technical requirements have become considered ineffective 6 .PMP assessment has evolved over the past couple of decades as both researchers and practitioners have attempted to identify the causes of the project failure and the various factors that lead to success 7 .For example, Fortune and White 8 citied critical PMP success factors as 'support from senior management', 'clear realistic objectives', 'strong and clear detailed plan kept up to date', and 'good communication and feedback'.
As a result, a new set of difficulties has been encountered in developing models for measuring performance because stakeholders' needs are often difficult to manage and measure 9 .There is sometimes also resistance to moving beyond the traditional criteria because of commercial pressures 10 .These difficulties have resulted in limited literature on more holistic performance assessment frameworks for project environments.
There is some degree of consensus among certain researchers that the EFQM model, based on philosophy of TQM, can be an effective performance assessment model 11,12 Project Excellence Model (PEM) is an adaptation of this model that links project success criteria with critical project success factors 6 .Bryde 13 has propose the project management performance assessment (PMPA) model that was based on the EFQM and links TQM and PM practices.In place of the nine criteria used in the EFQM model, the PMPA model consists of five high PM performance enablers: PM leadership, PM staff, PM policy and strategy, PM partnerships and resources and project life cycle management process.The final area in the PMPA comprises the PM Key Performance Indicators (KPIs) which represent the practices by which actual achievement is measured.The PMPA model is presented in Figure 1.
The success of a project has traditionally been though to run on PM performance with an emphasis on the achievement of time, cost and quality targets.It was often argued that these measures were overly simplistic for gauging managers' performance in the context of today's construction industry where.Many other factors come into play.In order to explain the relationship between project performance and PM performance, Toor and Ogunlana 14 presented a conceptual model in which they divided project management into two domains, namely of process and performance.The process domain deals with project objectives, devising an adequate project management system, and the delivery of product during the input, process, and outcome stages, respectively.The performance domain, in contrast, focuses on performance goals, the establishment of a performance enhancement strategy, and performance measurement during the input, process and outcome stages, respectively.Toor and Ogunlana 14 note that performance measurement can be carried out by establishing KPIs which offer objective criteria to measure a project's success.

RELATIONSHIP BETWEEN PM PERFORMANCE AND PROJECT SUCCESS
The traditional view of project success is associated with fulfilling time, cost and quality objectives (the iron triangle).Financial criteria have been used to measure project performance, including economic return and cost/benefit analyses and profits.Another way to evaluate the benefits of PM is to analyze the margins of a company's ongoing projects.
The main purpose of using a project management framework is to increase organisational value.The organisation can benefit from using project management framework by increasing the effectiveness of human effort in the organisation while increasing the efficiency of these efforts.Therefore, project success is measured by its efficiency in the short term and its effectiveness in achieving the expected results in the medium and the long term 15,16 .
The PM literature argues that a positive relationship exists between PM Performance and Project Success (PS) claiming the latter is dependent on appreciation of the importance of PM.They further emphasis that this role must be considered in terms of the wider organizational strategy and long-term expectations 6 .It has been argued that both PS and PM Performance are distinct yet inter-related concepts, and a positive relationship between them is sought.
A systematic PM consists of methods, toolkits and models.It can be viewed as the sequential application of structured processes for the purpose of institutionalizing standardized practices.Using a well-structured and well-implemented approach, capabilities can be stored and transferred over time, space and context.Additionally, PM can make organizations less vulnerable to the loss of tacit knowledge stored in individual memories.Project manager is a critical success factor (CSF) in a project, providing direction, goals, motivational support and assistance in resolving interpersonal and organizational issues 17 .
Farzana and Pinnington 6 concluded that [Management of] PM KPIs is the most significant individual variable contributing towards the success of any project.This suggests that if there is PM performance measurement in an organization it can significantly impact on Project Success.The PM literature advocates defining the targets and using measures to achieve required results.Therefore, methods should exist in an organization to formally develop these KPIs.Organizations should have a system that ensures that KPIs are developed from the perspective of all stakeholders and encompass not only short-term benefits (for example, meeting cost, time, quality objectives of current projects), but also the long-term benefits for the organization like learning and continuous improvement.
In this paper, to identify Successful managers and assess both KPIs and individual characteristics influencing their success, four indices relating to the PMs' previous projects have been studied.Conducting 27 interviews with academics and project managers, it was concluded that each of the managers who scored more than 60% (3 out of 5) would be considered as a successful manager.

KEY PERFORMANCE INDICATORS IN CONSTRUCTION PROJECTS
The KPIs represent a set of measurable data used for evaluating and measuring performances in implementation phase 18 .One of the challenges for contemporary project managers is to determine which critical measures will guarantee project success for all stakeholders.A project manager is to define measures and KPIs based on the partner relations between project manager, client and other stakeholders.
KPIs are compilations of data measurements which are used to assess the performance of a construction operation.In order to measure performance or calculate the effects of any given change on the construction process, one must first determine the appropriate KPIs to focus on in order to measure its impact 19 .Despite extensive research, up until now there has been no general agreement on a single set of KPIs for construction projects, and so there is a need for identifying a common set of indicators to be used by construction executive and project managers in measuring construction performance at the project level 19 .Toor and Ogulana 14 note that performance can be measured by establishing KPIs which offer objective criteria for the success of a project.Hence, this study focuses on finding specific KPIs which can lead subway construction projects towards success, and seeks to validate a framework to measure the PM's performance of these projects.Knowing what performance should be measured, how the performance can be measured, and what the measurement results mean is fundamental to the selection of KPIs.This knowledge is used to avoid the use of improper indicators that may lead to the bias in measurement results.A total of 9 potentialities are summarized from the literature survey to form the basis for identification of the most important indicators.They will be investigated in the later research and are listed in Table 1.

METHOD QUESTIONNAIRE DESIGN
A structured questionnaire survey method was selected to identify the KPIs leading to the success of subway construction projects.The KPIs were adapted from peer reviewed publications in the PM research area.The first section obtained useful descriptive data about respondents.The next section dealt with PM Performance and asked respondents to state how they perform in diverse areas.In section three, data was elicited about the success in the context of subway construction projects recently completed within the organization.The questionnaires distributed among project directors and the researcher attended all of the construction sites personally and conducted face to face interviews in order to explain the purpose of the questionnaire, to ensure that the questionnaire was completed by the director or his/her deputy, and to maintain the confidentiality of personal information of the project managers.
The questionnaire contains questions related to six key performance indicators, each of which is questioned by three items so that a total of 18 questions are created.As previously mentioned, these indicators are derived from different resources.Answers are closed and a Likert Scale from 1 to 5 is used.Before responding to the key questions, the project manager should answer four questions about personal characteristics which include age, education, work experience in the field of civil engineering, and subway construction projects.Finally, questions relating to the success of the project manager in previous projects are asked in order to distinguish successful managers.

VALIDITY AND RELIABILITY Reliability:
Reliability was investigated for each construct using Cronbach's alpha.Based on the results, 3 potential KPIs were removed from the PM Performance questionnaire.After eliminating these questions, the alpha coefficient for all scales was above the acceptable threshold level of 0.7.All the Cranach alphas for variables within Project Success were above the acceptable value of 0.7.Cronbach alpha of the whole survey tool after eliminating 3 questions was 90.1%.These results confirmed the appropriateness of further analysis of the obtained data.

Validity:
Both internal and external validity were considered for the survey.As the selection of the initial measurement items (KPIs) was based on a review of the theoretical and empirical literature, it is vital to assess internal validity.A pilot questionnaire test distributed among six potential participants, and they were asked via interview to complete a questionnaire and to present a critique of the questions.Some of the changes suggested by the participants in this survey were incorporated in the final questionnaire.The final questionnaire comprised 6 KPIs divided in 18 questions within 3 sections.

STUDY SAMPLE
The target population for this survey are PMs of subway construction projects in the metropolis of Tehran, a group of individuals who share a common trait or traits.Such populations can be divided into two groups-limited and unlimited-according to the constituent members: since the number of Tehran subway PMs working in the construction industry is estimated to be about 52 people, so the target population is limited.
In order to examine the characteristics of this population, namely the statistical community of the project managers, the sample should be selected that is representative of the features and characteristics of the community and selected in a way determined by the.As a result, any information about the population can, more or less, be retrieved from the sample.In this study, the sample comprises the majority of PMs who work in Tehran subway construction projects and a random selection method was used to select the sample.For this the researcher moved randomly around the city of Tehran to places where, according to the TUSROC information, a subway construction site was working and selected the managers.In order to obtain the best results, All 52 Tehran subway construction PMs were evaluated and questionnaires were distributed among all of them.

DESCRIPTIVE STATISTICS OF THE DATA COLLECTED
The average age of the PMs is equivalent to 33 years with a standard deviation of 6 years.The oldest PM is 45 years old and the youngest one is 23.Over 80% of the managers are less than 39 years of age.
The average work experience of the PMs in the field of civil engineering was 9.8 years with a standard deviation of 5 years.Two managers with 20 years, and one manager with 1 year experience in the field of civil engineering, are the most and least experienced managers in the field, respectively.The average work experience of PMs in project management and subway construction projects is about 6 years and a standard deviation of this variable is approximately 3.2 years.Most managers have 6 to 9 years work experience.

EFFECT OF PERSONAL CHARACTERISTICS ON MANAGERS' SUCCESS
Given the amount of the p-value from non-parametric Kruskal-Wallis test results we can concluded that, among the factors to consider, only the PMs' experience in the field of civil engineering has an impact on their success rate as shown in

EFFECTIVE KPIS IN PROJECT MANAGERS' SUCCESS
According to the questionnaire, each of the functional factors is the outcome of 3 interrelated questions.So if each of the questions is related to any indicator, it can affect the success of the project managers (figure 2).According to the values obtained from the tests, and comparison with a significance level of 0.1, some of the null hypotheses of questions are rejected, and others accepted.Thus, since the P-Value of functional factors of 10, 11, 13, 14, 16 and 17 are less than 0.1, it can be concluded that the following factors have affected the success of the Tehran Metro construction PMs (table 3):

•
Adequate and timely distribution of information among stakeholders

PM'S PERFORMANCE EVALUATION FRAMEWORK
Finally, an attempt has been made to use the foregoing information to provide a system for evaluating the performance of subway construction PMs.As noted above, performance appraisal systems are usually developed for a particular organization with a specific organizational culture using the expertise of professional's over several years.
The present system is an operational framework as shown in Figure 3.
Step 1: The PM performance data enters the system and is processed using various tools, techniques, such as expert judgment and key performance indicators.
Step 2: Performance is examined using the KPIs that have already been extracted.The three indicators for human resources management, communication management and procurement management are the main factors in the statistical analysis that show the greatest impact on whether PMs are successful Criterion (1): According to experts, if PM performance shows a score of more than 3 in the second step, then this criterion is fulfilled.
Step 3: At this stage, the performance of the PM is evaluated and should be reported.Corrective action should be done if required.

Figure 3. PM's performance evaluation framework
Finally, it can be stated that the subway construction PMs studied can improve their performance by focusing more on human resource, communication, and procurement management and increase the chances of project success.These three factors can be very helpful making success in subway construction projects.

CONCLUSION
Performance evaluation is one of the most important aspects of project management and Identification of the key performance indicators that affect the success of PM can lead to success of the project.Statistical analysis of the success factors and use of key performance indicators has revealed that 3 of the indicators have the greatest impact on PM success.These indicators include: human resource management, communication management and procurement management.This suggests that PMs who work in Tehran Metro construction projects, and have been successful in managing projects have taken more account of these performance indicators than other factors.This suggests that certain factors could pave the way to success for a project manager, such as use of a suitable contract when selecting vendors, developing a precise SOW and workable source selection criteria, training and appraisal of the staff, use of written job descriptions, adequate and timely distribution of information among stakeholders, determining communication channels before starting the project, and holding meetings and correspondence effectively to resolve conflicts.
Finally, future evaluation systems need to be more comprehensive in view of the need for success in construction projects, since the city of Tehran is growing dramatically.

Figure 2 .
Figure 2. Null hypothesis at a significance level of 0.1 Kerzner H. Project Management Metrics, KPIs and Dashboards.John Wiley and Sons, New Jersey,  2011.

Table 1 .
KPIs extracted from literature survey References

Table 2 Table 2 .
The values obtained by the Kruskal-Wallis one-way analysis of variance p-value at a significance level of 0.05

Table 3 .
Key performance indicators used in the questionnaire