Consultant Performance and Selection System

Size: px

Start display at page:

Download "Consultant Performance and Selection System"

Derek Gregory
5 years ago
Views:

1 Consultant Performance and Selection System Procedures Guide MTO Qualification Committee September 2015

2 Table of Contents Page Summary 1 Consultant Performance and Selection System (CPSS) 2 Corporate Performance Rating (CPR) 2 Starter CPR 3 Performance Appraisals 4 Consultant Joint Ventures 5 CPSS Application at the Expression of Interest (EOI)/Phase I Stage 6 CPSS Application at the Request for Proposal (RFP)/Phase II Stage 7 CPSS Application at the Request for Quotation (RFQ) Stage 8 Consultant Performance Appraisal Reviews 9 Consultant Infraction Reports 11

3 Summary Effective January 1, 2001, the Ministry of Transportation (MTO) modified its consultant selection procedures to take past performance into account at the Expression of Interest and Request for Proposals / Quotation (RFP / RFQ) stages. The objective is to improve the quality of engineering and construction administration services and reduce infrastructure life cycle costs. The revised consultant selection system is referred to as the Consultant Performance and Selection System (CPSS). Past performance is measured by a consultant's Corporate Performance Rating (CPR) that is the weighted-average of a consultant's appraisals over the last three years. A consultant s Corporate Performance Rating (CPR) is available for Planning, Engineering, Construction Administration, Area Materials Testing and Small Value Assignments. A Small Value Assignment is an assignment with an Acquisition Value of $25,000 or less. Effective October 15, 2000, a Consultant Appraisal Review process is available, as: - If a consultant firm disagrees with a performance appraisal received, they may request a review by the appropriate ministry regional manager, within 21-calendar days (unless 30-calendar days specified in the signed Legal Agreement). - If the matter is not resolved at the regional manager level, the consultant may request a review of the appraisal by Qualification Committee (QC). Effective September 1 st, 2015 the Qualification Committee Procedures and the Procedures for Processing the Consultant s Infraction Report have been revised. 1

4 Consultant Performance and Selection System (CPSS) Effective January 1, 2001, the ministry modified its consultant selection procedures to take past performance into account at the Expression of Interest (EOI)/Phase I and at the Request for Proposals / Quotation (RFP/RFQ) stages. The objective is to improve quality of Engineering and Construction Administration services received and thereby reduce road user costs and infrastructure life cycle costs. The revised consultant selection system is referred to as the Consultant Performance and Selection System (CPSS). Corporate Performance Rating (CPR) Past performance is measured by a consultant's Corporate Performance Rating (CPR), which is the weighted-average of a consultant's appraisals over the last three years. The CPR of a consultant firm is calculated, by the following equation: CPR = 3(Avg.Yr. 1) + 2(Avg. Yr. 2) + 1(Avg. Yr.3) 6 Avg Yr.1 = Average of all appraisals within the most recent 12 months Avg Yr.2 = Average of all appraisals in 12 months prior to Year 1 Avg Yr.3 = Average of all appraisals in 12 months prior to Year 2 A consultant s Corporate Performance Rating (CPR) is available for Planning, Engineering, Construction Administration (CA), Area Materials Testing and Small Value Assignments appraisals. A Small Value Assignment is an assignment with Acquisition Value of $25,000 or less. If a firm does not have any performance appraisals in one or two of the three years, the formula will self-adjust accordingly, to calculate the CPR based on the year(s) for which performance appraisals are available for the firm. The ministry's Registration, Appraisal and Qualification System (RAQS) automatically calculates CPR on a quarterly basis, for each consultant, using past performance appraisals available (at the end of the day, on the first Business Day of January, April, July and October of a year). Only approved performance appraisals are included in the CPR calculation. An appraisal is "approved" if the consultant signs off the Performance Appraisal Form or does not respond within the 21-day time 2

5 limit (to request a formal review). In case of a request by a consultant for a formal review, the appraisal is not considered approved until the completion of the regional manager review stage or the Qualification Committee review stage, depending on how far the consultant chooses to proceed with the review. At each CPR calculation, performance appraisals with the approval date older than the past thirty-six months are dropped from the calculation. For calculating one of the above CPRs, the corresponding appraisals are applied. For example, only CA appraisals are be used to calculate the CA CPR of a firm. A consultant s CPR based on Planning, Engineering, CA, Area Materials Testing or Small Value Assignments appraisals is applied for selection for the corresponding assignment type. The ministry staff is responsible to upfront identify the assignment type. Where an assignment includes more than one Grouping, a joint CPR is applied for consultant selection. For example, for a Total Project Management (TPM) assignment, which includes Engineering and CA, a joint CPR based on both Engineering and CA appraisals will be used for selection. To allow CPR application for both the stand-alone and joint assignments, the following quarterly CPRs are made available in RAQS, as: Starter CPR - Separate CPR s for Planning, Engineering and CA. - Joint CPR for Planning and Engineering. - Joint CPR for Engineering and CA. - Joint CPR for Planning, Engineering and CA. - Separate CPR for Area Materials Testing Assignments. - Separate CPR for Small Value Assignments. If a firm does not have any past appraisals in the Planning, Engineering or CA Groupings, the firm is assigned a Starter CPR for that Grouping. A Starter CPR for a Grouping is equal to the ministry wide average of all approved appraisals in that Grouping. A Starter CPR is calculated every quarter and may change from quarter to quarter. A Starter CPR is available to a new registered firm or to a registered firm venturing into a new area. A Starter CPR allows a registered firm with no past appraisals to compete for assignments. A Starter CPR is no longer available once a firm has one approved appraisal in RAQS in that Grouping. The firm s CPR will then be based on their approved appraisal(s) in the Grouping. 3

6 Performance Appraisals When a consultant assignment is completed, an appraisal will be completed for the prime consultant / consultant who has signed the legal agreement with the ministry. Appraisals will not apply to sub-consultants. In the case of consortiums or legal partnerships, one overall performance appraisal rating for the assignment will be completed and the rating will apply to each member of the consortium or partnership. When joint assignments involving Planning and Engineering are completed, a consultant will receive two performance appraisals, one for the Planning and the second for Engineering. For a Total Project Management assignment involving Detailed Design & Contract Administration (CA), a consultant will receive two separate performance appraisals, one for the Detailed Design and the second for the Contract Administration. Those appraisals, when approved, will apply to calculate the corresponding CPRs for a firm. For multi-year assignments, an annual appraisal will be completed near the end of each year, to provide feedback to the consultants until the final appraisal is completed. Where both interim / annual and final appraisals are approved and available in RAQS for the same assignment, only the final appraisal will be used to calculate CPR. For assignments of duration two years and greater where RFP or RFQ is issued on/after October 1, 2007, Interim, Annual and Final Appraisals for assignments will apply in CPR and will be subject to the Formal Review/Appeal and Approval process, if requested by a consultant. - For an assignment of duration two years or greater but less than two and half years, an Interim Appraisal will be completed at 12 months and Final Appraisal within sixty days of completion of the assignment. - For a multi-year assignment, Annual appraisal will be completed near the end of each calendar year until the Final appraisal is completed. Annual and Final Appraisals issued will be subject to approval and included in CPR calculation. - For assignments of duration less than two years, only Final Appraisal will apply in CPR and will be subject to Formal Review/Appeal process. - For all assignments issued prior to October 1, 2007, only Final Appraisal will apply in CPR and will be subject to Formal Review/Appeal process. 4

7 Where any approved interim and final appraisals are available in RAQS for the same assignment, the final appraisal will supersede any earlier interim appraisal and used to calculate CPR. In the case of annual and final appraisals approved in RAQS, the later annual or final approved appraisal will supersede any earlier annual appraisal(s). An approved Post-Construction Engineering Appraisal will apply in the calculation of Engineering CPR of a consultant as one of the appraisals together with other approved Engineering appraisals. A Post-Construction Engineering Appraisal will apply for the period of three years in the above CPR formula and will be dropped after three years. Consultant Joint Ventures A Consultant Joint Venture is treated as an Equal Partnership of two or more consultant firms. The consultants wishing to undertake an assignment as a Joint Venture are to provide a Letter of Intent jointly signed by Senior Partner / CEO of each firm, as part of their EOI/Phase II submission. The Letter of Intent states: - The firms will be working as Joint Venture for the purposes of the assignment. - The Specialties / areas of work that each individual firm will be responsible for. - At least one of the firms meets the RAQS Registration Requirements for Prime Specialty. - Identifies the lead firm and Ministry s contact for the purposes of an assignment. Each firm must submit the declarations at EOI/Phase I Stage that have no conflict of interest with the assignment. At the RFP/Phase II Stage, each firm must declare that they have no conflict of interest with the assignment, are tax compliant with the Ministry of Finance. Also each firm must provide the proof of insurance. For the purposes of an assignment, the ministry will deal with the Lead Firm as the ministry s contact. For the purposes of an assignment, CPR for a Joint Venture is calculated using the approved appraisals of all firms in a joint venture and applying the formula. 5

8 At the award of an assignment, Agreement will be signed by and will be binding on all the firms in a joint venture. All Provisions and obligations of Legal Agreement will apply equally to each joint venture firm for the purposes of the assignment. CPSS Application at the Expression of Interest (EOI)/Phase I Stage The EOI/Phase I submissions and past performance (CPR) each are scored out of 100 points. The highest EOI/Phase I score and the highest CPR are each assigned a score of 100 and the other values are pro-rated lower Weights of 50% each are applied to both the EOI/Phase I and past performance scores. The weighted EOI/Phase I and past performance (CPR) scores are added, and the total is used to rank the applicants. Based on the ranking, the ministry will short-list three to five consultants for the next step. An example of application of CPSS at the EOI/Phase I stage is illustrated below: EOI/Phase I Submission s Firm A B C Project Organisation (Max. 20) Project Team Experience & Performance (Max. 80) Total out of Weighted (50%) Past Performance s Corporate Performance Rating (CPR) out of Weighted (50%) Total Weighted s Total Weighted Ranking

9 CPSS Application at the Request for Proposals (RFP)/Phase II Stage The RFP/Phase II submissions, CPR and prices are scored out of 100 points max for each. The highest RFP/Phase II score, the highest CPR and the lowest price each are assigned a score of 100 and the other values are pro-rated lower. Effective July 28, 2006, the following weights of technical scores, past performance (CPR) and price apply for selection: EOI: Technical = 50%, Past Performance (CPR) = 50% RFP: Technical = 40%, Past Performance (CPR) = 50%, Price = 10% RFQ: Past Performance (CPR) = 50%, Price = 50% The weighted RFP, past performance (CPR) and price scores are added, and the total is used to rank the applicants. The highest Total Weighted wins the assignment. An example of the application of CPSS with the revised weights at the RFP/Phase II stage is illustrated below: Firm RFP Tech. out of 100 Wght (40%) Perf. Rating (CPR) out of 100 Wght. (50%) Price A B C D E out of 100 Wght (10%) Total R a n k If the Total s for two or more consultants are tied, the consultant with the lowest price wins the assignment. 7

10 CPSS Application at the Request for Quotation (RFQ) Stage The CPR and price are scored out of 100 points maximum for each. The highest CPR and the lowest fee each are assigned a weight of 100 and the other values are pro-rated lower. Weights of 50% each are applied to the CPR and price scores and the scores are converted. The weighted past performance (CPR) and price scores are added, and the total is used to rank the applicants. The highest Total Weighted wins the assignment. An example of the application of CPSS at the RFQ stage is illustrated below: Firm Price out of 100 Wght Price (50%) Perf. Rating (CPR) out of 100 Wght Perf. Rating Total Wght A 80, B 60, C 70, D 100, E 95, Rank If the Total s for two or more consultants are tied, the consultant with the lowest price wins the assignment. 8

11 Consultant Performance Appraisal Reviews Review Level One: Manager of Engineering / Construction Within the twenty one (21) calendar days of the transmission date of an appraisal, the consultant may submit a request for a review, based on substantial and reasonable grounds, in writing to the regional / office Manager, with the supporting documentation provided. For some assignments, the time limit of thirty (30) calendar days if specified in the signed legal agreement will apply. The Regional Manager will investigate the request for the appraisal review. The Manager may interview / meet with consultant to discuss the appraisal and the supporting documentation submitted by the consultant. The Manager will advise the consultant on the decision. The copies of the decision and the pertinent documentation will be forwarded the Secretary, Qualification Committee (QC). If the consultant firm accepts the decision of the Regional Manager, or if the consultant does not respond within 21 calendar days (30 calendar days if specified by the signed Legal Agreement) of the transmission of the decision, the appraisal rating decided upon is considered final. In case the consultant accepts the decision within 21 calendar days (30 calendar days if specified by the signed Legal Agreement), the effective date of the appraisal will be date of consultant signoff. In case the consultant does not respond, the effective date for the appraisal will be 21 calendar days after the transmission of the Regional Manager s decision. The rating of the appraisal decided by the Regional Manager is applied in the firm s CPR at the next quarterly CPR calculation. If the Regional Manager's decision is not accepted, the consultant may request a review by Qualification Committee (QC), within 21 calendar days of the transmission of the Regional Manager's decision. 9

12 Review Level Two: Qualification Committee Within the deadline specified, the consultant may submit for a Review Level Two, in writing, based on substantial and reasonable grounds, to the Secretary, Qualification Committee (QC). The consultant must identify the sections of the performance appraisal that are disputed, and give detailed reasons with the supporting documentation provided. Qualification Committee (QC) will consider a Review of an appeal for one or more of the following reasons: - The ministry s issuing office has not followed the prescribed process for the performance rating - There is new or additional information to support the request for Review Qualification Committee (QC) will consider any new information provided by the consultant if relevant and not considered by the Regional / Office Manager but supports a challenge of a decision made at the Level One Review. Qualification Committee (QC) will not consider the sections of the performance appraisal that were not disputed in the request for the Regional / Office Review. Qualification Committee (QC) will instruct the appropriate Functional Office in Head Office to independently investigate the request, based on the information in the ministry files and the supporting documentation provided by the consultant. The investigator may request further information from the consultant or the ministry Office regarding the performance appraisal issued. The investigator prepares an appraisal review and forwards it to the Secretary of Qualification Committee (QC). The appraisal review is discussed by Qualification Committee (QC), at the next scheduled meeting. Both the consultant and the Regional Office are informed on the decision of Qualification Committee (QC) and the decision is immediately implemented. The rating of the appraisal decided by Qualification Committee (QC) is applied in the firm s CPR at the next quarterly CPR calculation. 10

13 Consultant Infraction Reports (CIRS) Please refer to the Procedures for Processing the Consultant s Infraction Report (September 2015). 11

MINISTRY OF TRANSPORTATION

MINISTRY OF TRANSPORTATION CONSULTANT PERFORMANCE AND SELECTION SYSTEM (CPSS) Procedures Guide Qualification Committee September 2017 TABLE OF CONTENTS Page Executive Summary 1 Consultant Performance and