Icao Doc 10056
Hasan Fogg
2.1.1 For decades, individuals have been involved in the assessment of work performance of others. In the 1950s companies used personality-based systems to measure performance, quickly followed by more fair performance appraisals that were actually measuring if objectives and goals were met.
2.2.1 The integrated concept of competence as a holistic set of knowledge, skills and attitudes was developed in the 1990s. Historically too much emphasis had been placed on determining whether trainees could pass exams and insufficient emphasis on whether they could perform in the role expected. A good assessment system does not assess each attribute separately with a checklist, but is integrated and goes well beyond solely task skills.
2.2.3 The view on competence assessment as an integrated and ongoing process shaped the thinking about performance appraisal in aviation, and can be derived from the guidelines on competence assessment from ICAO (DOC 10056) and Eurocontrol. The regulations and guidance will be explained in paragraph 2.4.
2.4.1 ICAO Annex 1, the regulative framework for personnel licensing, describes mostly how ATCOs shall be trained to initially receive their license. On renewing the license, the manual does not have many prescriptions. The only standards are related to a number of hours or months (ICAO Annex 1, (4.5.2.2.1 b) 1) and 2)). ICAO aims to address this issue in Amendment 6, from the year 2020.
When establishing the validity of a unit endorsement, unit standards of performance and seasonal variations should be taken into account. Appropriate means should be in place to monitor the competence of the air Traffic controllers. The means should be proportionate to the validity time. If the proposed validity time of the unit endorsement exceeds 12 months additional means should be in place to monitor and ensure the continuous competence of the air traffic controllers. If the ATC unit is proposing to increase the validity time of the unit endorsement a safety assessment should be conducted. The safety assessment may cover several units.
(a) If the assessment of practical skills is taking the form of a dedicated assessment consisting of a single assessment or a series of assessments, the last assessment declaring the license holder competent should take place within the 3-month period immediately preceding the unit endorsement expiry date.
2.6.1 Last year, IFATCA PLC asked member associations to fill out a digital questionnaire on competence assessment. The goal of this survey was to check how competence assessment is performed around the world and to inquire if there are concerns about the way things work. The survey was sent out by e-mail and also highlighted at IFATCA Regional Meetings. 43 member associations replied, providing a great amount of insights in the matter. Most replies came from Europe and North and Central America, but also many other countries filled out the questionnaire. Competence assessment exists in 93% of the member states that replied. In Uruguay, Kenya and Gambia there is no active competence assessment for ATCOs.
2.6.2 Although not all countries from the African continent replied, an enquiry at the IFATCA AFM regional meeting revealed that many members in the AFM region do not have competence assessment in place, apart from Zambia and Togo.
2.6.3 There are differences between countries in how competency assessment works in practice. In most cases, there is an on-the-job proficiency check that has to be passed. An assessor sits next to an ATCO working live traffic, for about one to two hours per single rating. The regularity of these checks varies from four times a year to once every three years among ANSPs around the world, but once a year is most common. This is often combined with a yearly theoretical test with questions about rules, procedures and general ATCO knowledge such as air law and meteorology. If the result of the competence assessment is not satisfactory, an ATCO will typically be given a training programme in a simulator environment to overcome deficiencies. Thereafter they will work a number of shifts under supervision of an OJTI to oversee their performance.
2.6.4 Theoretical tests vary from a yearly multiple-choice test, with a common pass rate between 70% and 80%, to oral testing (questions during or after the practical assessment). Nowadays it is also common to check small parts of the theory more regularly, i.e. random exam questions during the self-briefing when starting a shift.
2.6.5 Who is appointed to assess the performance of an ATCO differs per country? Generally, all assessors are experienced ATCOs, who have an OJTI endorsement. In some countries, only unit or sector supervisors assess the work of their controllers. Most ANSPs, however, have a separate assessor role in place next to the role of instructor. There may be a financial incentive for ATCOs that choose to be an assessor. There are countries in which management has no say in the selection of assessors; for instance, in Iceland they are chosen by the ATCOs themselves.
2.6.6 The survey revealed that there are countries in which assessors, once elected, are relieved from any checks on their own competence. A distressing percentage of 13% of respondents indicated this. In most countries assessors are also assessed themselves while working on the job, and in about 50% of the cases they are also reviewed on how they perform their assessor duties. This can be in a live environment, but also by group meetings where training experts provide training and feedback on their reports and proceedings.
2.6.8 In general, the survey shows that ATCOs have no major concerns about the way competence assessment is performed in their organisation. Nobody likes exams and checks, but many respondents say they believe the system works quite well.
2.6.10 Common issues also involve frequency of assessments and quality of the assessors. Some respondents complain that, due to staff shortages, their practical assessments do not take place as frequently as the procedures prescribe. Furthermore, the way assessors are selected and trained and trained is often a point of criticism.
2.6.11 There are indications from MAs that their employer is intending to link results of competency assessment to the amount of salary a controller receives. This is a very distressing development. Competency and performance of ATCOs shall never be linked to remuneration.
The results of proficiency checks should be treated confidentially and management involvement should only be necessary in cases of negligence or on the recommendation of the appointed check controller.
Before a proficiency checking system is implemented, adequate training facilities should exist to enable further training to take place where necessary. (accepted as Guidance Material, Cairo 81.C.16-20)
IFATCA is in support of a proficiency checking system for all air traffic controllers exercising the privileges of an ATC License or an equivalent Certificate of Competency for all qualified persons engaged in the duties of air traffic control.
Before any proficiency checking system is implemented the respective Member Association and employer should undergo extensive negotiations and agree to resolve internal differences in respect of their own socioeconomic situation, (which includes retraining and job security). (Cairo 81.C.22)
Member Associations should indicate to their employers that if assessments are conducted controllers must have the opportunity of sighting these assessments and discussing them with the assessing officer. Additionally a controller must have the opportunity of registering, on the assessment form, his comments regarding the assessment and the manner in which it was carried out.
Where a proficiency checking system has been implemented, a controller who is selected to act in the Check Controller role should undergo a specialist course of training that will prepare him/her for the task, and provide guidance on achieving a fair, objective, and valid assessment. This training course should achieve consistency between check controllers.
This Assessor qualification should be the subject of periodic refresher training, at periods not exceeding 3 years, to ensure that skills are maintained and new techniques and procedures are incorporated. (Cancun 02.C.2)
2.7.4 The policy itself is valid, but the terms used (proficiency checking; check controller) are outdated. Also, it lacks structure and does not include how often and in what way competence assessment should be performed.
Audio and visual recordings and Ambient Workplace Recording (AWR), together with associated computer data and transcripts of air traffic control communications are intended to provide a record of such communications for use in the monitoring of air traffic control operations, and the investigation of incidents and accidents. Audio and visual recordings and AWR are confidential and shall not be released to the public.
3.4 A survey conducted by IFATCA PLC last year revealed data on how competence assessment is implemented around the world. In most countries, an ATCO undergoes a practical simulator test once a year, or is observed while working a shift, together with an oral or written test on theoretical knowledge. In the African region, competence assessment is not yet common.
3.5 Generally all assessors are experienced ATCOs, who are carefully selected. There are great differences between countries in how assessors are kept current and if they are assessed themselves, after being installed in the position.
3.6 The survey also shows that while nobody likes being evaluated, most Member Associations have little problems with the way competence assessment is being carried out in their organisations. There is always room for improvement, for instance on the frequency of the assessments and the quality of the assessors.
3.6 IFATCA policy prohibits the use of recorded data for competence assessment. The proficiency of controllers should be checked on-the-job in a live traffic environment, or in a simulator. Also, competency assessment shall never be linked to remuneration.
64591212e2