KIA Automated Performance Evaluation System - KAPES
Kenya Institute of Administration

The Problem

Customer data gathering and evaluation has for a long time been an overwhelming task to many institutions more so to those of academic nature. The Kenya Institute of Administration’s (KIA’s) whose mandate is to train, undertake research and consultancy gathers feedback for the following departments directly involved in service delivery: training, Information, Communication and Technology (ICT), admissions, accounts, hospitality, housekeeping, procurement, security and maintenance. This paper presents an innovation in reengineering of the KIA Automated Evaluation System (KAPES) used by participants). The previously inefficient and ineffective system has been transformed into a fast and reliable online one. The new system enables the Institute carry out evaluation fast and with ease to realize results for reporting to all departments. This system is acknowledged as an administrative and management achievement for effective and efficient operations leading to enhanced and quick feedback mechanism and ultimately high customer satisfaction. The paper concludes with a brief discussion of the possible replication of KAPES in other Institutions of a similar nature.
It is customary that participants give feedback on completion of their training at the Kenya Institute of Administration’s (KIA) to facilitate service improvement and participant (customer) assurance. This starts with the program assistant presenting and giving instructions to participants on how to fill in the end-of-event evaluation form at the training rooms. The duly filled in forms are then presented to the programme coordinator who keys in the information and caries out statistical analyses in order to generate meaningful results for interpretation.

This process had a lot of weaknesses since report generation was tedious, and feedback was not effectively given in time especially for coordinators with many programs. The analysis was also dependent on whether the coordinators were good in statistical analysis. This meant that repeat customers experienced the same problems that they had previously pointed out during their previous training at KIA. Similarly, there were delays and inadequate actions taken by all departments in effecting participants recommendations. Above all, this led to low customer satisfaction with the service delivery.
Further, there was a weak link between programme administration and management and participants’ expectations leading to frequent operational challenges where improvement was required.

Overall, the manual evaluation system hindered customer satisfaction because departments could not easily point out where there were problems and how they could be resolved.

Solution and Key Benefits

 What is the initiative about? (the solution)
The innovation has had the following achievements positive effects:

i. There is centralized and controlled feedback because KAPES has become a one-stop interface with all queries and suggestions to KIA given by participants analyzed. This has minimized duplication of resources leading to enhanced synergy in the departments.

ii. There is reduced paperwork and overheads, smoother day-to-day running of the departments and better logistics because whoever requires information quickly logs into the system. The system has also led to maximization of efficiency and effectiveness.

iii. Information is readily available in form of an avalanche of user-friendly online reports which aids systematic decision making.

iv. The system has assisted the Institute towards its objective of operating paperless.

v. The system places immense productivity and decision making power in the hands of KIA staff. The system also gives staff an opportunity to harness the latest information technologies, hence leveraging on our investment on IT infrastructure.

vi. With the seamless customization and adoption of KAPES, the Institute was certain that the system would further be improved with the changing business requirements. This has guaranteed KIA maximum system uptime and low maintenance cost, support and profitability. This has ultimately helped KIA look into the future with confidence.

Actors and Stakeholders

 Who proposed the solution, who implemented it and who were the stakeholders?
The proposal was forwarded from the Training Department when after facing prolonged challenges in gathering, analyzing and reporting customer feedback. The proposal was taken up by ICT section which embarked on the development of the system that would help alleviate challenges faced by the Training Department. The system has been adopted by other departments namely: ICT, admissions, accounts, hospitality, housekeeping, procurement, security and maintenance.

(a) Strategies

 Describe how and when the initiative was implemented by answering these questions
 a.      What were the strategies used to implement the initiative? In no more than 500 words, provide a summary of the main objectives and strategies of the initiative, how they were established and by whom.
The main objectives of the innovation are to:
i. Develop KAPES in relation to KIA’s Business process Reengineering (BPR) of training evaluation;
ii. Align KAPES with KIA ISO Standards 9001:2008 for quality operations;
iii. Carry out fast operations at the Institute;
iv. Increase customers and staff satisfaction level;
v. Reduce the cost of programme evaluation;
vi. Provide timely feedback from programme participants, and
vii. Ensure for continual improvements of performance.

(b) Implementation

 b.      What were the key development and implementation steps and the chronology? No more than 500 words
Since the inception of the Institute in 1961, programme evaluation was done manually by administering questionnaires also called “happy sheets” suggesting that participants might have given favourable comments. Further, there was a likelihood of the facilitator tracing a participant who apparently gave negative feedback. The questionnaire was also too lengthy and most participants did not respond to all questions. There also was much paperwork and tedious analyses especially in large classes before the programme coordinator wrote a report. This led to delayed actions in response to participants’ feedback across departments.

In an attempt to have quality participant feedback provided at the end of the training, there was need for departments to have timely, accurate, reliable, sufficient and readily accessible information provided through an integrated online evaluation system. This would improve efficiency and effectiveness in respective operational areas.

The end-of-event evaluation (KAPES) which is an online process of capturing participant feedback at the end of all KIA training was therefore developed in 2009. This was done on the premise that participant feedback should be acted upon promptly for service improvement.

An electronic data capturing portal was developed by the end of 2009 to capture participant feedback for analysis and reporting. Participants log on to the portal and give their feedback by rating a set of six multiple-choice questions on a scale ranging between Poor (1) and Excellent (5). The portal also gives an open-ended question to allow participants freely make any other comments not covered in the rated questions. The mode of feedback on the portal is set anonymously and automatically, which means that one cannot identify the source of the comments nor the portal administrator. This gives participants freedom to give their very honest feedback.
In order to ensure successful implementation of this system, some changes were made as follows:
i. The evaluation form was re-casted to capture essential areas of operation by reducing the number of questions in the evaluation form from 16 to 6 questions;
ii. The evaluation system was re-engineered so that all the service departments could directly access participant feedback online;
iii. All programme coordinators and assistants were trained to administer the evaluation process effectively, and
iv. Evaluations are systematically scheduled on the last day of the programme

By the end of 2010, a comprehensive review was undertaken to establish the effectiveness of the online evaluation system based on the feedback from the users. Recommendations thereof were used to reengineer the KAPES to meet users’ needs. This led to a more user friendly system that has been in place throughout 2011.
Flow Chart Diagram (Before and After)
To date, 3,800 participants have successfully evaluated KIA services as evident in the portal. The data collected is used by the KIA management to make decisions accordingly. This ensures that the same complain is not raised again which would indicate lack of management response.
Current reports on the portal show that participant satisfaction is above 85%. Aspects such as programme coordination, facilitation, quality of food, meeting objectives and customer expectations on average receive excellence ratings. The cleanliness of both lecture and residential rooms has not been as highly rated.
The ICT section realized the need to train some users of KAPES such as programme assistants who are involved in participant satisfaction audit. This has enabled them perform the exercise diligently and interact more with participants as they undertake the exercise. This also helps them capture the mood and perception of participants about the institute in general.

Based on user feedback, a further review is scheduled to be undertaken in 2012

(c) Overcoming Obstacles

 c.      What were the main obstacles encountered? How were they overcome? No more than 500 words
The following were the obstacles encountered in the development and implementation of the system:

7.1 Obstacles before 2009
i. The fear of victimization should the facilitator trace a participant who apparently gave negative feedback;
ii. The questionnaire was also lengthy and many participants ended up not filling all the questions;
iii. The evaluation forms filled by participants could sometimes get lost;
iv. There was much paperwork especially when a class was large and data analysis was difficult because involved tedious data entry before writing a comprehensive report;
v. There was possible failure to analyze collected data;

7.2 Obstacles after 2009
i. KAPES was resisted by staff who were used to the manual system;
ii. Repeat participants who were used to the manual system resisted KAPES;
iii. Initially staff lacked computing skills on system usage;
iv. There was persistent system failure and inaccessibility to the portal;
v. Management did not use feedback from the system fully;
vi. Management was overwhelmed by the many decisions that required immediate action;
vii. Some of the older participants lacked computing skills;
viii. KAPES lacked backup statistical analysis tools to monitor the trends, and
ix. There was inability to audit the system continuously.

(d) Use of Resources

 d.      What resources were used for the initiative and what were its key benefits? In no more than 500 words, specify what were the financial, technical and human resources’ costs associated with this initiative. Describe how resources were mobilized
The innovation was internally developed at a time when KIA already had basic ICT infrastructure mainly; Local Area Network (LAN), computers and computer laboratories. Other resources required for this innovation were mainly:
i. The training of program assistants and ICT technical support;
ii. The portal,
iii. The online evaluation tool;
iv. Computer laboratories

Sustainability and Transferability

  Is the initiative sustainable and transferable?
The KAPES is a management and administrative tool that can ensure that any training Institution develops a workable programme evaluation strategy to ensure that participant feedback is promptly captured and acted upon. The system has been customized to meet particular needs. It implies that with effective usage, less time and other resources are wasted and effective decisions made. The system can therefore be replicated in other training institutions.

Lessons Learned

 What are the impact of your initiative and the lessons learned?
This system has had great impact in terms of fast processing of participant feedback and timely response. There has been massive cost reduction as the institute went paperless on all programme evaluations.
The lessons learned are numerous including: the need for top leadership commitment; availability of modern and sufficient ICT infrastructure; sensitization of key stakeholders on the use and benefits of the system; review of the system within the framework of effective change management; need for continuous improvement and assurance of sustainability.
It is envisaged that KAPES will lay the foundation of the development of more online systems to support programme administration and management, and other operations at KIA. This will continue being fundamental pillars for business growth of KIA as a national and regional capacity development Institution.

Contact Information

Institution Name:   Kenya Institute of Administration
Institution Type:   Government Agency  
Contact Person:   Jane Mwangi
Title:   Senior Principal Lecturer  
Telephone/ Fax:   254204182311
Institution's / Project's Website:
Address:   23030
Postal Code:   0604
City:   Nairobi
State/Province:   Nairobi
Country:   Kenya

          Go Back

Print friendly Page