United Nations Public Service Awards Case Studies

Karnataka has made significant progress towards universalisation of elementary education, which is constitutionally compulsory. Children up to 14 years of age and of classes 1 to 8 are included. In fact, access to schools in Karnataka is one of the best in the country, where 97 percent of the population has a primary school within walking distance of every habitation. More recently, it has focused on reducing large regional disparities, gender differences, school drop-out, discrimination against children with special needs, and other barriers. Karnataka also recognizes the urgent need to address a serious and persistent problem : the low quality of education.

It is true that post independence policy pronunciations have put access to educational facilities as the priority. NPE 1986 went further and included the need for providing education of ‘Comparable quality’. This is an issue, which has a direct bearing on equity issue. For that very reason now quality dimensions in a situation of such complex plurality of state acquired very significant role.

‘Monitoring for quality must be seen as a process that enables and provides constructive feed back in relation to the teaching, learning process within specific classroom contexts’ (National Curriculum framework 2005).

There have been opinions expressed regarding availability of data on classroom evaluation at school level. With the introduction of MLL based learning in Karnataka formats were devised and introduced for capturing performance of students. These are competency based. However, there is no data available at state level. Efforts of base line studies under DPEP and Janashala were confined to certain blocks and districts only.

Hence the need for assessment of schools at state level has been stressed in many of the initiatives of the state. Edu-vision document of 2003 envisaged establishment of a separate organisation and charge it with the development of effective systems for assessing schools. This was also listed as a priority area in the Departmental Medium Term Fiscal plan (DMTFP) (2003-04 to 2006-07). Justification for such an assessment organisation also stems from the issue of accountability of the system. Accountability should form the basis for funding of specific programs.

‘Quality concern, a very key feature of systematic reform implies the system’s capacity to reform itself by enhancing its ability to remedy its own weaknesses and to develop new capabilities’- (National Curriculum Framework-2005).

What is the initiative about? (the solution)
The entire exercise of Karnataka School Quality Assessment Organisation has been aimed at bringing transparency in public administration. People have a right to know how the hard earned tax payers money is spent. The quality dimensions either to confine to lower levels is now available for the entire state, for all the schools. People can judge whether the allocations on Primary Education is really well spent in terms of attainment of quality. The results also indicate the initiatives to be taken on areas of class room management for academic excellence. Assessment of learners achievement and through it quality of each school is the need of the hour. The fact that unprecedented flow of funds to primary education demands accountability. We need to know where we stand in terms of quality. One of the major outcomes of first assessment is the review of class room transactions. Initiatives like teacher training and academic inputs need long term strategy. Funding also needs review in terms of set goals for primary education. The Second major impact is the involvement of S D M C s. Sustained efforts for involvement of people in quality aspects needs to be strengthened. Inherent weaknesses in academic delivery at the school level need to be tackled. As indicated in the World Bank document for the next three years, we need to have learners profile for all levels of elementary education.

Who proposed the solution, who implemented it and who were the stakeholders?
Annual Calendar of events 2005-06 Plan of Action - April 2005 Formation of Empowerment Committee- May 2005 Preparation of School Handbook (Preliminary work)-May 2005, June 2005 Preparation of budget advocacy materials viz. posters, brochures, handbills and budget -June 2005, July 2005 Preparation of Teleconference training manual-July 2005 Satcom based training programme for DDPI, BEO, BRC, BRP & CRPs (Teleconference-1)-July 2005 Tender process for publication of Advocacy materials viz. School handbook, brochure, handbill, posters and certificates- July 2005 August 2005 Satcom based training programme Teleconference-2 August 2005 Standardization of tools-August 2005 September 05 Appointment of Nodal officers-August 2005 Formation of State Core group and MRP team-September 2005 Radio broadcast programme-September 2005 Printing of school handbook, brochure, posters, handbills-September 2005 Mukhamukhi Programme (Interactive programme)-September 2005 October 2005 Distribution of school handbooks, brochures, poster and handbills - October 2005 Core group training - October 2005 M.R.P training programme at District level - October 2005 November 2005 Awareness programme for HMs & SDMC members-November 2005 Evaluators training programme-November 2005 December 2005 Teleconference-3, Satcom based training programme in Radio broadcast - December 2005 Assessment process - January 2006 Scanning of OMR, consolidation of Analysis of data - January 2006 February 2006 Reporting and research activities - March 2006

Describe how and when the initiative was implemented by answering these questions
a. What were the strategies used to implement the initiative? In no more than 500 words, provide a summary of the main objectives and strategies of the initiative, how they were established and by whom.
National Policy on Education 1986 is very significant because of the new initiatives it heralded. The issues of access and retention have always taken the lion’s share of time and attention of planners for justified reasons. NPE 1986 in addition to strengthening the above spoke of imparting education of ‘comparable’ quality. National Curriculum Framework of 2005 has taken the idea of quality dimensions further. Two statements made in this document are very significant. “Monitoring for quality must be seen as a process that enables and provides constructive feedback in relation to the teaching-learning processes within specific classroom contexts”. Another statement relevant to quality issues, from the National Curriculum is, “Quality concern a key feature of systemic reform implies the system’s capacity to reform itself by enhancing its ability to remedy its own weaknesses and to develop new capabilities”. These two statements put in a nutshell the need to have assessment of quality. Assessment of quality should aim at (1) systemic reforms and (2) improvement of teaching – learning process. There has been significant thinking in Karnataka on the issue of assessment of quality. Eduvision document of 2003 spoke of the need to assess quality through a statutory organisation. Consultants for World bank submitted their own blue print for quality assessment. Mrs. A.V. Saliba, consultant for world bank, made a number of suggestions for constitution of an independent body for assessment of quality. Government of Karnataka had already built in the necessary budgetary provisions before constituting KSQAO during May 2005. The main issues expressed in the World Bank document concerned the need to generate reliable information on the performance of elementary schools. By this time Karnataka had the experience of surveying educational attainments through project work of DPEP, Janashala programs. However they were confined to certain blocks or districts. Another significant development has been the work of Azim Premji Foundation through its Learning Guarantee Program. These experiences have been very significant learning opportunities for us. Functionaries of Azim Premji Foundation have been our partners in the present endeavours.

b. What were the key development and implementation steps and the chronology? No more than 500 words
Having achieved significant progress toward universal elementary education, Karnataka seeks to address the quality of its school education. Development of effective system of assessing schools is the main task of the organisation. Assessment of schools can be through different methods which depend on the learning process, their objectives and investments on learning. Assessment of learning should be priority, after all, the sum total of inputs for learning is reflected in the levels of attainment. Objectives of KSQAO were evolved after the state level workshops conducted during May-June-2005. The main objectives of KSQAO are, 1) Assess the learning outcomes of students in selected competencies of different subjects prescribed for the class by using universally accepted scientific methods. 2) To enable stakeholders particularly the parents to visualise the needs for assessing quality and analyse ensuing outcomes, so that they are better equipped to strive for educational progress. 3) To make available reports of assessment to educational administrators, policy planners and common man at large, so that meaningful data is available for an effective educational planning. 4) To make available the reports of this study to functionaries at various levels, so that self-analysis is promoted in the system. 5) To make available the data to participating schools so that school and cluster level academic initiatives are promoted for effective classroom transitions. 6) To create awareness on various issues concerning quality of education at different levels so that school assessment takes the centre stage. 7) To make available the data, analysis, interpretations and information in general, so that meaningful research in the field of education is facilitated. Definition of Quality :- This has been one of the contentious issues in all the debates that KSQAO has hosted. There are opinions galore on issues concerning quality. There were opinions on supposed short comings in assessment of learning outcomes as indicators of quality. There were many who felt that classroom transactions should be assessed. There were others who felt that inputs do matter, however it is the output that should have the final say. The most important thing was that the whole exercise veered round the stupendous numbers we were grappling with. We were looking at a staggering school number of 46,000. Children in each class were in lakhs. Therefore inspite of all the debate on dimensions of quality we had to arrive at very simple definition of quality which would make sense to the common man who was the real stake owner. KSQAO adopted the following statement for assessment of quality. “Enroll all eligible children and retain them in the system. They should qualify for promotion to the next class on performance”. So any assessment of school would include, · Admission in schools · Attendance of children · Attainment of competencies prescribed · Percentage of those qualified for promotion

c. What were the main obstacles encountered? How were they overcome? No more than 500 words
The organisation has been entrusted to generate, supply and disseminate, to schools, communities and government bodies, relevant information, derived from sound, independent and sustainable assessments, as means for improving outcomes, especially learning, in school education. Tests and Assessment we were aware of the national criteria for defining Minimum Level of Learning (MLLs). We were very anxious that the negative connotations regarding attainment levels needed to be addressed first. We decided that attainment levels be benchmarked as they unfolded. We would separately generate data for comparative study with previous surveys on the norms laid down previously. The following decisions were taken. Class II would have a combination of oral and written tests Classes V and VII would face only written tests Competencies which are core to learning during the first two trimester would be tested Schools would get a complete list of competencies selected, model question paper and question banks Refining of test items Extensive field trials were undertaken by DIETs through well-known experts who were available locally. Formats for capture of data Workshops were organised to finalise the formats to be used for capture of data. Student information proforma for Class-2, 5 & 7 School Development Monitoring Committee information capturing format and Intelligent Character Recognision format were designed. Responses obtained from schools were computerised. The ICRs from schools were scanned at the DIETs of the districts. Formats were designed to generate reports at school, cluster, block, district level. Constitution of State Resource Team (Core team) A team of dedicated primary school teachers with a sprinkling of secondary teachers and class II officers constituted a core team. The team in addition to conceptualization of the ideas was in charge of implementation of the program. The same team took over the concepts, precepts and implementation strategies for KSQAO. Trainings One of the biggest achievements of KSQAO has been training aspects. All activities from conception to implementation were evolved by the core team. State level workshops were conducted to draw up detailed itinerary of trainings required, the training inputs and detailed literature for both trainers and the trained. The first training was for the state core team itself. This was greatly facilitated by agencies like Azim Premji Foundation. There were 7500 teams. Nearly 30,000 persons had to be trained to administer precisely tests, fill up the formats. We selected district resource teams carefully and preferred trainers of DPEP and Janashala projects. The main issues on which training was imparted were – 1) Administration of oral and written tests 2) Filling up of formats 3) Filling up of ICR The state core team trained the district resource persons at the end of November 2005. Administration of Tests The actual administration of tests was planned with attention to very minute details. Detailed schedule of each team of evaluators was drawn up. Question papers were packaged school wise. The requirements for each school were precisely worked out. Cluster resource centre became the centre from which materials and men reached each school on the scheduled day of administration of tests.District Institute of Education and Training principals constituted teams to visit these schools. This resulted in co-ordinated monitoring at school, block and district level.

d. What resources were used for the initiative and what were its key benefits? In no more than 500 words, specify what were the financial, technical and human resources’ costs associated with this initiative. Describe how resources were mobilized
The changed scenario Analysis of Result - 2005-06 v Test items framed after extensive try-out v Competencies classified as crucial to learning short listed v At the end of the day classroom transactions need to be targeted. v Underlying assumptions that learning outcomes as indicators of classroom transactions. v Analysis to benchmark learning levels visa-vis learning continuum Analysis of data v Efficacy of competencies selected under MLL v Course of action both academic & administrative under scanner prioritised for follow-up v Training interventions and text book review part of follow-up v Efficacy of administrative setup with BRCs and CRCs being focal issues. v Strategic inputs for strengthening of CRC as the delivery point for academic & administrative issues Significant Observations v The programme has gained acceptance among the officers v Schools have exhibited abundant enthusiasm for quality education v Several non-participating schools have come forward and insisted on inclusion of their students v Participation of D.Ed students in Evaluators team reduces malpractices during assessment period v Children have participated joyfully v A medium subsequently an idiom for competency based testing on this stupendous scale is put in place v Training strategies evolved for competency based testing on this huge scale v DIETs and CRCs prioritised as academic agencies equipped to handle interventions of this type Assessment Reforms : Influence the way normal examinations need to be replaced by competency based examination Ø Assessment of schools will throw up, 1) Needs for classroom inputs & interventions 2) Needs for addressing teacher training issues 3) Needs for parent / public intervention & support 4) Needs for all to decide where we stand ? Assessment Statistics : Total Districts – 32 Total Blocks – 191 Total Clusters – 2007 Class No. of Schools No. of Students Class-2 7742 159066 Class-5 34377 781953 Class-7 17058 676664 Total 1617683 Overall State Performance (in percentage) Subjects class Kan Maths EVS/ Science Social Science Total 2 65% 61% 78% - 67% 5 51% 46% 53% 47% 49% 7 54% 40% 50% 46% 48%

Is the initiative sustainable and transferable?
The issues involved in the process of assessment centre around tests. Selling the idea of examination in the present context was very challenging. Karnataka has re-organised its school education by resorting to trimester system of learning. One of the main planks for such a change was the negative connotations associated with examination. So when KSQAO proposed a test to be held during Jan 2006 for classes 2, 5 and 7 there were mixed reactions. People welcomed the move as a step towards accountability. Certain critics found the idea of another exam anachronic with changes introduced under trimester system. Certain quarters expressed reservations because Govt. could use the results to malign teachers. Amid such a volatile situation what withstood the KSQAO in the right stead was the programs for advocacy started during June 2005. KSQAO firmly believed in taking teachers into confidence. A handbook for each school was prepared. This handbook included the list of competencies, bank of activities, model question paper and question bank. There were detailed guidelines on how to use the book to prepare children for the assessment. Brochures were printed detailing the highlights of the program and circulated extensively. Hand bills and posters were prepared and sent to all the schools and villages. The biggest element of support was expected from SDMCs. These committees were to play a crucial role in the coming days. Trainings were organised for members of school development and monitoring committees. However the participation of SDMCs has been a mixed bag. It has ranged from excellent to mediocre. However the need to involve SDMCs in a very big way is one of the finds of KSQAO. Positive outcome was that children participated without any fear. People at large welcomed the move saying that accountability will be ensured by such assessment. Teachers’ reactions were mixed. Inspite of our campaign regarding objectives and justifications of assessment, some of them felt that results could undermine their own standing in society. Others felt that this will initiate new initiatives due to competition between schools.

What are the impact of your initiative and the lessons learned?
1) There were issues we had to tackle under the Advocacy program, more significant was the fact that we had to find our own way as there were no precedent. 2) We had to be careful regarding formats for capturing of data from schools development and monitoring committee (SDMC) 3) There were also significant issues of oral and written competencies. 4) Test items needed to be refined 5) It was not possible to draw such a large number of teachers out of schools for a week for evolution work. Hence we decided to utilise the service of DEd students. Assessment of learners achievement and through it quality of each school is the need of the hour. The fact that unprecedented flow of funds to primary education demands accountability. We need to know where we stand in terms of quality. One of the major outcomes of first assessment is the review of class room transactions. Initiatives like teacher training and academic inputs need long term strategy. Funding also needs review in terms of set goals for primary education. The Second major impact is the involvement of S D M C s. Sustained efforts for involvement of people in quality aspects needs to be strengthened. Inherent weaknesses in academic delivery at the school level need to be tackled. As indicated in the World Bank document for the next three years, we need to have learners profile for all levels of elementary education.


Institution Name:		Karnataka School Quality Assessment Organisation
Institution Type:		Government Department
Contact Person:		Kumar T.M.
Title:		Director (Examinations)
Telephone/ Fax:		91-80-23562283, 91-80-23341615
Institution's / Project's Website:		91-80-23347670
E-mail:		ksqaobangalore@yahoo.co.in
Address:		Karnataka School Quality Assessment Organisation, Karnataka State Education Examinination Board, 6th cross, Malleswaram, Bangalore
Postal Code:		560 003
City:		Bangalore
State/Province:		Karnataka
Country:		India

The Problem

Solution and Key Benefits

Actors and Stakeholders

(a) Strategies

(b) Implementation

(c) Overcoming Obstacles

(d) Use of Resources

Sustainability and Transferability

Lessons Learned

Contact Information