Answers to the conference issues from the Czech Republic

1.1. Reasons for the selected type of examination at the end of upper secondary education

The responsibility for creating the national curricula for the state education still lies within the authority of the Ministry of Education. The 1995 amendment to the Education Act enables the existence of different types of curricular documents. The task of post-compulsory upper secondary schools is twofold: to prepare pupils for employment or the activities within the national economy, administration, culture, the arts or other areas of life (the terminal function) or to prepare pupils for a continuation of their studies in higher education (the transfer function). In 1995, the Ministry of Education, Youth and Sport adopted a measure aimed to reform the manner of secondary school leaving. The background of the adopted measure lies not only within the actual situation of secondary school leaving, but corresponds to the essential demand, that a new model of Maturita exam should, on one hand, ensure the comparability of Maturita results and should not, on the other hand, limit school self-administration and thus the desired breadth of educational variety at the secondary schools. The decision to take this kind of measure was, of course, dictated by an effort to integrate the Czech Republic into the European Union, and concretely, to guarantee an international acceptance of the Czech Maturita exam as one of the standard "outputs" of the secondary education.

As a result of the political changes in Central Europe in the nineties it has now become possible to start collaborations not only in the field of education but in the society as a whole. To these social changes should especially contribute numerous activities with respect to the teacher-training in the evaluation of teaching results and the examination and testing methodology.

The education reform in the Czech Republic is supposed to make an important contribution to the social transformation process-taking place in the country. Just in these days, the Czech Ministry of Education, Youth and Sport is preparing a new Education Law. The modernisation of the country's educational system needs more guidance. One of the methods to reach this goal is to create national and evaluation standards of education. Such standards must provide the Czech government with certain means of control over the quality of education by means of assessment tools to monitor the level of achievement referring to these standards. Because of a strong relationship between the education reform and the process of social transformation in the Czech Republic, the Czech Ministry of Education, Youth and Sport has asked CITO to submit the following project proposal for funding under the MATRA program. Two main aims of the co-operation in the project framework were defined:

1. To build the capacity of Czech specialists in developing and producing professional instruments for educational measurement to be used in the new Maturita exam;
   
2. The implementation of the goals of the Maturita exam - a broad assessment of student's knowledge and skills - in society; introducing the relation between teaching and examination as a guarantee for the quality of education to the general public and all parties involved.

• To unify and conform professional opinion at solving the Maturant 2001 project
  
• To use the knowledge obtained from the Maturant 2001 probe
  
• To prepare the groundwork for training specialists for test development, their presentation and evaluation, both for the state part of the school-leaving examination 2001 and for the methodological process of the specific school part of Maturant 2001 project

1.2. Definition of the "external" and "internal" dimension of the final examination

A suitable general model of a standardised way of secondary school leaving seems to lie within the integration of an external part of the Maturita exam. The external part of the Maturita exam will be realised through centralised written tests on general subjects - i.e. Czech language, foreign languages, mathematics and general study assumptions) (Note 1). For this reason, the Czech Ministry of Education, Youth and Sport declared and initiated in 1996 a long term programme MATURANT 2001 with the aim to find the optimal form, content and organisation of the external part of the Maturita exam. The expected programme conclusion is the year 2001, when the new model of Maturita exam should become obligatory. The programme includes:

1. Preparation and verification of the evaluation assessments which would become the essential foundation for the formulation of content and form of respective tests and for the construction of adequate item banks of tasks.
   
2. Preparation and verification of the organisational and logistical model of the testing proceedings, including the preparation of testing, administration, processing and examining of test results, incl. protection and security of information.
   
3. Providing of a realistic viability of the new Maturita exam, especially by a notice placed in advance to all schools and pupils.

The preparatory project MATURANT 2000 included regular yearly check-up studies. Their aim was to verify a hypothesis, which should lead to an optimal model of the external part of Maturita exam. In 1996, the first study called "Maturant 97" was tested at 100 "gymnasiums" (general upper-secondary school) and 100 secondary technical schools. Its results became the foundation for the goal set-up of the next, second study, which was run in April 1998. This study took place at all secondary schools and all the final year secondary school students participated. It consisted of Czech language and literature test, a test of English or German, mathematics (basic and advanced version) and a test of general study assumptions or general scholastic abilities (a short Czech version of the "SAT").

The overall objective of the state programme is to support the renewal and the modernisation of the Czech society as the whole, and especially the country's educational system within its transition to a pluralistic democracy and market economy. Centrally written examinations (CWE), based on clear and explicit programmes and syllabuses, are an indispensable means of quality control and communication in the Czech educational system, where it is very important to develop a system of central examinations that could cater for the need of quality control and management of educational change. Because of the increase of the number of secondary schools in the Czech Republic during the past five years, there is a need to create a flexible steering and evaluation system based on the measurement of the "output", which is defined as the results of primary and secondary schools and of their students. This system should be flexible enough to respond to any societal changes and leave enough room for the legitimate wish for the school autonomy.

The expected role of the CWE in the Czech Educational System is seen in many, because the new assessment and examination system is seemly able:

• To communicate important objectives in the society and education
  
• To maintain quality standards in primary and secondary schools
  
• To allocate students to different streams in the educational system
  
• To provide entrance certificates (e.g. for universities)
  
• To provide school management with information to evaluate and steer the educational process (quality control)
  
• To provide clients with information (interested parents, schools for further education, e.g.)
  
• To monitor results for teachers, to steer their didactical approach (quality control).

• Professional public discussion on the first version of the evaluation model MATURANT 2001 (the estimated date for the conclusion of discussion is December 2000)
  
• Expertise on principles of modern test theory and item banking used by many institutions and teachers involved in education
  
• Launching of the item banks construction for basic subjects of Maturita exam
  
• Standardised specimen exams for Czech language and literature, mathematics, English, German and French, reflecting specifications in terms of subject content, skills tested, question formats and difficulty index, to be used as a standardised yardstick for future Matura exams in 2001
  
• Decision on the form of experimental Maturita exam model, preparation of mandatory rules (June 2000)

• Understanding and support for the standardisation of educational comparative methodology and the evaluation of results in education will be won in institutions related to the Ministry and in a number of primary and secondary schools.
  
• Clear decision by the Czech Ministry of Education, Youth and Sport on the desired evaluation model will be possible on the basis of public discussions and the project results.
  
• The final result of the project will be presented as an advice, so the Minister of Education can decide on the matter of the evaluation model. Sound support will be gained in the Ministry for the selected evaluation model. Number of teachers, inspectors and staff-members the CEEA will be able to implement the evaluation methodology as well as the operational consequences of the evaluation model in practice.

1. What should a secondary school student learn to reach his/her own personal needs and also the social ones, which are necessary to enter the future labour market before the entering of the CR into EU?
   
2. What do they actually learn in the recent stage of the transformation of the Czech education system?
   
3. How can the level of students be assessed and how can students be assigned to the right type of education?
   
4. How can the quality of education be controlled by the state and school authorities?
   
5. How can the teachers, students and other social partners be involved in the process of the leaving examinations (Maturant 2001) reform and in the quality control of the output in education?
   

The purpose of the mentioned programme in the second phase in the year 2000 should try to:

1. Create adequate item banks for the CWE in the Czech Republic as a collection of test items, which will be pre-tested on groups of students during the year 2000. Consequently, detailed information about the item characteristics and especially about the level of difficulty will be available before the administration of the pilot CWE in the first half of the year 2000. In principal, large numbers of similar examinations with the same level of difficulty can be selected from such a collection. This will be a distinct advantage for the government, that yearly wants to express equal requirements through examinations and that wants to anticipate flexibly the differences in educational processes, because schools differ in the way they value certain objectives, and not all the students are ready for examinations at the same level.
   
2. Determine the final draft of the CWE for the secondary schools in the Czech Republic and needed strength of the support, as it is presently offered as the directions for the growth of the reform of school leaving examination system.
   
3. Specific objectives of this period are also seen in:
   
   • The methodological assistance in the development of the CWE for the secondary education, where state and private institutions, responsible for the developing and monitoring of the evaluation process, are involved:
     
   • Training and technical assistance in the field of national assessment of educational progress in secondary education;
     
   • The assistance with setting up a new examination system in secondary schools and the assessment of educational outcomes at a national level, esp. reviewing school-leaving examinations in core subjects (Czech language and other foreign languages - English, French, German - mathematics and possibly in other eligible subjects);
     
   • Working together on content analysis of examination papers, item construction, empirical study of levels of proficiency during pre-testing and analysis of pre-test results;

1.5. Effects on class teaching

The last pilot studies MATURANT 97 - 99 had clear impact on the class teaching in many secondary schools, but we are afraid, that using multiple-choice items only can cause an orientation of teaching only on reproductive abilities of students.

ISSUE 2: THE EXAMINATION AS A SCHOOL-LEAVING "MATURITY EXAM" (ABITUR, MATURA, ETC.) AND AS AN "ENTRANCE EXAMINATION" TO HIGHER EDUCATION (UNIVERSITY):

2.1. Should the "final examination" serve
a) Both aims - Why and how?
b) Only as the school leaving "maturity exam" - Why and how?
c) Only as the "entrance examination" to higher education - Why and how?

All secondary school leavers must pass the final exams successfully, otherwise they can not be accepted to higher vocational schools, colleges and/or the university. Maturita qualification acquired at Secondary Technical and Vocational Schools is regarded in law as equal to Maturita qualification from a Gymnázium and entitles the holder to apply for studies at any university. These students therefore have become the "core" of the applicants accepted at higher education institutions as far as studies of a technical, agricultural, economic or artistic nature are concerned.

We expect the final examination to serve both the aims. Due to the autonomy of Czech universities, it is up to each university what load it gives to the final exam. The general aim of the planned reform of final exam is the expectation, that the universities will in future give more value to the final exam.

2.2. Final examinations as an intermediary between secondary and higher education and/or a way to a vocation and employment

Upper secondary education is a multi-structured but internally co-ordinated system. It guarantees education and practical vocational training for almost the entire population of young people following the completion of compulsory education and preceding their take up of employment or their continuation in higher education. The Maturita certificate is obtained by all graduates of four-year (exceptionally five-year) branches of study, or branches of the extension studies. These certificates declare the capability of the bearer to perform a vocation included in the characteristics of study branches at the scope established by the relevant profile of a graduate and attest to the attainment of the educational and professional goals as established by the relevant standards. Moreover, the Maturita certificate attests to its bearer's ability for further study at universities, colleges or higher schools provided s/he passes the entrance examinations.

2.3. Individual achievement at final examinations as a selection criteria for admission to higher education (e.g. in case of numerous clausus, etc.)

The individual achievement of students at the final examinations in secondary schools has only very limited impact on their admission to universities, because each higher school has its own criteria and the quality of the student's achievement during Maturita exams has so far had very limited role on their entry to the university.

ISSUE 3: COMPULSORY AND OPTIONAL SUBJECTS OF THE FINAL EXAMINATION:

3.1. Proportion of compulsory to optional subjects and the aim of such a division
The recent Maturita at the Gymnázium consists of two compulsory subjects (Czech language and literature and one foreign language) and two optional subjects. The examination is in a written and an oral form. The school determines the content of the Maturita examinations; clear and objective requirements are not set externally. The Maturita at Secondary Technical and Vocational Schools comprises of a written and oral examination in Czech language and literature, of an examination in an optional subject and of theoretical as well as practical examinations in technical subjects. The content of the examination in technical subjects is set by the school head and, as at the Gymnázium, it does not have the character of an external examination.

The Ministry of Education expects that the future student will have to complete three compulsory subjects (Czech language and literature, one foreign language and mathematics) during the state (external) examinations.

3.2. Role of mathematics and foreign language(s)

The role of mathematics and foreign languages is seen as a very important part of the future final examinations. Even though the foreign languages are accepted by students, mathematics is producing controversial attitudes. The Ministry of Education expects, that all students should take the final external examination in mathematics minimally at the basic level (applied, "practical" mathematics).

3.3. Optional subjects: only academic or also others?

We expect that the optional subjects will be a regular part of the internal examinations. Students of the secondary general schools (Gymnázium) will be allowed to choose from a list of optional academic subjects, students of technical/vocational schools will choose from a list of "profile" subjects (according to their next higher orientation).

3.4. Are students specially prepared for various levels of difficulty?

The contemporary model of the final examinations does not discriminate between the levels of difficulty. The reform of the final examination system expects differentiated levels of the external final examinations. The level of difficulty will be chosen by the student without respect of his/her study at the attended secondary school.

ISSUE 4: ASSESSMENT AND CERTIFICATION OF ACHIEVEMENT:

4.1. Definition of national standards (if defined)

In recent years, the Ministry of Education has concentrated to a great extent on the drafting and elaboration of quality standards. In the Czech Republic there exists a special national standard of education for the Gymnázium and a special one for vocational (technical, business, pedagogical) schools. Both the standards are defined by content and competencies, which must be achieved by the students. The goals and the student's final competencies are unfortunately rather too elementary and not sufficiently concrete. As a part of the implementation of the external examinations into the Maturita process, there is expected the creation of new evaluation standards where the output requirements for both levels of difficulty of external examinations will be precisely defined.

4.2. What is to be measured: knowledge alone or also other skills and abilities?

The contemporary model of the final examinations was oriented mainly on the assessment of knowledge. We expect that new final examination model will emphasise the assessment of key competencies (skills, abilities). There is also expected that the emphasis on mentioned competencies will be the same during the internal examination process.

4.3. Role of external assessors (experts, teachers, university teachers…)

Recently, the assessment of the student achievement is in the responsibility of the School Maturita Committee. This committee consists of teachers from the relevant school. The head of the mentioned committee is a qualified teacher from another secondary school. The activities realised in this committee are part of teacher's workload. During the administration of the pilot study Maturant 97 - 99 inspectors were involved as administrators of the subject tests. Many teachers of secondary schools were involved in the next pilot studies of Maturant 98 - 99, but only after they had completed a short training in the administration procedures. Immediately after the nation-wide examinations had been administered, a responsible organisation (ÚIV) processed the scoring data of all students in all examination subjects, and the results were submitted back to secondary schools. Every secondary school could then compare the results of their students with other secondary schools. The involvement of experts in evaluation and assessment is still very limited, but many state institutions and some private agencies are involved in the above-mentioned process concerning the preparation of the external examinations.

4.4. Development of question papers

The tests in the last nation-wide pilot-studies consisted of a number of multiple-choice questions only. The subject tests, consisting entirely of objective questions, i.e. precoded questions, were machine-scored according to an accompanying answer key. Despite the use of criterion-based tests, there is great discussion weather to use only multiple-choice items or also open-ended items in the future tests. The answers to the open-ended questions or assignments will be assessed or scored in the future external examination by teachers on the basis of centrally drawn up marking schemes. A short description of the implementing organisation of our country is as follows:

1. The Institute for Research in Education (Výzkumný ústav pedagogický v Praze, VÚP) is a state organisation run by the Ministry of Education. It is responsible for the development of the curriculum and the standards for primary and general secondary education.

2. The Centre for Educational Evaluation and Assessment (CEEA)
   At the beginning of the second half of 1997 there was established a specialised centre for the measurement and evaluation in education which is part of the Institute for Research in Education (Výzkumný ústav pedagogický - VÚP). The main tasks of the Centre for Educational Evaluation and Assessment (CEEA) is to develop the final examination test papers, as well as other tools and instruments of educational and psychological measurement for the primary and secondary schools on the mezzo and micro levels. Part of the project objectives would be to provide the CEEA and other state and private institutions, involved in the quality control in education, with specific school measurements, related examination instruments, methodological procedures and technical assistance, concerning the standardisation of the external written examinations in secondary schools, which will be in the future regularly produced by the mentioned institution.

3. The Research Institute for Vocational Education (VÚOŠ)
   Research Institute for Vocational Education is a state organisation run by the Ministry of Education. It is responsible for the development of the curriculum and the standards of vocational education.

4. The Czech School Inspectorate (ČŠI)
   Currently, the assessment of the quality of education is the responsibility of the Czech School Inspectorate. Often, the work of the institution is affected by the lack of clarity of the goals and procedures followed in their operations. This circumstance then impacts unfavourably the way in which external assessment of the quality is viewed by the schools. The CSI has already started the process of necessary changes.

5. The Institute for Information on Education, Prague (ÚIV)
   The Institute for Information on Education (administrative and organisational preparation, guarantee of actual testing at schools, result processing, technical and informative study support) is a state organisation run by the Ministry of Education.

6. The Foundation SCIO, Prague
   SCIO Foundation (test items preparation and part of result processing) is a private institution operating in education, which was chosen by means of competition by the Ministry to prepare MATURANT 98. It developed a system of norm-referenced tests for the students of secondary schools (The National Comparative Examinations) in many subjects (mathematics, Czech language and literature, foreign languages, history, test OSP - a Czech version of the SAT e.g.), which was designed to help students in the decision-making stage to enter a higher school or a university.

7. The Foundation KALIBRO, Prague
   KALIBRO Foundation is a private institution oriented not only on students of secondary schools, but it serves the same evaluation service for principals of the primary schools.

There is a school examination committee, which is responsible for the grading of the student's achievements. The results of the final internal examinations are announced to the students the same day when s/he takes the examination. During the three last experimental pilot studies of the MATUTANT 97 - 99 all the results were machine-scored. All test results, with multiple-choice items only, were machine scored (in 1999 it was 115.000 students of secondary schools who sat the exam) by a professional computer firm and then sent during 14 days to each secondary school.

ISSUE 5: ORGANISATION OF THE FINAL EXAMINATION:

5.1. Beginning and duration of the final examination: one or more examination sessions per year?

The internal final examinations have two regular sessions. The first session commonly falls on the second half of May. The second session is for students who were unable to attend the first term or failed in the final examination. The second session commonly falls on the first half of September. The organisation of the examinations is defined by state regulations and lies in the hands of the school management. We expect the model, which will be implemented, to follow the same pattern. The last period in which the nation-wide pilot studies were held were announced to secondary schools well in advance, at the beginning of the school year.

5.2. Rules for and the supervision of examinations - Is supervision part of teacher's workload or is it paid separately (If so, how much - in Euro)?

In the future, there is expected a large participation of secondary school teachers in the test creation, the assessment and the administration of the external examinations, but this process must be preceded by a very intensive in-service training. Their supervision during the last pilot studies was not accepted as a professional workload. The teachers involved in the administration of subject tests have been so far paid as external co-workers separately. We expect, that the teachers involved in this process will be paid in recompense for their activities in many responsible bodies (school subject committees, school Maturita committee, national subject committees e.i.).

5.3. Regulations for students with special needs

The students of secondary schools with special needs were in the last pilot studies excluded. The secondary education of students with special needs is in the Czech republic secured not only by special secondary schools for students with hearing, mobility and blind impairments, but many of these students are integrated into regular secondary schools (approximately 3 - 5%, especially LD students). These questions are not now resolved and are the topic of the discussion.

Note 1 - The new Maturita exam model will preserve a "school /internal" part of the Maturita exam, which will remain under the school director's authority, the integration of "external" and "school" part of the Maturita exam" should maintain the educational variety and should not limit school director decision-making abilities, and thus make Maturita comparability possible.