|
The General Directorate of Examinations and Tests in the Ministry
of Education began work on the Examination Reform and Assessment
Project (ERAP) in 1995, and continued until the year 2001. The project
was supported by the British Department for International Development
(DFID), and managed by the British Council in Jordan. The DFID support
was concentrated on training (in the UK and Jordan) and providing
technical assistance inside Jordan. The project had the following
main components:
• Tawjihi
• Diagnostic Assessment
• Investigation
• Achievement Tests
• Quality Assurance
• Assessment of Practical
Training
• Statistical Analysis
• National Test
Development of General Secondary Education Certificate Examination
(Tawjihi)
The main objective of this component is to develop the General Secondary
Education Certificate Examination (Tawjihi) by introducing new methods
of assessment, designed to measure a full range of curriculum objectives,
and in particular to develop the measurement of the higher mental
abilities, such as analysis, evaluation and problem-solving.
Work on this component started in mid-1995. A Specialised Committee
was constituted for each main subject in the academic sector, and
the commercial branch of vocational secondary education. This committee
included the GDE supervisor of measurement and evaluation for the
subject, a representative of the General Directorate of Curriculum,
and a number of subject supervisors drawn from the field. The committee
members were given initial training in Jordan and the U.K by experts
from the Scottish Examination Board.
The committees, with consultant support, then analysed the school
textbooks and curricula, and identified the content and the abilities
being taught. They developed examination specification tables, to
ensure that the examination was comprehensive (i.e., it assessed
all the content and abilities identified) and balanced (i.e., all
the various elements were given their due weight).
The committees then developed exemplar questions, to assess the
abilities of knowledge and understanding, as well as handling information,
analysis and evaluation, problem solving and the like (depending
on the subject matter being assessed). They also developed “arrangements
documents” (i.e., guides to how to prepare a good examination)
for all the grade 11 and 12 courses. These included guidance on
content analysis, specification tables, and exemplars of questions
designed to measure the various abilities, and a procedure for test
analysis, to assure that the test measures the content objectives
and abilities in a balanced way. These guides were distributed to
the Heads of Supervision in all the Directorates of Education. Specimens
of these documents, in both English and Arabic, are available from
the GDE.
In May 1996, in every subject, an experimental test was given to
a random sample of academic and commercial students of the first
secondary level. Feedback questionnaires were also administered
to supervisors, teachers and students. This exercise had two main
objectives – to introduce the new type of general examination
to students, teachers and supervisors, and to get their feedback.
The results were analysed statistically, and a report prepared.
In the summer of 1996, the GDE trained the subject supervisors and
the teachers of the second secondary classes (academic and commercial)
on the new assessment methods.
During the 1996-97 academic year, the development process was extended
to the other vocational branches (i.e., nursing, industrial, agricultural,
hotelier and home economics). Similar content and ability analyses
were carried out, new models for assessment were developed, and
arrangements documents produced to inform supervisors and teachers.
This project has produced significant changes in the structure of
the General Secondary Education Certificate Examination. These include
changes in the structure of the examination papers themselves. They
also include changes in way in which the examinations are constructed
using specifications prepared in advance, to ensure that the examination
measures in a balanced manner the goals of the curriculum, and assesses
a range of abilities, from simple recall to higher mental abilities.
Training workshops for supervisors and teachers have also been conducted
in coordination with the General Directorate of Training. These
are designed to develop their skills in constructing school examinations.
These have been followed up by field monitoring of the school examinations,
and giving feedback about them to get advantages and benefit from
them in the field of teaching and assessment.
During the 2000-2001 academic year, a higher specialised committee
was constituted to assess the training manuals. It was supported
by specialised sub-committees for each of the academic and vocational
courses. The training manuals have been reviewed and revised by
these committees. Following the revision, the General Directorate
of Curriculum will add them to the teacher’s books for Grades
10-12 in the academic subjects, and issue an independent manual
for grades 11-12 in the vocational branches. This will promote the
adoption of the principles and models developed during the project
in constructing school examinations.
Click
here to find downloadable resources about this subject.
Up
Diagnostic Assessment
The introduction of a Diagnostic Assessment approach in different
subjects was one of the recommendations of the 1987 National Conference
on Education Reform. Diagnostic assessment is a teaching-learning
method which involves systematic gathering of information about
individual students' attainment to identify their strengths and
weaknesses. Many different types of assessment activity are used
– formal testing, classroom questioning, observation of seatwork,
etc. Teachers use assessment to identify individuals who are having
difficulties, and those who are successful, and allocate them to
groups. They design or select appropriate classroom activities to
meet the needs of each group. Some groups may receive remedial activities,
designed to help them overcome specific difficulties. Others, who
have mastered the basic objectives, may receive enrichment activities
designed to challenge them. The teacher then monitors performance,
and the cycle repeats. Diagnostic Assessment aims at furnishing
help for the teachers to identify the educational obstacles which
confront their students, as well as to put forward some remedial
plans for these obstacles and to identify the sources of power so
that to enrich and consolidate them.
Following an initial period of preparation, including overseas training
for key staff, the General Directorate of Examinations began introducing
Diagnostic Assessment in Arabic Language and Mathematics in May
1995, though a programme of school-based development and trialling
of materials. Development began in Arabic Language and Mathematics,
at grade 6 level. Impact assessment proved that it had a positive
impact on student learning achievement, and it was subsequently
extended to a wider range of grade levels, and to Science and English
Language.
A range of guidance and training materials have been developed,
as well as demonstration videocassettes. These have been distributed
to all Directorates of Education. Local subject supervisors and
teachers have been trained on planning and implementing diagnostic
assessment. An introduction to Diagnostic Assessment for teachers
has also been prepared, and will be distributed to all the teachers
around the Kingdom.
Diagnostic assessment is an effective way of giving attention to
individual differences, and ensuring that as many students as possible
perform to the highest level that they can. Teachers report many
benefits from using diagnostic assessment. They say the diagnostic
assessment processes help them to clarify their objectives, give
them a deeper understanding of their subject better and alert them
to what students are likely to find difficult. They also report
that know their students much better as individuals, and have much
more detailed and meaningful information about their strengths and
weaknesses. As a result, many teachers feel empowered, able to help
their students in a much more positive way. Students also value
the information they get about their strengths and weaknesses, and
say they feel their education is more individual and more effective.
In addition, some diagnostic tests have been prepared in Arabic
and English Language for grade 7. These have been piloted, and guidelines
for interpretation developed. Training materials on diagnostic testing
have also been developed, and are available from the GDE.
Click
here to find downloadable resources about this subject.
Up
Investigation
Essentially, investigation is a way of developing students’
abilities to use research, thinking and practical procedures to
solve a problem. It is both a teaching method and assessment at
the same time. In most cases, there are many ways to obtain an answer
to the question; but while each of them may give an answer, some
may be better than others. Students go through four basic stages
to carry out a typical investigation – planning, collecting
evidence, processing information and presenting findings.
Investigation therefore helps students to develop a variety of abilities,
including:
• The ability to apply textbook learning to real-life problems,
analyse the critical elements of them, and choose and apply appropriate
techniques to solve them.
• The ability to plan and write the reports, including fluency
in language, mathematics and creative thinking.
• The ability to seek out different sources of information
and select relevant material from them.
• Self-confidence, and the ability to interact with teachers,
schoolmates and society at large
• The ability to cooperate with others.
• Skills in interviewing, recording and recording.
• Practical skills such as drawing and photocopying.
Work in this component started in mid-1997, with a programme of
school-based development and trialling in science, mathematics and
social sciences. Impact assessment has showed that the approach
is very effective in producing both educational and motivational
benefits; and it has been extended to Commercial Science and Islamic
education.
A general introduction to investigation has been prepared, and distributed
to all teachers in the Kingdom. Teacher’s guides, training
materials and exemplar investigations have been prepared in five
subject areas, and distributed to all educational directorates.
A large number of subject supervisors and teachers have been trained
using these materials, and investigation is now in widespread use.
Click
here to find downloadable resources about this subject.
Up
Achievement Testing
Once the new models for the Tawjihi had been developed, work began
on disseminating the principles and practice underlying them to
teachers and supervisors at a wide range of grade levels, in all
academic subjects and vocational branches. During the first half
of 1988, the GDE developed and refined the guidance materials produced
for the Tawjihi, to make them applicable to a broad range of grade
levels. Training materials in specification design, and setting
questions to assess different abilities were produced in each subject
or branch. These materials have been distributed to all the directorates
of education.
Training workshops in achievement test design and question setting
began during the summer of 1998, and have continued since that time.
During 2000, an introductory guide to constructing achievement tests
was developed, to be distributed to all the teachers in the Kingdom.
At the end of the second semester of the school year 2000/2001,
each Directorate of Education began to administer achievement tests
in at least two subjects for both the tenth grade and the first
secondary class (grade 11), following the criteria prepared by the
Directorate of Tests. The purpose of these achievement tests is
to inform students and teachers of students’ standards of achievement,
and to introduce them to different types of question which assess
the various skills and higher abilities, and thereby help the student
to prepare for the examination of the general secondary education
certificate.
Click
here to find downloadable resources about this subject.
Up
Quality Assurance
Qualifications are the currency by which many students, especially
in the vocational sector, achieve employment. It is essential therefore
that students, parents and employers have confidence in the qualifications
offered by the education system, that the qualifications are relevant
to needs in Jordan now and in the future and that standards are
comparable to other countries. This is particularly important in
the case of qualifications like the School Certificate – Vocational
Subjects, which is set, administered and scored by the school, with
no external monitoring.
A rigorous quality assurance system is essential for maintaining
public confidence in the School Certificate – Vocational Subjects.
Such a system provides for standards of assessment to be defined,
for the degree to which standards are met to be measured, for any
identified problems to be solved and improvements implemented. Commitment
to quality has other benefits also, it means taking action continuously
to improve all aspects of the processes for assessment and award
of certificates and thus contributes to maintaining the relevance
and ‘currency value’ of the certificate.
Beginning in mid-1998, the GDE developed a system of quality assurance
for the School Certificate – Vocational Subjects, with consultant
assistance, and an overseas attachment programme for two key staff
members. This system defines the roles of the “internal assessor”
– i.e., the teacher, and the responsibilities for providing
guidance (mainly with the GDE) and internal and external verification
(through the school coordinator, local supervisor and GDE supervisor).
A training package was prepared to introduce the procedures of quality
assurance, and is available for download from this site, or as hard
copy from the GDE.
Piloting of this system began in the clinical and industrial branches
of vocational secondary education, in schools in two directorates.
It started in the beginning of the 1999/2000 academic year. Impact
assessment proved broadly positive, and the trial was extended to
a sample of schools in the other vocational branches, during the
school year 2000/2001.
Click
here to find downloadable resources about this subject.
Up
Assessment of Practical Training
Vocational education plays an important part in the development
plan for the economy in the Kingdom, as the well-being of the economy
relies on the abilities and skills of the workforce. The need to
ensure the provision of a pool of skilled workers begins in schools
where students must be given the highest level of practical training
in vocational education. This in turn depends on the presence of
an efficient training staff who must be provided with the best form
of training both before becoming teachers and while in service,
using resources which reflect the kind of resources used in industry
and taking account of the latest technological developments. A vital
element in training for vocational education is the establishment
of an assessment system to ensure that skilled students can be identified.
This must be a nationwide system where the student in Amman and
the student in the north or south of Jordan is assured of a quality
assessment process which gives them an equal chance to demonstrate
their expertise, knowing that they will have a fair and just evaluation
of their abilities with a prized certificate acknowledged by all
who use the certificate to make judgements on the pupil’s abilities
for employment or further education.
It is important that all teachers of practical subjects apply the
same standards to their assessment instruments to ensure that candidates
in all directorates have equal opportunities of demonstrating their
skills to the best of their abilities. To achieve this goal, teachers
must be aware of the need to ensure that their tests are both valid
and reliable. For this reason, the GDE has carried out a pilot project
across all the vocational branches (agricultural, commercial, nursing,
industrial, hotelier and home economics) to establish standard and
relevant criteria and practices for the assessment of practical
training, and to train the teachers of practical subjects in assessment,
so as to raise the quality of their own assessments.
Work on this component began early in the year 2000, with a training
programme in the UK for the GDE supervisors of the six vocational
branches, during which draft training and assessment materials were
developed. The supervisors of the vocational streams and a sample
of teachers were trained on the principles of assessing practical
training, and the specific techniques to be used, and the assessment
materials developed during this training were piloted in 35 schools
across 16 directorates, during the academic year 2000-01. A statistical
analysis of the students’ results has been made, both to assess
the range and level of students’ practical skills and to provide
remedial feedback, to enable teachers to deal with the students’
weaknesses in the future.
Click
here to find downloadable resources about this subject.
Up
Statistical Analysis
Work on this component began in 1997, when the first statistical
analysis of the Tawjihi was conducted, to assess the impact of the
changes in examination procedures introduced during that year. The
initial purpose was to assess the relative difficulty of different
streams, using the newly-introduced common core subjects as a proxy
for overall student ability/achievement, and the relative difficulty
of different optional subjects, using data from the different pairs
of options chosen.
Since that time, a more general statistical analysis of the results
of the examination of the general secondary education certificate
has been conducted for the academic years (1997-98) and (1998-99).
The analysis is conducted at the question level, to assess question
difficulty and the correlation between question score and total
score, and the whole test level, to assess the mean and standard
deviation, and the relationship between the total scores on the
various subjects.
A joint committee has been established, consisting of professors
of assessment and evaluation from the Jordan University and specialists
from the Ministry of Education. The role of this committee is to
study the statistical analysis results, to identify weaknesses both
in the test and item construction, and in students’ performance.
This information is used to make recommendations for the coming
year’s general examination, as well as to inform the Ministry
of Education about the weak points among the students, as a basis
for remedial work.
Click
here to find downloadable resources about this subject.
Up
National Test
The national test aims at achieving the following five major goals:
• To provide Ministry of Education specialists with necessary
information about the extent to which the students of the Kingdom
as a whole have achieved the major objectives of the national curriculum
for basic education, as a key element in overall quality assurance.
• To provide information about what the typical student knows,
and can do, on graduation from the cycle of basic education, as
an indicator of overall educational quality.
• To analyse the strengths and weaknesses of typical student
performance, as a basis for setting targets for strengthening overall
student learning performance.
• To provide detailed information about the strengths and weaknesses
of typical student learning achievement, as a basis for setting
priorities for developing curriculum, instructional materials and
teaching methods.
• To act as a model of good assessment practice for schools
and Directorates of Education.
Development work in this component started in the second half of
1999, in six subjects:- Islamic Education, Arabic Language, English
Language, Mathematics, Social Sciences and Science. In each subject,
the content of the textbooks for the basic stage was analysed, and
a specification document was developed covering the main content
areas and skills. In English Language, the listening skill was included.
Items were developed centrally according to the specification, piloted
and item-analysed. In each subject, a final test form was assembled
according to the specification, from the items that met the item
analysis criteria.
The first national test was administered in May 2000, on a 5% sample
of the 10th grade students in all directorates of education throughout
the Kingdom. This percentage was equivalent to about 4500 male and
female students, who were drawn from 150 schools which were randomly
selected. Equal numbers of boys’ and girls’ schools were
chosen, and the number of rural and urban schools chosen reflected
the number of rural and urban students. The number of schools selected
from each directorate of education was chosen on the basis of the
number of students in the directorate. Within each directorate,
the required number of schools were randomly selected. All the students
in a single section were tested, up to a maximum of 30. Where the
number of students in the section was less than the required minimum,
students from another section were added where possible, in alphabetical
order.
The test papers were corrected centrally, to guarantee uniformity
of criteria. The students’ results were entered to the computer
and statistically analysed. A report was prepared, including both
statistical analysis and some analysis of students’ achievement
against broad goals, and variation in performance by gender and
school.
In 2001, the process was repeated, with some modifications. The
test instruments were refined and developed. A short listening test
was introduced in Arabic language. In May 2001, the test was administered
to a 5% sample of ninth grade students including all the directorates
of education, military, private and UNRWA schools.
Specification documents and reports for 2000 and 2001 can be found
on this site.
During future years, the national testing will be extended to include
the primary stage. Item banking using item response theory (IRT)
calibration will be introduced, to allow different forms of the
national tests to be equated, for example from year to year.
Click
here to find downloadable resources about this subject.
Up
|
