The Buckingham Journal of Language and Linguistics Volume 3 pp 1-9
Assessing pupils’ language proficiency in a bilingual education programme: Insights from two primary schools in Malawi
Andrew Chimpololo*
Assessment in bilingual education is fundamental as far as cognitive academic language competence (CALP) is concerned. Teachers have to continuously evaluate the level of their pupils’ linguistic competence to accordingly tailor instruction towards the attainment of CALP (Cloud, Genesee and Hamayan 2000). This paper presents the findings of a study that investigated the assessment techniques used by teachers at two primary schools in Malawi’s transitional bilingual education programme. The results indicate a monotonous use of standardized assessment tools such as written exercises, oral assignments and end-of-term tests. The teachers seem not to realize that performance evaluation is an on-going process that constantly provides important feedback on pupils’ academic language proficiency. The teachers also ignore the focus on high-level cognitive abilities in their assessment. Furthermore, there is little variety in their choice of assessment methods to accurately measure different aspects of the pupils’ linguistic competence. To the contrary, research advocates that varied and continuous assessment techniques are the best ways of evaluating pupils’ academic performance in bilingual education programmes.
Assessment is one of the most controversial issues facing
bilingual educators in the developing world. While schools are obligated to
systematically monitor the linguistic competence of pupils in the second
language, at the classroom level teachers strive to know pupils’ levels of
proficiency in order to gear instruction appropriately (Hargett 1998). In
Malawi’s bilingual education programme, there seems to be no clear guidelines
on how the curriculum and assessment have to be aligned so as to accurately
measure pupils’ progress in the acquisition of English.
In their critical features of ideal bilingual education, Cloud, Genesee and Hamayan (2000) emphasize the role of assessment in any educational programme – bilingual or not. They observe that bilingual education succeeds only when teachers are reflective about the curriculum and instruction in their assessment. Teaching should only be modified in response to the results of formal and informal assessment of pupils’ progress and to the teachers’ observations of the appropriateness of curriculum materials and activities. This paper presents the findings from a study that investigated the assessment orientation of teachers at two primary schools in Malawi.
The study was part of a broader needs analysis research that examined the pedagogical practices at the two schools in light of Cummins’ (1986) four major characteristics of bilingual schools and Cloud et al’s (2000) critical features of ideal bilingual education. However, this paper only discusses the tools of assessment used by the teachers and their implications on the measurement of the English language proficiency of the pupils. It also examines how language learning and assessment is integrated in content-related subjects.
Malawi was already
multilingual when the British first came in the last half of the 19 century.
There were at least 12 local languages being actively spoken in the country
(Kayambazinthu 1998). However, the first language policy was introduced by the
colonialists in the early 1900s (McCracken 1968). Under this policy, English was
the official language in the domains of administration, judiciary, legislature,
commerce and trade while two local languages – Chichewa and Chitumbuka – served
communication functions in the education and administration sectors.
In 1966, two years
after her independence, Malawi adopted a second language policy that declared
English and Chichewa as the country’s official languages. The two languages
were also mandated as languages of instruction in the education sector. When
Malawi adopted multiparty politics in 1994, it marked a turning point in the
language policy for the nation. There were calls from various quarters for a
revamp of the policy to incorporate other local languages in the system. The
reasoning was that in a democracy, respect of human rights was a priority, and
that includes free choice of language in the diverse forms of communication. In
1996, government took a significant but controversial step that directed
teachers to use local languages most familiar to the pupils in their area as media
of instruction in Standards 1 to 4. However, this directive met heavy public
criticism. It was argued that the move lacked proper preparation and
consultation. In response, the government reversed the decision and
commissioned the Centre for Language Studies at the University of Malawi to
conduct sociolinguistic studies on how best the major local languages could be
incorporated in the education system. A new national language policy has now
been formulated and is awaiting government approval for implementation.
The education system
in Malawi is basically divided into three levels: primary, secondary and
tertiary although pre-school instruction is available to those who can afford.
Primary education is further classified into the infant section, which has
Standards 1 and 2, the junior section comprising Standards 3, 4 and 5, and the
senior section which runs from Standard 6 to 8. In Standard 8, pupils sit for
national examinations referred to as the Primary School Leaving Certificate
Examinations (PSLCE). Chichewa is the medium of instruction from Standard 1 to
4 and doubles as a subject. However, English is offered only as a subject at
this level. From Standard 5 onwards, English takes over as the medium of
instruction while Chichewa continues to be taught as a subject in the
curriculum. This abrupt language switch is an issue of major concern to
educationists as it likely impedes pupils’ academic success.
It can be observed
that two languages, Chichewa and English, are the only media of instruction at
the primary school level in Malawi’s transitional bilingual education
programme. What is remaining though is the incorporation of the other major
local languages as tools of instruction during the first four years of
education alongside Chichewa. Under the new policy, the commonly spoken
language in a particular area or district will automatically be assigned as the
medium of instruction in surrounding schools since languages are geographically
distributed in Malawi.
This study was conducted in Blantyre City at two public primary schools – Zingwangwa (ZPS) and Chimwankhunda Dam (CDPS). The two institutions share the same sociolinguistic features, with pupils coming from families with diverse linguistic and socio-cultural backgrounds. The study involved teachers and pupils at the two schools. The teacher subjects were selected using two sampling methods. Firstly, they were chosen through simple random sampling whereby, according to Borg (1981), all individuals in a defined population have an equal and independent chance of being selected as a member of the sample. From numbered lists that were categorized according to the eight Standard levels, the researcher picked the odd numbers until 10 teachers were obtained for each school. This means that for each school, there was a teacher from each of the eight Standard levels and an additional teacher for Standards 1 and 2 to tally the total to 10. Secondly, convenience sampling was used to select an additional 5 teachers for each school from Standards 4 and 5. The responses of these teachers were crucial in determining the preparedness of Standard 4 pupils for English-only instruction in Standard 5 and beyond. Thus, altogether 30 teachers took part in this study, with 15 teachers coming from each of the two schools.
Ten pupils – 5 from each school – participated in this study. The pupils included two from Standard 5 pupils, one from Standard 7 and two from Standard 8. These pupils were selected because of their academic experience of both the Chichewa-medium instruction (from Standard 1 to 4) and English-only instruction (from Standard 5 onwards) which made them well-placed to make substantive and informed comments about the language switch that occur from Standard 4 to 5. For the classroom observations, complete classes were necessary. Observations of classroom practices were therefore conducted in Standards 1, 4 and 5 at both schools. Standard 1 was chosen as it forms the foundation stage of academic language development while Standards 4 and 5 provided the opportunity of examining the language transition and its effects.
The study adopted the combined approach by employing both qualitative and quantitative research tools. It used classroom observations and two questionnaires for data collection. The classroom observations, which lasted 12 days, were less structured to reduce the ‘intruder’ effect. To ensure that the process lead to a professional growth, the observations were characterised by Hopkins’ (2002) five key features of classroom observation. Firstly, a joint planning meeting involving the researcher and the teachers preceded the first series of observations. This led to the establishment of a climate of trust between the observer and the observed, and an agreement on the ground rules such as a timeframe for the observations and place where the researcher had to sit during the observations. Secondly, the observation approach was made as specific as possible. Items to be observed were thus predetermined and acted as reference points during the exercise. As Hopkins says, the more specific and negotiated the focus of the classroom observation, the more likely it is that the data gathered will be useful for developmental purposes.
Thirdly, a criterion in the form of a schedule or checklist was established to ensure that the contribution of the classroom observations to professional development was greatly enhanced. The criteria acted as a roadmap for development as well as a provision of standards by which to discuss the outcomes of the observations. It also allowed the gathering of appropriate information on classroom behaviour. Data was continuously recorded in the research diary during the exercise.
Fourthly, the researcher was always conscious of his observation skills during the exercise. He guarded against the natural tendency to move quickly into judgement by having a clear focus of the observation and agreeing on the ground rules beforehand. He also used his interpersonal skills a lot, especially realising that he was invading another person’s private space, to create a sense of trust and support in situations where the teacher may have felt threatened.
Lastly, the researcher tried as much as possible to make sure that the observation results were neither rushed nor impressionistic. To achieve this, the results were based on factual data that was carefully and systematically recorded during the process. This proved very helpful in developing strategies for building on what has been learnt. Adherence to these key features assisted the researcher to collect objective data on the aspects of classroom behaviour that were targeted.
Each observation session was approximately 35-minutes long and the same standard level was observed in a particular day in the two schools to allow on-going analysis. This method was chosen because it provided the best opportunity of understanding the instructional and assessment processes (Estacion, McMahon, Quint, Melamud and Stephens 2004). Altogether, 24 sessions were observed at the two schools – 12 sessions each school. However, 9 sessions were for English classes and 3 for mathematics for each school. The mathematics classes were included to check whether teachers integrated language learning and assessment skills with curricular content. The exercise was directed by an observation schedule that was developed to enable researcher focus on specific aspects of the study.
As an addition to the schedule, the researcher developed a running record form – adopted from Estacion et al. (2004) - on which he recorded all classroom details that were observed during each session. The observer thus focused purposefully on capturing the specifics of assessment practices while making high inference judgements about them. While the running record form enabled the researcher to time the assessment practices, the observation schedule helped him to stay focused. Thus, the two instruments were simultaneously used during the observations but served different purposes.
The teachers’ questionnaire was administered to 15 teachers. The questions focused on the nature of assessment techniques used by teachers to provide a good comparison to the observation findings. On the other hand, the pupils’ questionnaire obtained information on the pupils’ assessment experiences and their effects on performance. To avoid interpretation anomalies, the researcher asked the questions orally and then recorded the responses on the questionnaire by himself.
Data obtained in this study was largely analysed using two frameworks. Firstly, Becker’s (1958) analytical induction necessitated field during the data collection exercise itself. And the second framework, the grounded theory by Glaser and Strauss (1967), involved constant comparison whereby the researcher initially categorised the data by comparing incidents applicable to particular categories and then refined and integrated the emergent categories and their properties through axial coding (Creswell 2002).
To determine the assessment orientation of the teacher, the researcher recorded the number of times that a particular assessment technique was used. In terms of instructional tasks, the researcher analysed the types of questions – oral or written - that the teacher asked or gave the pupils during a session to determine whether they involved low-level or high-level cognitive processes. He then made a general evaluation toward the end of a session to determine the overall conclusion. For validation purposes, results from the classroom observations were then compared with the responses given by the teachers themselves and the pupils in the questionnaire survey.
This study investigated the assessment orientation of teachers at ZPS
and CDPS. The results indicated that teachers at both institutions limit
themselves to standardised modes in assessing pupils’ academic achievement. In
all the 24 sessions observed at ZPS and CDPS, teachers either used written
exercises or oral questions to evaluate the pupils and never kept records of
the pupils’ progress for their future reference. Class exercises were given
there and then, and so did they matter. Thus, the assessments were momentary
rather than continuous. Consistent with
the classroom observations, all the teachers indicated in the questionnaire
survey that they use standardised techniques in their assessment of pupils’
performance (see Table 1 below).
TABLE 1: Comparison of results from
classroom observations and teachers’ questionnaire on assessment techniques
|
|
ZPS |
CDPS |
||
|
Observations (as a % of the 12 sessions) |
Questionnaire (as a % of the 15 teachers) |
Observations (as a % of the 12 sessions) |
Questionnaire (as a % of the 15 teachers) |
|
|
Use of standardised assessment tools |
100 |
100 |
100 |
100 |
|
Use of continuous assessment tools
|
0 |
0 |
0 |
0 |
However, as depicted
by Table 1 above, there was no evidence on the use of on-going assessment tools
at ZPS and CDPS. None of the teachers assessed their pupils using continuous
techniques such as observations, conferences, portfolios, and dialogue
journals. According to Cloud et al. (2000), conferences involve semi-structured
face-to-face conversations between a pupil and a teacher about work that the
pupil has completed or is working on. During portfolios, on the other hand, a
teacher purposefully collects a pupil’s work that documents their efforts, achievements,
and progress over time in given areas of learning, either language or subject
matter, or both. Hudelson (1994) describes dialogue journals as written or
orally-recorded discussions between pupils and teachers about school-related or
other topics of interest to pupils. He further notes that utilising oral and
written personal narratives of pupils is important as it provide teachers with
useful information for individualizing instruction.
In their responses, the pupils reaffirmed the teachers’ preference of
standardised assessment tools. All the 10 pupils indicated that their teachers
only use written exercises, written tests and oral questions for assessment
although one pupil added an isolated case where conferencing was used. The
teacher had arranged a five-minute chat with the pupil to clarify instructions
on tackling a written exercise. Contrary
to the state of affairs at both institutions, teachers in bilingual education
need to plan for language growth as they do for content and vice-versa (Met
1994). Met notes that while using standardised tests of language proficiency
for determining eligibility for special services, teachers also need to realise
that the evaluation of language growth is an on-going assessment of the pupils’
proficiency and involves assessing where pupils are in relation to where they
ought to be. This information is necessary in identifying areas where further
development of pupils’ linguistic ability is needed. Continuous assessment
should therefore be part of the instructional delivery system since it helps
teachers to know precisely whether pupils possess the language skills they need
for academic performance by tying language to its purpose, which is content
learning. Such assessment is also authentic in that it measures the pupil’s
linguistic proficiency in the real contexts in which language use occurs –
which is learning of academic subject matter – and has content validity since
it assesses the broad range of language skills needed in the classroom.
Furthermore, the results indicated that teachers at the two institutions
mostly target low-level tasks in their assessment. Both the classroom
observations and the questionnaire survey revealed that most exercises and
tests required the pupils to recall what they had been learnt. In 79 percent of
the sessions, class exercises and tests involved low-level cognitive processes.
Similarly, all the teachers in the survey indicated that they mostly used
recall questions when assessing pupils. In Bloom’s (1956) taxonomy of learning tasks,
recall falls in the low-level tasks category. The results suggest that the
majority of lessons offered by these teachers involve low-level cognitive
processes which, on their own, fail to mould successful learners. In a recall
exercise, pupils are only required to go into their rote memory and reproduce
what is there. They do not employ their high-level skills such as analyzing,
evaluating and creating.
This study suggests that teachers at both ZPS and CDPS
confine themselves to standardised techniques when assessing the academic
performance of their pupils. Results from the classroom observations and the two questionnaires
indicated a preference of written exercises, oral tests and end-of-semester
examinations over continuous assessment methods. Research, on the other hand,
postulates that academic assessment has to be an on-going process that
objectively informs both the teacher and pupils about the academic language
achievement of the pupils so as to guide subsequent instruction and learning. While
assessment techniques at ZPS and CDPS are repetitive and unvaried, teachers need to
employ multiple assessment tools in bilingual education in order to measure
different aspects of linguistic competence and obtain a clear picture of their
pupils’ performance.
Furthermore, the study reveals that teachers at the two institutions
target low-level cognitive tasks in their assessment by requiring pupils only to recall what they had
learnt. Research, however, points the other way. In addition to tools that
assess factual and procedural knowledge, teachers need to present learners with
problems that involve transferring what has been learned to new situations and
application of related underlying concepts and principles.
Borg, W.R. 1981. Applying Educational Research: A Practical Guide for Teachers. New York: Longman, Inc.
Bloom, B. S. ed. 1956. Taxonomy of Educational Objectives, the classification of educational goals – Handbook I: Cognitive Domain. New York: McKay.
Cloud, N., Genesee, F., and Hamayan, E. 2000. Dual Language Instruction: A Handbook for Enriched Education. London: Heinle.
Creswell, J.W. 2002. Planning, conducting and evaluating quantitative and qualitative research. Upper Saddle River: Pearson Education, Inc.
Cummins, J. 1986. Empowering minority students: A framework for intervention. Harvard Educational Review 56 no. 1: 18–36.
Estacion, A., McMahon, T., Quint, J., Melamud, B., and Stephens, L. 2004. Conducting Classroom Observations in First Things First Schools. Working Paper. Kansas City: IRRE.
Glaser, B., and Strauss, A. 1967. A Discovery of Grounded Theory. New York: Aldine.
Hargett, G.R. 1998. Assessment in ESL and bilingual education: A hot topics paper. Oregon: NWREL Comprehensive Center.
Hudelson, S. 1994. Literacy development of second language children. In Educating Second Language Children: The Whole Child, the Whole Curriculum, the Whole Community. ed. F. Genesee, 129–158. Cambridge: Cambridge University Press.
Kayambazinthu, E. 1998. The language planning situation in Malawi. Journal of Multilingual and Multicultural Development, 19 no. 5: 369-439.
McCracken, J. 1968. Aspects of Central African History. London: Heinemann.
Met, M. 1994. Teaching content through a second language. In Educating Second Language Children: The Whole Child, the Whole Curriculum, the Whole Community. ed. F. Genesee, 159–182. Cambridge: Cambridge University Press.