Changing study habits by changing assessment methods - is it possible?

- a study in engineering education

Work-in-progress report

Anders Berglund

Department of Information Technology

Computer Systems

Uppsala University

e-mail: Anders.Berglund@docs.uu.se

In this study we are investigating if students' learning and study habits can be improved by a change in the forms of assessment. The study has been performed on engineering students in Computer Science and Technology at Uppsala University, Sweden. It is still to early to draw any firm conclusions, but available data points towards good learning. Some important factors that influence students' learning, such as deadlines for assignments and dates for exams, and the choice of study partners have been studied.

1. Assessment as a tool to influence study habits

Assessment is an important means to influence what, how, and when students study. As lecturers we cannot control how students study, but we can, through our choice of assessment materials and methods, signal priorities and direct attention. Used correctly, assessment methods can be a powerful tool to influence the students' study habits. The study reported here is investigating how changes in assessment methods influence the students' study habits¹, and thereby their learning (see Figure 1).

Assessment methods --> Study habits --> Learning

Fig 1. The relationships of influences studied

The need for a review and renewal of assessment methods is well documented. For example, Jacobsson (1995) points out that students in an Engineering Physics programme tend to study intensively before an examination. Trowald (1996) discusses several unwanted effects of today's assessment methods. Hult (1998) argues that we need a debate about assessment in Engineering education. Booth (1992) shows that the correlation between the students' actual understanding of fundamental concepts and the grades they receive is not as strong as we, as

lecturers, would like.

2. The study with alternative assessment methods - Expresso

In this study, we have been observing the effects of changing assessment methods in three courses during a given study period. Students in the system engineering specialization of the Engineering Physics programme at Uppsala University, Sweden, took three courses during a ten-week study period in the students' third year. The courses were: Algorithms and Data Structures, Computer Architecture and Signals and Systems. Approximately sixty students attended the courses.

2.1. The assessment methods used

Different assessment methods were used in the three courses for several reasons:

We wanted to:

• adapt the assessment methods to the different characters of the different courses,

• encourage different ways to study among the students,

• increase our experience of different assessment methods, as well as letting the students get familiar with other methods than the classic final exam.

The changes were, in brief:

• in Algorithms and Data Structures, the final exam was replaced by weekly assignments,

• in Computer Architecture, the final exam was replaced in part by public seminars with opposition and defense on programming projects,

• in Signals and Systems and the remaining part of Computer Architecture,

• the final exam had reformulated questions designed to reflect the higher level in Bloom's taxonomy (Bloom, 1956).

The assessment methods chosen were based on a pilot study (Berglund, 1996). For a longer discussion on the details of the assessment methods used, see Berglund (1998) and Berglund (1999).

Several changes in the teaching were made as a consequence of the changes in assessment, a decrease in number of lectures, more individual help in labs, students proposing solutions to assignments on the blackboard, and lectures given more in dialogue with the students.

2.2. The evaluation of the assessment methods

Since the evaluation of students performance sets the grade of the students, a comparasion of grades would only give a weak indication of how the alternative assessment methods could have influenced learning. Other factors, like for example "how difficult the exam is" would probably have an influence that is at least as important on the results.

Instead, we interviewed ten selected students at intervals during the study period about their understanding of basic concepts of the subjects concerned, their experience of learning during these courses compared to their previous study experiences, their study habits, and other factors that influence their learning.

2.3. Observations until today

Currently we are analysing these interviews. Although it is yet too early to draw any firm conclusions, there are some preliminary observations that can be made.

2.4. Observations concerning study habits

• The assessment does appear to be a powerful tool to influence students' learning. The assessment method, the "expected difficulties", and the dates for and order of exams are important factors when the students plan their studying.

• Students' priorities between different assignments are to a very large extent influenced by deadlines. Most students tend to work towards the closest deadline, even though a later

• deadline might be more important. The choice of priorities between assignments is thus not only ruled by the "importance" of a certain assignment.

• Deadlines are important not only within a course, but also over the courses. Deadlines in one course seem to influence the learning outcome in another course. This is a factor worth considering when planning courses.

Other factors that influece study habits include the priorities of the student's study partner, their attitudes towards the teachers, and earlier experiences.

2.5. Observations concerning outcome of learning

• The students showed clear signs of a development of concepts during the courses. This is valid for the courses with a final written exam, as well as the course where the exam was replaced by weekly assignments.

• Some factors points towards students having "learned more" during these courses, than they do during courses with "normal assessment". This is a rather vague indication which needs to be studied further, both during the analysis of the interview data and in subsequent studies. It seems clear that most of the students have spent more time studying during these courses, which might explain this effect. It is still to early to say if this possible increased work-load is a consequence of the new assessment methods, the fact that we made a change, or the fact the we were studying the students' learning.

3. Conclusions

It is clear that assessment methods can be changed, and we can already at this stage see that some changes are advantages. A final exam is not a requirement to encourage the students to learn. Assessment is a strong tool for influencing students' learning, and maybe the only one we, as lecturers, can control. There are however other factors, such as deadlines and study partners, that play important roles.

REFERENCES:

A. Berglund, P.O. Foyer, V. Karlsson, A. Svärdström: Full Scale Study with New Approaches to Examining the Students on the Engineering Physics Programme in Uppsala. SEFI conference Educating the Engineer for Lifelong Learning in Vienna, October 96.

A. Berglund, M. Daniels, M. Hedenborg, A. Tengstrand: Assessment to Increase Students' Creativity: Two case studies, European Journal of Engineering Education, Vol 23, No 1, 1998

A. Berglund: EXPRESSO - Examinationsfomer som ett sätt att förändra studievanor - en studie på teknisk fysikprogrammet (in Swedish), in press, NyIng, Linköpings Universitet, 1999

B. Bloom: Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain, David McKey, 1956.

S. Booth: Learning to program: A phenomenographic perspective, Acta Universitatis Gothoburgensis, Gothenburg, 1992.

H. Hult: Examination inom ingenjörs- och civilingenjörsutbildningarna, (in Swedish), Institutionen för systemteknik, Linköpings Tekniska Högskola, 1998.

P. Jacobsson(ed.): Quality Review F. Review of the Swedish and Finnish M Sc Programmes in Engineering Physics, Kungliga Tekniska Högskolan, 1995.

N. Trowald: Uppfattningar om examination - en intervjustudie av högskolelärare, (in Swedish), Högskoleverket, Sweden, 1997.

Anders Berglund is a lecturer in Computer Science at Department of Information Technology, Uppsala University in Sweden. His main interest of research is Computer science education, and there the learning of the individual student. His PhD project concerns knowledge creation in a distributed international environment in aproject where students in Helsinki, Finland, shall teach Java to students in Marseille, France, over Internet.