[2] We should note here that we are not suggesting that there is a single method which is "best" for all students. But some methods do produce better average results: see, for example, Bloom, BS, "The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring (1984) Educational Researcher 4-16"

[3] Kulik, Kulik and Cohen, "A Meta-Analysis of OUtcome Studies of Keller's Personalised System of Instruction" (1979) 34 American Psychologist 307.

[4] At least the majority of students can learn. There will be exceptional circumstances which may prevent a particular student from learning particular material.

[5] "Mastery learning" is the term used by Bloom, BS Human Characteristics and School Learning, New York, McGraw-Hill, 1976.

⁶

[7] For a discussion of criterion-referenced and norm-referenced examining procedures, see Heywood, J Assessment in Higher Education, 2nd Edition, Wiley, New York, 1989; see also Ebel RL and Frisbie, DA Essentials of Educational Measurement, 4th Edition, Prentice-Hall, Englewood Cliffs, 1986

[8] Clark, CR, Guskey, TR and Benninga JS, "The effectiveness of mastery learning strategies in undergraduate education courses" (1983) 76 Journal of Educational Research 210.

[9] The tests provide the students with a means of assessing their own understanding of the material. Mastery learning is not the only way of providing students with self-assessment skills: see Rawson, SL and Tyree, AL "Self and Peer Assessment" (1989) 1 Legal Education Review 135

[10] For a thorough introduction to the Keller Plan, see Keller, FS, and Sherman, JG The Keller Plan Handbook Benjamin, 1974.

[11] See Keller, FS, "Good-Bye Teacher..." (1968) Journal of Applied Behaviour Analysis 179-189, 83.

[12] Keller, FS, "Good-Bye Teacher..." (1968) Journal of Applied Behaviour Analysis 179-189, 83.

[13] For some typical examples, see Allan, PS, "Developing a Remedial Keller-Plan Course" (1978)3 Higher Education 213; Schwartz, PL, "A controlled trial of Teaching Clinical Biochemistry by a Keller Plan (1980) 55 Journal of Medical Education 1013; Ferguson, W, "Personalized Instruction in Business Education" (1981) 50 Journal of Experimental Education 9

[14] See below for a full discussion of the role of proctors.

[15] Keller and Sherman, op. cit. p. 78 recommend four separate tests for each unit. We were able to reduce this load somewhat using the computer examining system described below.

[16] Watson, J "Final Examinations and Long Term Retention" (1986) 17 Journal of Research in Mathematics Education 60-68.

[17] For example, Allen, P S op. cit., self instruction was emphasised, in Schwartz, P L op cit. from a class of 196 medical students 30 were randomly chosen in a biochemistry course to study by the Keller Plan. Schwartz concentrated on the final examination results of thee two groups in his paper.

[18] Kulik, Kulik and Cohen, "A Meta-Analysis of Outcome Studies of Keller's Personalized System of Instruction" (1979) 34 American Psychologist 307

[19] Kulik, Kulik and Cohen, ibid. 311, 317

[20] Kulik, Kulik and Cohen, ibid., 313, 317

[21] Kulik, Kulik and Cohen, ibid., 317 - 318

[22] See Morgan J, "The Socratic Method: Silencing Cooperation" (1989) 1 Legal Education Review 151 and the articles cited there.

[23] It may seem strange that there is no intellectual property component in such a course. It is strange, but is explained by a desire to avoid "overlap" with longer-established courses at the University of Sydney.

[24] The tests are supervised in 1991. On balance, we would recommend supervised tests.

[25] Actually a different but equivalent test. The tests are "equivalent" in the sense that similar questions are used and the same objectives are tested. See the discussion on examination by computer below.

[26] In the 1991 course, this has been reduced to 4 marks per module. The reduction was made to make the course conform more closely to Faculty guidelines: see below.

[27] The LES file which implements this question is reproduced as Appendix A

[28] The DataLex Project is a joint research project managed by Alan Tyree of the University of Sydney, Graham Greenleaf of the University of New South Wales and Andrew Mowbray of the University of Technology, Sydney. The major activities of the Project have been in legal artificial intelligence and legal information retrieval.

[29] And, it should be noted, these are not problems which are unique to the computer examination system. By using human proctors, however, the problems can be caught at marking time, and so may be less serious. In traditional examinations, the problem may never be raised if the student does not have the right to see the results of the test and to challenge the marking.

[30] These ambiguities and deficiencies are inevitable in any original test. One of the advantages of computer testing is that the question responses can be analysed and the questions improved.

[31] We expect to have preliminary results of the project by the time that this paper appears in print. The SAGES system uses a combination of parsing algorithms and statistical methods to classify an answer.

[32] At least the traditional plan where written material is the primary source of teacher-student communication. It may be possible to build a course on Keller principles which would emphasise the development of oral skills.

[33] All real-life Keller Plan courses compromise the self-pacing principle to some extent since it is invariably necessary to fit in with an existing university course structure.

[34] See Keller, F S, "Testimony of an Educational Reformer" (1985) Engineering Education 144-148

[35] Unfortunately, the evidence is that codes of conduct have little effect on cheating except in those institutions where there is long tradition of the codes: see Ebel, op. cit.

[36] except in the module tests!

[37] Without notable success; see below.

[38] Peter S. Allen, "Developing a Remedial Keller-Plan Course" (1978) 3 Studies in Higher Education pp.213-220.

[39] See below for a discussion of alternatives to paid proctors.

[40] The computer lab at the Law School has two Honeywell Superteam computers. These are connected to 28 IBM-compatible machines each with their own 30mb hard disk. The lab is used for teaching legal information retrieval using the DataLex AIRS software, for student word processing and a variety of other projects. In 1991, the Honeywell has been replaced by a Sun Sparc II.

[41] We are developing a PC based examination system which will hold questions in an encrypted form. Contact the authors for further information.

[42] See Keller and Sherman, op. cit.

[43] Although we were able to introduce self and peer assessment of written essays: see Rawson, SL and Tyree, AL "Self and Peer Assessment" (1989) 1 Legal Education Review 135

[44] Even when the "innovation" is more than 20 years old; if it has not been used in law, then it is an "innovation"!

[45] Keller and Sherman, op. cit. p 75

[46] The marking scheme is Fail,Pass, Credit, Distinction and High Distinction. Current Faculty guidelines call for 40% C, 20%D and 5% HD, the percentages being of the non-failing students. There is some minor latitude for variation from these figures.

[47] Munro, D "Quality distance education = computer based feedback and electronic mail" in Barrett, J and Hadberg, J (eds), Using Computers Intelligently in Tertiary Education, proceedings of the ASCILITE Converence 1987

[48] The 1991 version of Technology Law is still the most popular option in the Faculty even though we have indicated that marks will be scaled ruthlessly to conform to Faculty guidelines.

[49] he justification for the marks is not merely our feeling that they were fair. The result is consonant with educational research: Bloom, op.cit.