[comp.edu] Setting programming exams in Comp. Sci. courses

chris@cs.uwa.oz.au (chris mcdonald) (06/02/91)

I am interested to learn if any other Computer Science departments
set "programming exams" as part of their assessment of students,
and if so to learn how you run them and how successful they are.

Using just my 2nd year undergraduate 25 lecture "Compiler Writing" course
as the example, we currently set one or two projects to be completed over
about 6-8 weeks. These practical exercises are worth 40% of the assessment
for the unit, the other 60% coming from a 1.5hr exam.
The projects typically take an average student 20-50 hours to complete
(some variation!)

These projects, though successful in highlighting many sections of the
course, introduce other problems :

*   The 240 students doing the course using 38 terminals presents obvious
    problems with reduced access and crowding. Problems such people
    occupying a terminal for 12+ hrs or writing terminal "locking" programs
    are common.

*   There is little guarantee that a student's submission is entirely their
    own work - it may be the result of a group effort or include some
    plagiarized material.

*   There is insufficient motivation for the students to work consistently
    on their preceding laboratory sheets or during the project itself
    (we tell them it will be crowded but they still work in the last week).

To solve these problems and to get a more accurate estimate a student's
ability to complete a problem in a reasonable time, I have proposed using
a programming exam next semester (commencing July 15).
The emphasis here is that it is a *practical* exam rather than a theoretical
one (which will be taken in November and be worth 60%).

The logistics will be that each of the (240) students will be required to
attend a 4 hour laboratory session in the 2nd last week of semester.
They will nominate a time (as they do for labs) and provision will be made
for some by having two of three evening (6pm-10pm) sessions.

The practical component of the course will include eight or nine week length
laboratory sheets which the students should do to become familiar with the
work, understanding and making additions to a (900 C lines) compiler for a
Pascal-like language.

The practical exam will consist of eight to ten different questions (each
exam session only does one) involving further modifications to the compiler,
justifying their additions/changes through documentation and, perhaps above
all, doing this in the time limit. Some exam questions may in fact be
re-worked laboratory sheets and knowledge of this will encourage everyone to
do all the sheets.

Each student sitting in an exam session will be placed in a restricted
"programming environment", in their own isolated directory structure, with
access to only a dozen programs like the editor, C-compiler, copy, list, etc.
Each student will be given copies of the files they need.
It will not be possible for students to copy files to each other via the
directory structure or e-mail.
Each exam will be supervised though not demonstrated.

The challenge will be for me (together with my tutor) to design a number of
questions of equivalent difficulty, each which can be reasonably completed
(the software that is) in three hours, have a sample or preferred solution
and a well defined marking scheme.

--------------------------------------------------------------------------------

I welcome your thoughts on this proposal, both reasons as to why you don't
think it will work and suggestions as to how it could be improved.

--------------------------------------------------------------------------------
Chris McDonald.            _--_|\
                          /      \
                          X_.--._/
                                v
Department of Computer Science,   ACSnet: chris@budgie.cs.uwa.oz.au
University of Western Australia,  ARPA:   chris%budgie.cs.uwa.oz.au@uunet.uu.net
Mounts Bay Road,                  FTP:    bison.cs.uwa.oz.au,  130.95.1.23
Crawley, Western Australia, 6009. SCUD:   (31.97 +/-10% S, 115.81 +/-10% E)
PHONE:       +61 9 380 2533       FAX:    +61 9 380 1089

dbreiden@mentor.cc.purdue.edu (Danny Breidenbach) (06/02/91)

In article <chris.675829285@budgie> chris@cs.uwa.oz.au (chris mcdonald) writes:

>These projects, though successful in highlighting many sections of the
>course, introduce other problems :

>*   There is little guarantee that a student's submission is entirely their
>    own work - it may be the result of a group effort or include some
>    plagiarized material.

Why not assign the projects to groups?  A formally assigned group of 3, 4 ,or
5 students can learn tremendous amounts by working together.  It could
easily cut down on the crowding problem you mentioned earlier.

In a calculus class this past year, we had every assignment done by groups.
Each group turned in one assignment.  We gave the group the power to complain
if one member did not do any work, but we viewed this as a problem for the
group to deal with (with our help) whenever possible.  We also gave an exam
in which the group did the exam.

The logistics of such a venture are scary, and it takes a great leap of faith
to attempt such a plan; but I assure you, our results were astounding!

I cannot put all the credit on our use of collaborative learning; we used
non-traditional techniques and microcomputers in our teaching.  These
factors weighed heavily in our results.  But we definitely saw great benefits
in the group work in our course.

I think your exam session idea sounds wonderful.  But don't discount the
utitlity of having students groups be a POSITIVE aspect of learning.  I find
it *almost* amusing to see the words "group effort" used in such a negative\
manner as you did.  Think about it.

--Danny

hb136@leah.albany.edu (Herb Brown) (06/03/91)

In article <13086@mentor.cc.purdue.edu> dbreiden@mentor.cc.purdue.edu (Danny Breidenbach) writes:
>
>In a calculus class this past year, we had every assignment done by groups.
>Each group turned in one assignment.  We gave the group the power to complain
>if one member did not do any work, but we viewed this as a problem for the
>group to deal with (with our help) whenever possible.  We also gave an exam
>in which the group did the exam.
>
>The logistics of such a venture are scary, and it takes a great leap of faith
>to attempt such a plan; but I assure you, our results were astounding!
>
>--Danny


I totally agree. In our (Mathematics) department, we offered a full range of
courses from Calculus for non-science/math majors to senior analysis and
algebra in a specially designed Computer Classroom where the students learned
mathematics by interacting with their computer, with the instructor at the 
blackboard, with the instructor's computer and with fellow students. They were
encouraged to colaborate throughout the semester, including the writing of
examinations. Some chose to work alone, some worked in groups of 2 or 3. The
weaker students, who would normally drop the course, were helped by the
stronger students to learn enough to pass. The stronger students advanced
beyond the level of the material through exploration. This was the first
year of our Computer Classroom and the results seemed positive.

Herb

 
-- 
----------------------------------------------------------------------------
Herb Brown  Math Dept  The Univ at Albany  Albany, NY 12222  (518) 442-4640
hibrown@leah.albany.edu or hibrown@cs.albany.edu or hibrown@bst.albany.edu
----------------------------------------------------------------------------

cosper@seq.uncwil.edu (Kit Cosper) (06/04/91)

As  a Comp Sci student I find "online exams" as they call them here to be
quite challenging, but above all, fair.  You must take into consideration
that there are only ~20 students per section and each section has its own
time to take the  exam.  Most problems usually consist of a practical 
application of new material, although on occasion we are given  "dirty"
code that must be debugged.  In this environment the Prof is able to 
keep track of things  using the UNIX 'w' command, that way he can detect
any attempts at dishonesty.  Given the complexity of the course you 
mentioned and the number of students you are up against a much greater 
beast.  

For language courses, from a student's perspective, I think they are the 
BEST way to go.

Kit

-- 

Kit Cosper
cosper@seq.uncwil.edu
Donations  of witty or thoughtful comments appreciated

cslaurie@cybaswan.UUCP (Laurie Moseley ) (06/16/91)

I do not understand the fuss about group work. How on earth do you
stop it ? Not only am I not going to be in our terminal rooms at
3 a.m. for several days before a deadline, we actually expect students
to help one another. I would guess that a high proportion of the real
teaching (the sort that produces the Aha! phenomenon) is done by
students informally. If you think that cheating is taking place,
include a programming section in the final examination and make the
passing of that section (as well as the whole exam) compulsory.

My experience is that few students narrowly pass such a section.
They either pass very comfortably, or they fail abysmally. The marks
on such a section are markedly bimodal at either end of the distribution.

Our philosophy is "Make all assignments compulsory i.e. you fail if
you do fail to hand in even one of them, grade them to give feedback
to the students, but do not take these grades into account in the
final assessment". If they have learned anything in an introductory
course, they will be able to build linked list, binary trees, hash
tables and the like in very short order. We find that the conceptual
structures for doing these common operations are almost like form
letters in the students' heads.

Of course, the ones who cheated on the compulsory (but not counted)
assignments will be completely lost in such an examination. The
others will actually enjoy it, since we all enjoy doing things which
we do well and which we find simple.

Laurie