Introduction

Minutes

Members

Oral Presentation Assessment Committee Minutes 2000 - 2001

Wed., Dec. 20, 2000. Oral Communication Steering Committee.

Attending: K. Roberts, G. Brouch, E. Santavicca, Q. Chavez, G. Burgess,
L. Garcia, L. O'Brien. (I. Fawcett excused).

1) We continued shaping the outcomes statement, which at this point
stands as such:
"Students will be able to communicate an intended message to a
target audience
using appropriate verbal and nonverbal skills at ______ level."

We are satisfied at this point that the statement contains three of the
four criteria needed for an outcomes statement - the who, the time frame
and the action. The blank space in the statement will at some point
contain the fourth needed criterion - the measurement. We felt that we
needed to begin working with the rubric before deciding on that
measurement.

2) Ed has offered to contact Michael Moore, the lead designer of the
rubric, at Moorehead State. He will also attempt to find other
institutions who have used this rubric in order to see what their
outcomes statements looked like.

3) The manual on "how to use" the rubric has been ordered. It should be
here before we meet again.

4) We set up a meeting schedule for the Spring semester. We will meet on
the first and third Tuesdays at 2:30 in T101.
January 23
February 6 and 20
March 6 and 20
April 3 and 17
May 1

5) We will begin working with the rubric by watching student videos at
our next meeting on January 23.

Thanks again to all of you. I'm looking forward to working with you next
semester.

Liz

Oral Com Assmt Team; Tues., Jan 23, 2001; T101

Attending: K. Roberts; G. Brouch; Q. Chavez; I. Fawcett; G. Burgess; L.
Garcia; E. Santa Vicca; L. O'Brien

(Kurt Chambers, the final member of the committee, wishes to stay
involved despite his teaching schedule that conflicts with our standing
Tuesday meeting times.)

We reviewed the Outcomes Statement one more time, agreeing to keep it as
is for now. We feel we need more time before we can fill in our final
blank of the measurement criteria.

We reviewed the manual that arrived from the National Communication
Association office. We agreed to ask LTD to put the whole thing on a web
site and agreed on the hard copy pages that we would need for our work
sessions.

We agreed to order the video tape available for training purposes.
Meanwhile, Liz will identify some preliminary training videos from a
random supply in her office.

We wanted to see what would happen if we did a "pre-test" of the rubric.
After a short discussion to define some terms (e.g. disfluencies) and
clarify some concepts (e.g. topic vs. thesis) we watched a student
informative speech video, "assessed" it, and discussed it. The team
seemed very encouraged that we all "assessed" the speaker within a few
points of each other on the rubric despite minimal to no training.

Next meeting: Tuesday, Feb. 6, 2:30, T101

Oral Presentations Steering Team, February 6, 2001, T101

Attending: G. Brouch, G. Burgess, Q. Chavez, I. Fawcett, L. Garcia, L. O'Brien, E. Santa Vicca

Excused: K. Chambers, K. Roberts

Team: please let me know if I left anything out or misrepresented something.

• We reviewed some of the Writing Team's handouts and questioned if we eventually needed to have some similar documents.
• We realized we'll need to prepare a handout on:
• Guidelines for faculty collecting samples. So far, we have chosen the
following guidelines:
- 5-10 minute speech
- in-class presentation
- informative, persuasive or special occasion speech (though we
assume we'll get mostly informative speeches).
• Form for faculty to attach to samples that will describe the specifics of the
oral presentation assignment, the audience present and the occasion.
• Directions to the technician collecting samples.
• Handout for the student to provide some brief demographic data.

• O'Brien provided update on the existence(?) of this supposed training video mentioned in the manual for the rubric. A phone call and e-mail have been sent to the main designer of the rubric, but he has failed to return the message at this point. We'll keep waiting. Meanwhile, we will try to locate samples of unsatisfactory, satisfactory and excellent speeches for training purposes.

• We watched another sample student speech and again applied the rubric. After tossing out the high and low scores, all others were within 3 points of each other (out of 24 possible). We are again encouraged that the rubric will work, but we realized how important proper training will be. It was easier to apply the rubric to the stronger speech we watched two weeks ago. This one was a weaker speech and was not as easy to assess.

• We began to talk about our training day on March 27. We will need to have our training samples decided upon by that day. We will continue to work on these details.

• We decided it wasn't too early to send out a campus-wide message seeing who will be offering oral presentation assignments this semester. We will do this in attempts to find samples for the campus-wide assessment day.

• The team supported Marian's offer to hire an 'Assessment-Priority Media Technician' and to purchase a video camera, tripod and microphone. We realized that the filming and tape quality must be excellent so as not to distract raters. They must be able to focus on the speech and the speaker and not have to wade through a poor quality video.

• We agreed to extend these meetings from one hour to 90 minutes from now on.

Next meeting: Tuesday, February 20, 2:30, T101

Oral Presentation Assessment Steering Team; Tues., February 20, 2001; 2:30 p.m.; Women's Dressing Room, John Paul Theatre.

Attending: G. Brouch, Q. Chavez, L. Garcia, L. O'Brien, K. Roberts, E.
Santa Vicca

Excused: G. Burgess, I. Fawcett, K. Chambers

• We spent most of the day talking about the direction of 'Where do we
go from here?' We feel that we made good progress, but that there are
still some key questions to be answered. The end result of our
discussions was this current plan:

1) Create a written survey to be sent to all faculty (residential for
sure, maybe adjunct) to see if they teach a course where oral
presentations are sometimes or always assigned.

2) From this survey, we will identify a 'menu' of classes (60-75?) where
oral presentations are assigned.

3) These classes will be assigned to one of three groups which would be
rotated on a 3-year basis. Each year would include a group of classes
from varying disciplines, varying levels (000-100-200), etc.

4) Instructors in each group of classes would volunteer to let a camera
technician attend class one or two days during the semester to collect
video samples of student oral presentations.

5) The video would be sent to the trained rating team.

6) This rating team would provide feedback to the instructors (only if
they requested it) and would report the assessment results to the
administration and research specialist.

Questions still to be answered:
What sample size will NCA accept to show assessment is occurring?
How many samples will make a good pilot?

To Do:
Queta - will begin to draft the student speaker bio data questionnaire
Ginny - will begin to draft the campus-wide faculty handout to attach to
any oral presentation assignment.
Liz - draft form to attach to samples, to be filled in by faculty member

Liz - draft directions to technician
Ken - begin to draft faculty survey (this one has since been put on
hold).

Oral Presentation Assessment Steering Team; Tuesday, March 6, 2001; 2:30-4:00 p.m.; John Paul Theatre

Attending: G. Brouch, G. Burgess, Q. Chavez, L. O'Brien, K. Roberts, E.
Santa Vicca

Excused: L. Garcia

- We decided on our approach for the campus training day, March 27.
1) Ken will describe how we chose our rubric.
2) Gerry will present the rubric.
3) Ginny will show how to score the rubric and will lead the scoring
discussion
after we show the first video-taped speech ("Why you should study
ASL").
4) Ed will lead the post-discussion.

- We then looked over the first drafts of our three documents.
Recommendations for changes were made and will be presented at the next
meeting.
1) Oral Presentation Recommendations for Students (attached to this
e-mail)
2) Faculty Form (to help us understand the assignment)
3) Technician Form (directions for obtaining the videos)

Next meeting: Tuesday, March 20, 2:30

Oral Presentation Assessment Steering Team; Tuesday, March 20; 2:30; T101.

Attending: G. Brouch, G. Burgess, Q. Chavez, L. Garcia, L. O'Brien, K.
Roberts.

Excused: E. Santa Vicca

- We finalized the agenda for next week's campus training day. We felt
that there wasn't much more we could do until we make our presentation
and get reactions. We'll then know where to go from here.

- We did OK the re-draft of our documents. A few more changes were made.
Ginny Brouch will be the first to use them when her students give their
presentations on Wednesday, March 21. We'll know if they work or not.

Oral Presentations Steering Committee; Tuesday, April 3, 2001; 2:45
p.m.; T101

Attending: G. Brouch, G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E.
Santa Vicca

- Ginny Brouch reported on the successes/problems of the first video
tape collection event from her class. The microphone wasn't on for the
first 5 so we got 2 oral presentations we can use. It's a start.

- Marsha Graber (Interior Design), Alan Haffa, Kay Hilder and Gerry
Burgess will be offering their classes for this semester's pilot. We're
hoping to have 20 presentations or so by the end of the semester. Each
instructor has a copy of the recommendation sheet to share with
students. We have contacted Mike Poplin to schedule the camera tech for
Graber, Haffa and Hilder. Burgess will decide soon on the date for the
tech to come to his class and Poplin will be contacted.

- Ed and Gerry reported on the successes / concerns from our campus
training day on March 27. The good news was that even with minimal to no
training, the majority of the attendees scored the sample presentation
relatively similarly. They seemed to like the rubric. There were some
concerns with terms and language on the handout and on the rubric. We
are working on that.

- We again reiterated our feeling that the rating team will need to be a
specifically trained team and that this rubric is best not used by
individual teachers with no training. We simply need the instructors to
be willing to allow us to collect samples from their classes. We did
feel that instructors will feel empowered by having the recommendation
sheet to give to their students to better communicate the oral
presentation assignment.

- We began to plan our agenda for Assessment Day on April 20. We will
not be ready to assess that day but will be ready to share the
recommendation handout, inform about our rubric, explain our assessment
plan, and let them watch an oral presentation to better understand the
rubric. We will also help brainstorm on possible oral presentation
assignments teachers could incorporate in to their classes.

- Another good meeting!

Next meeting: April 17, 2:45, T101