Introduction

Minutes

Members

Oral Presentation Assessment Committee Minutes 2001-2002

Oral Presentation Assessment Steering Team, Meeting 1, August 27, 2001, H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. Santa
Vicca

Visitors: K. Merkel and M. Enciso

We discussed the need (?) for a sixth committee member due to the
retirement of Ginny Brouch. We decided that the five members were a
sufficient group for now.

We chose a new meeting day and time. The committee will meet for one
hour every other Wednesday, 8:30-9:30. Dates will be:
Sept 12
Sept 26
Oct 8
Oct 24
Nov 7
Nov 21
Dec 5

We looked at a list of potential items to discuss this semester (in no
particular order):
- Revisit Outcomes Statement
- Invite instructors to our assessment sessions?
- Send out Competencies Summary sheet to all instructors?
- Workshops?
- Visit NFO?
- How can we make Oral Pres Assmt part of the campus culture?
- How will we collect samples?
- Should we visit department meetings to talk with faculty?
- Will we need to collect information on student presenters?
- Who will assess? When? Will they need training?
- Do we need to revisit the language on our handouts?

We decided that none of the above could be rationally discussed until we
had assessed the samples (roughly 29 presentations) we collected last
semester. We will therefore devote our next meeting (Sept 12) to
assessing at least 5 of the samples, one from each tape.That assessment
session should help chart where to go from here.

Oral Presentation Assessment Steering Committee; Wed., Sept. 12, 2001; 8:30-9:30 a.m.; H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

• We reviewed the update report written by O'Brien for Brent to include
in his report to NCA. A few changes were made and will be sent on to
Brent.

• Four sample oral presentations were cued and ready to be assessed.
After viewing only one, we spent the rest of the morning discussing our
assessment tool:
- We realized that we were not necessarily comfortable with the
24-point scale. We questioned whether or not 3 categories only
(Unsatisfactory, Satisfactory, Excellent) per competency was sufficient.

- We debated the merits of using some other scales, including a
40-point (8 competencies x 5 category choices) and 56-point (8
competencies x 7 choices) scale. We toyed with rescoring the
presentation viewed using the 40-point scale and agreed that we
preferred the flexibility it gave us when scoring.
- We then realized that we had yet to decide on what was a
satisfactory score and where those numbers would eventually play out on
our outcomes statement (that has yet to have a measurement minimum
declared).
- The morning almost came full circle when we again debated the
usefulness of the 24-point scale.

• We ended the meeting realizing that we were not yet ready to assess
our samples. We must first decide on how to use our tool.

• For the next meeting, a 40-point scale tool will also be available. We
will experiment with it, and then work towards choosing between it and
the original 24-point scale. We will also continue our discussions as to
our outcomes statement.

Next meeting: Wed., Sept. 26, 2001; 8:30 a.m.; H221

Oral Presentation Assessment Steering Committee; Wed., Sept. 26, 2001; 8:30-9:30 a.m.; H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

We began the meeting by revisiting our assessment tool, The Competent Speaker. We looked at a 40-point version and again discussed the 24-point version. Our problem was that we just couldn't decide on which one would be better to use to determine whether or not a sample oral presentation "passed," was "acceptable," was "satisfactory," etc.

After several more minutes of discussion on the various merits of each of these versions (as well as other possible versions), we finally agreed to simplify our scoring columns to better meet the needs of assessment. We realized that the degree of their success in meeting the competencies on the tool was not as important as whether or not the student met the various competencies. We realized that simple "yes" or "no" columns next to each competency was more relevant.

We agreed that a minimum of five "yes" checks would be considered a "passing/acceptable/satisfactory" score. To provide data to the institutional researcher, we agreed to have a "no" check equal zero and a "yes" check equal one. A student at five or above would satisfactorily meet the compentencies, whereas a student presentation scoring four or less would not.

We then watched another oral presentation sample and assessed. Scores, for example, were as follows:
Rater 1 = 6 yes, 2 no
Rater 2 = 8 yes, 0 no
Rater 3 = 5 yes, 3 no
Rater 4 = 6 yes, 2 no
Rater 5 = 6 yes, 2 no

The average of the "yes" points was 6.2 (31 divided by 5 raters). This student presentation would be acceptable.

We felt very good with our decision and realized that we needed this time and discussion in order to reach it.

We will rate more oral presentations at our next meeting, October 10.

Meanwhile, we will begin work on collecting more samples from around campus.

Oral Presentation Assessment Steering Committee; Wed., October 10, 2001; 8:30-9:30 a.m.; H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

1) We first tackled the updating of the outcomes statement. The one we
had drafted at the beginning of our work last year lacked the
measurement. Additionally, we have learned more about what we are doing,
and we needed to have that reflected in the statement. After much
creative wordsmithing, we have agreed upon the new Oral Presentation
Outcomes Statement:

"Phoenix College students in college level courses will be able to plan
and deliver an oral presentation to a target audience at a satisfactory
level*"

* Satisfactory level is defined as scoring a minimum average of 5 on
the 8-point adopted instrument.

2) We then agreed to begin planning for sample collection for this
semester. We will write a letter to all residential and adjunct faculty
and attach said letter to a copy of our student oral presentation
recommendations handout. We will try to get these out by next week and
see if we can get some faculty on board. Meanwhile, O'Brien spoke to the
NFO group and found two definite faculty partners in Roland Walker and
Ofelia Cañez.

3) Finally, we had enough time to rate one more speech from the Art of
Web Design class, Spring 2001.

Next meeting, October 24, 8:30.

Oral Presentation Assessment Steering Committee; Wed., October 24, 2001; 8:30-9:30 a.m.; H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

We spent the hour rating oral presentations and becoming yet more
comfortable with our rating tool. We were able to rate four
presentations.

Next meeting, Wed., Nov. 7, 2001

Oral Presentation Assessment Steering Committee; Wed., November 7, 2001; 8:30-9:30 a.m.; H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts
Excused: E. SantaVicca

We agreed on the changes made to the technician form that will now
collect students' names as per the directions of the campus Assessment
Steering Committee.

We rated 5 more oral presentations.

Next meeting: Wed., November 21, 8:30, H221

Oral Presentation Assessment Steering Committee; Wed., November 21, 2001; 8:30-9:30 a.m.; H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

We rated five more presentations today.

Next meeting: Wed., December 5, 2001

Oral Presentation Assessment Steering Committee; Wed., December 5, 2001; 8:30 a.m., H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

We rated 5 more presentations and set meeting times for next Spring.
Those meeting dates will be:
Jan 16
Jan 30
Feb 13
Feb 27
March 20
April 3
April 17
May 1

Oral Presentation Assessment Steering Committee; Wed., January 16 ,2002; 8:30 a.m., H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

- We assessed five more oral presentations.

Oral Presentation Assessment Steering Committee; Wed., January 30, 2002; 8:30 a.m., H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts
Member excused: E. SantaVicca

- We planned out the items to include during our presentation for the
February 8 Assessment Day.

- We assessed four more oral presentations.

Oral Presentation Assessment Steering Committee; Wed., February 13, 2002; 8:30 a.m., H221.

Members present: G. Burgess, L. Garcia, L. O'Brien, K. Roberts, E. SantaVicca

Today's topic was "Where do we go from here?". Topics discussed
included:
- What does NCA want to see? Are they just concerned whether or not the
students are meeting the competencies (yes or no) or are they concerned
about how well the students are meeting the competencies?

- We will ask Jeremy to determine of one or more of the competencies is
not being met (across the board). We can then see an overall deficiency
in one of the competencies.

- Is all this assessment for NCA or is it for some other purpose?

- Is our goal to have students meeting our competencies when they leave
PC, graduate and/or transfer?

- Do we need to separate students who have taken COM100, 225 or 259 from
those that haven't? Will we still see the same 'two-thirds meeting
compentencies', 'one-third not meeting competencies' ratio?

- Do we need annual data or can we have a data sample that's done over a
3-year period?

- Should we get a full-time instructor "vs." adjunct instructor
distribution?

- Perhaps we could do a round-table discussion of instructors who tend
to elicit successful oral presentation behaviors in their students.
Compile responses and distribute to full-time and adjunct instructors.
This will help answer the NCA need to "communicate our findings."

We then rated two more oral presentations.

Oral Presentation Assessment Steering Committee; Wed., February 27 , 2002; 8:30 a.m., H221.

Member present: Margaret Souders

Members excused: Ken Roberts, Gerry Burgess, Linda Garcia

The chair spent the whole meeting time welcoming, explaining and
training the new committee member. She feels ready to go at our next
group meeting.

Oral Presentation Assessment Steering Committee; Wed., April 3, 2002; 8:30 a.m., H221.

Members present: Gerry Burgess, Linda Garcia, Liz O'Brien, Margaret Souders

Member excused: Ken Roberts

We rated four more presentations today.

Next meeting: April 17