What are program learning outcomes (PLOs)?
Program learning outcomes (PLOs) state the knowledge or skills faculty members most value and strive to help their students achieve. They emphasize complex reasoning and the application of knowledge, rather than rote memorization or counting achievements. They are usually expressed in terms of skills or abilities students should demonstrate at the end of the program and should be feasible to measure through direct evidence such as the quality of assignments, exams, theses, dissertations, and presentations. At UC Santa Barbara, all degree programs are required to have PLOs. It is understood that as programs evolve, their PLOs may need to be updated. All new or revised PLOs require Academic Senate approval. If you want to develop or revise PLOs, please see Rethinking your PLOs and Updating your PLOs for more information and contact the A-Team for support.
Who created the PLOs for UC Santa Barbara?
During academic year 2011–12, faculty members from each program in the College of Letters and Science and the College of Creative Studies developed their own set of PLOs for every degree they offer. Each program's faculty members voted to approve each set of PLOs. In the General Education (GE) program, 94 faculty who teach GE courses collaborated to develop GE learning outcomes. Earlier, the College of Engineering had established PLOs to comply with the requirements of their accrediting agency, ABET.
What exactly are we assessing (and what are we not assessing)?
It is important to know that you are not assessing individual instructors, students, or staff, and you are not assessing course-level outcomes. This is not about the performance of individuals or the short-term impact of a course. Rather, you are assessing how well your degree program is meeting its learning goals (PLOs) overall. Because PLOs articulate what knowledge and skills students should have by the end of a program, it is necessary to look at your students' work. However, you are not judging the performance of individual students or the faculty who teach them. You are looking for ways that your program can improve learning opportunities for students.
For example, if you collect student papers in a senior course and use a rubric to assess the demonstration of a particular skill, and you determine students overall poorly demonstrate that skill, then you are asked to think about your program holistically to identify why students are missing this skill. While it is easy to say "That's a weak cohort of students" or "Something must be wrong with how that course is taught", you want to take a step back and think of the preparation students have had prior to the course. You may even want to consider your admissions process or ways that students receive support from your program outside of the classroom. When you think about the issue at the program level, rather than the course or student level, you are asking questions that consider your whole curriculum, program culture, and expectations. Are you expecting them to excel at something in the senior year that students have not had a chance to practice since their first quarter of their freshman year? It may be that a way to address this issue is to modify multiple courses along the curriculum to give students a chance to practice the skill over the four years, or perhaps the material covered in that first quarter of their freshman year can be moved to a junior or senior-level course, instead of modifying the course from which you collected the papers for the assessment. Or your program may decide to put together some online materials or workshops for students. Ultimately, you are assessing your program and you need to approach your assessment strategies holistically.
When should PLOs be assessed?
PLOs can be assessed at any point throughout the curriculum. Because PLOs explain the knowledge or skills students should demonstrate upon completing a degree program, assessing PLOs from student work that is created at or towards the end of a program is necessary. However, if either through your teaching experiences, discussions with colleagues, or your program's assessment activities, you uncover that a learning goal described in one of your PLOs is not being met, you will likely need to collect data from earlier in the program to identify where to place appropriate interventions. You will also need to determine the progression of your PLO along the curriculum, so that you compare students against expectations suited for their stage in the program. For example, if you assess the PLO from student work at the beginning of your program and find that first-year students do not demonstrate the level of understanding you expect from a first-year student, then perhaps students are entering your program lacking sufficient preparation. You may choose to look at your admissions criteria, or you may decide to provide more support in the introductory courses to help students get to an appropriate level that will enable them to achieve the PLO by the end of the program. If you assess the PLO from student work when they transition to upper-division courses and find they perform poorly, then it may be that the lower-division courses need some restructuring or a course or two may be needed to help bridge students between lower and upper-division coursework.
What kinds of evidence of student learning are useful in assessing PLOs?
Direct evidence, in which students demonstrate a capability, skill, or habit of mind, is considered the strongest. Examples include student work such as exams, papers, presentations, performances, research projects, and field work; standardized tests of disciplinary knowledge; observations of students executing a task; pre- and post-program tests. Surveys in which students report on their abilities or skills, graduation rates, and the number of students progressing to advanced degrees are considered indirect evidence, and should be bolstered by direct measures for the results to be most cogent. Please contact the A-Team if you would like support with collecting indirect evidence.
Are course evaluations useful for assessing PLOs?
On this campus, course evaluations capture students’ feelings about the instructor, the syllabus, assignments, required and supplementary materials, and exams, and these evaluations are not correlated with what students have learned in the course. Thus, they provide no direct evidence of student learning. However, feedback from students may be a valuable form of indirect evidence that can help you identify where and why problems exist. Typically, these require more specific questionnaires than the course evaluations. If you are interested in this type of feedback to compliment the findings from the assessment of student work, please contact the A-Team. They can help you develop surveys and conduct focus groups with your students.
How is program-level assessment different from grading?
Grades typically address course-level learning rather than program-level learning. Grades on tests, papers, or presentations typically reflect other learning or behavior not expressed in the PLO being assessed, and so it can be hard from grades to identify if the PLO was achieved. For example, a test may cover a breadth of topics, and so the grade on the test may not adequately measure the learning described in the PLO. Two students may get the same grade on the test but one may have gotten the question(s) related to the PLO correct while the other did not. It may even be that a question that tests for knowledge related to the PLO also tests for knowledge outside the scope of the PLO. Assignments may also consider factors unrelated to the PLO such as class participation, and grading may be inconsistent across faculty or TAs, especially when multiple sections of a course are offered in a quarter or year.
What happens if the need for program improvement is uncovered through the assessment activities?
By design, assessment helps you identify areas in your curriculum that you can improve to better meet your PLOs. Once you identify an area for improvement, the next step is to consider how to modify your curriculum to address the issue and ultimately improve learning. You can then re-assess the PLO to see if your changes made a difference or if you need further changes. It is acceptable to find problems within your program as long as you are proactive and do your best to address them in a timely and reasonable manner.
Where can I get help with assessing my PLOs?
Additional help with assessment studies, including information on campus grants for assessment research, is located in the Tools and Resources section. Other useful information, such as visits to campus by external assessment experts who can meet with interested departments and highlights of assessment studies that have been carried out, may also be found in the News and Events section.
Do I need IRB approval to conduct an assessment study?
If your assessment study results may be shared outside your program, such as through publications or conference presentations, it probably would require approval. If the information collected is useful only to your program, you do not need approval. For more information, please visit http://www.research.ucsb.edu/compliance/human-subjects/approval-process/ or contact the human subjects committee in the Office of Research at (805) 893-3807, or email@example.com.
When is my program expected to report on our assessment plans and findings, and what should be included?
Each program has been assigned to one of three reporting groups. Due dates for each group, along with a template to be used for reporting, available here. Please be sure you use the template for your group assignment and reporting date. The template contains a description of everything you need to report for that cycle.
How can one program learn from another’s assessment studies?
UC Santa Barbara faculty members interested in learning from their colleagues’ assessment experiences are welcome to contact the A-Team.
What role do students play?
Assessment of student learning must take into account the responsibilities of students. Effective learning requires students to actively participate in their educations: from carefully choosing what institution to attend, deciding what degree(s) to pursue, and selecting courses for which they are prepared, to attending in class, seeking help, and completing their assignments to the best of their abilities. Students value the opportunity to provide meaningful feedback to faculty, particularly through focus groups. For help conducting focus groups with your students, please contact the A-Team.
How do I ask additional questions or comment on the material posted to this site?
Please contact the A-Team with your questions, comments, advice, and examples of interesting, successful, or challenging studies.