Useful Links:
- Assessment Overview Slideshow - Helpful Worksheet ("Equity Assessment Guiding Questions")
- Tableau Dashboards
- VPN Access Setup (Required for viewing Tableau Dashboards)
- Undergraduate Equity Tableau Dashboard
- Flow of Major Diagram
- Course Grade Distribution
- Graduate Dashboards (no login required)
What Exactly Are You Looking For?
Assessment projects are designed to support equitable achievement of your learning outcomes, across students in your major. This is achieved by looking at your PLOs through a lens of disaggregated data about student learning across effectors of opportunity, compared with the program’s observable goals/benchmarks for student learning. The Council on Assessment is looking for the following in an Assessment Report and Plan:
SECTION I
- A concise summary of your findings from the last cycle of assessments
- What the program learned from them (i.e., is X meeting your standards? would you like to see Y improve?)
- What the department has done/is doing with that information.
Here, we can see whether your prior plan was (a) realistically achievable and (b) practically useful for your program.
SECTION II
- Some current information about your program (degree name, student enrollment, faculty involved in this assessment) and a review of your PLOs and curriculum map.
This is your opportunity to describe any changes made to the program in the last three years, as well as to request assistance in updating your PLOs, if you would like.
SECTION III
- An equity-focused plan that looks at data about student learning, to formulate questions and collect evidence regarding the program's explicit goals for student learning (i.e., your Program Learning Outcomes).
- This plan should utilize institutional data about your program (department dashboards, equity dashboards, etc: provided to your program by our assessment team), examine direct evidence (i.e., actual student work) as well as indirect evidence (e.g., surveys and focus groups about students' experiences), include a criterion-based rubric/assessment tool specific to your inquiry, and a offer a timeline of when/where data will be collected and who/how it will be assessed.
Your plan focus can be either the same PLO as the prior cycle (if you need further data or need to adjust your method of investigation) or a different PLO (if that inquiry is complete, or the program currently wants to prioritize another program outcome).
We have created a Template for PLO Assessment Plans that lays out all of these parts in question-answer format, for your convenience!
Midway through the second year of implementing your plan, we will request a brief Progress Report, mainly to ensure that—if you needed to make changes to your initial plan, or would like any support in completing it—we can help you in a timely manner.
And that’s it! Just those two documents, every three years. PLO Assessment achieved.
Step-by-Step: Creating an Equity-Focused Assessment
As outlined on UCSB’s Institutional Accreditation page, equity-focused assessment of your program’s learning outcomes begins by looking at Institutional Data: department profiles, equity dashboards, and course grade distributions on Tableau, as well as any other pertinent data from UCSB/UCOP.
First, formulate pressing questions (aim for 5) about undergraduate and graduate learning in your program. Then list details in these data (4-7 things) that you noticed, with reasons why you found them interesting, what they made you wonder about. Next, prioritize this list based on which of these areas of interest you see as most important to equitable achievement of student learning in your program. For each of these prioritized items, look at the dashboards and list other useful data related to this topic of interest.
Finally, use these reflections to develop a PLO Assessment focus: 2-3 questions about an item on your prioritized list, related to courses and/or instruction in your department, focused on achieving a specific program learning objective. Consider what direct evidence (criteria-based quantitative data and expert assessments) you can collect to better understand what is happening with student learning in the program, as well as what indirect evidence (surveys, focus groups, interviews about learner experience) you can collect to understand why it is happening.
So, to create a PLO Assessment plan step-by-step:
- Take inventory of your program’s current known challenges, upcoming curriculum changes, etc. where data would be useful. If you’d like help thinking through topics and data that would be most interesting and helpful to your program, feel free to meet with one of us! Or use this Guiding Questions form.
- Look at institutional data. If you’re in an undergraduate program, start with Major Flow dashboards, Course Equity tabs 2a/b/c/f (major changes and pre-major enrolls, student probation and completion rates), and Course Grade distributions. If you are in a graduate program, start with UCSB statistics (student enrollment, time to degree & advancement) and UCOP data (Exit Surveys, Grad Statistics, Doctoral Rates, and Program Details). VPN access will be necessary for viewing undergraduate data. If you don’t know who in your department has the password to access these dashboards, send us an email!
- Decide on an area to investigate for your 3-year Assessment Plan. You can write this into question #12 on your copy of the PLO Assessment Report and Plan template, which we’ve crafted to help you complete your plan proposal.
- Name the Program Learning Outcome(s) that this particular skill/knowledge area relates to your current approved PLOs.
- Determine from what courses or milestones you will collect data for your assessment, and a method for sampling student work there (template question #14).
- List criteria for PLO achievement & describe observable levels of achievement for the data you will collect. Consider also any survey/interview questions that you would like to ask students about their experiences with learning this material, navigating the program, and insights or suggestions for the future.
Information on crafting useful Scoring Guide/Rubrics is here. Or email us for support!
When Do You Want It?
UC Santa Barbara operates under a three year cycle for assessing program learning outcomes. Every three years departments report on the recent PLO assessment, and create a plan for the next PLO assessment.
Is your program in Group 1, 2, or 3? Here is a list of Assessment Groups/Cycles by Department.
Assessment Groups/Cycles by Department
At the beginning of the three year cycle (in January), your department must report on its assessment findings from the prior cycle AND its plan for the next three year PLO assessment.
Midway through year two of your three year PLO Assessment Plan (in October), a progress report is due. This is a required document.
Email all documents to Josh Kuntzman, our assessment coordinator.