UPDATE TO ASSESSMENT PLAN - June 22, 2000

To: Cecilia D. Gray,
AssociateVice President, Academic Affairs
and
Joseph Sheley, Dean, SSIS

From: Dick Kornweibel

This memo responds to your separate but related requests for information on the status of implementation of the Social Science program's assessment activities. First, let me report that we were unusually fortunate to have begun collecting data useful for learning outcome assessment without knowing what we were doing. As far as raw materials we have student portfolios for every semester since Spring, 1997. We also have detailed information (self reported) by students on their preparation referenced against the California Curriculum Framework for several semesters. This summer we will collect from university sources comparative data on the performance of our majors relative to history majors on the WPE and in our key comparative courses (History 192 and 193). Under our assessment plan I am to present the accumulated data for the first time to our program committee during the Fall, 2000 semester. Whether the program committee would propose changes on the basis of this first presentation of data is of course unknown. Since we will be doing program review next year, my guess is that the program committee will want to consider assessment data and program review recommendations simultaneously.

Our self study is in final stages of preparation. By the time an outside consultant reviews the program during AY 2000/2001, we will have our analysis of the data referenced above for the reviewer's consideration and we would also hope for suggestions on improvement of our assessment plan.

Recently Dean Sheley distributed a useful document AAdvancing SSIS Learning Outcome Goals and Assessment Strategies. My judgement is that in major ways our program's articulation of learning outcome goals and assesement strategies is consistent with the framework presented in the document. We will need to make some minor adjustments. The dcument concludesPages two and three present provide a checklist for the development of a plan:

Question one asks if our assessment plan identifies our majors. Our plan identifies our majors and suggests their career goals. At this point we don't have a strong (or a strategy for acquiring) strong data on alumni.

Question two asks if our assessment plan articulates what knowledge and skills we expect student to have at graduation. Please refer to our Assessment plan, under the heading Program Goals.

Question three asks if we can confidently demonstrate that our curriculum supports the student acquisition of the knowledge and skills we have identified. Our Assessment plan suggests that while we will review or data annually, we may not want to make major changes for a few years.

Questions four asks if our vision has a regional focus. At this point our program suggests a state rather than a regional focus.

Question five asked for details on our assessment strategy. Our submitted assessment plan has those details.

Question six asks for an implementation schedule. This is also addressed in our assessment plan.

Question seven asks if we have made plans to inform students of our vision for their achievement and the assessment strategies designed for the measurement of progress towards meeting that vision. We did not include that as an element in our assessment plan and will amend the plan. Likely we will want to post on a web page but also prepare some kind of a printed document for students as well.

Question eight asks whether we have plans to update and reevaluate our learning outcome goals and assessment strategies. We plan to rely on the program committee's response to the annual report as noted in the assessment plan; second, on materials that emerge from the periodic program review and finally from any changes or additions to the externally generated standards for subject matter preparation for social science teachers.

 

ASSESSMENT PLAN

The dual and linked missions of the CSUS Social Science Program are to provide an academic major and meet standards of subject matter preparation in social science for a California single subject teaching credential. The current program is the third program created at CSUS for this purpose. Each successive program has been based on then current university criteria for a major as well as state standards for credential certification. The program structure also maintains elements derived from one of the campus' oldest (but now abandoned) majors, the distributive social science (21-9-9) program. The current program is essentially the expansion of a predecessor program with modifications based on Social Science Teacher Preparation in California: Standards of Quality and Effectiveness for Subject Matter Programs, issued by the state's Commission on Teacher Credentialing in 1992. Standard fourteen of that document requires that an approved program: "...uses multiple measures to assess the subject matter competence of each student..." and Standard fifteen requires that a program: "'...has a comprehensive, ongoing system of review and development...". Thus, the approved CSUS Social Science Program in its design anticipated current CSUS requirements for assessment as well as the university's system of program review. However, as practiced in recent years, program practice has been to assess if individual students meet standards of subject matter preparation. This must continue but this document attempts to identify means by which we may also monitor indicators of student learning for students as a group.

SOCIAL SCIENCE PROGRAM GOALS

1. Develop student skills in reading, writing, analysis and speaking.
2. Provide students with the opportunity to acquire appropriate subject matter preparation which is the mandated subject matter for middle and high school teaching in social science. This is primarily in American government, international politics, economics, geography, US history and world history. Students should have both breadth of preparation, that is, a comprehensive introduction to the fundamentals of the identified areas as well as some opportunities to study representative subject matters in depth.
3. Assist students develop critical and analytic skills, principally, but not exclusively in history. Develop in other words the ability to identify and analyze problems of interpretations and be familiar with different systems of analysis.
4. Provide instruction in the basics of the origins of the social science disciplines and assist students in developing the ability to distinguish among the disciplines' systems of knowledge.

SOCIAL SCIENCE ASSESSMENT PROGRAM DESIGN

The majority of social science students begin at CSUS as community college transfer students, the second largest constituency is composed of post baccalaureate students seeking subject matter preparation. The major has very few freshmen students so the basic program design is constructed to meet the typical circumstances of community college transfers. However, it adapts easily to either freshmen or post B.A. students.

1. Students upon admission are advised about program requirements. An initial individual assessment determines what prior work can count to meet program requirements. Students are informed that they may use program courses both for the major and for subject matter preparation and that many courses may also be used to satisfy university graduation and general education requirements.
2. In the initial advising assessment students are advised that their first priority is to complete any remaining lower division (introductory) course work and to take the Writing Proficiency Exam in the first semester. They are also informed that all courses must be passed with a grade no lower than C- and that a GPA significantly higher than the university minimum (2.0) is required of anyone who wishes subject matter certification.
3. Students are also informed that the program has two summative or capstone courses: The History 192/193 and the Social Science 193 and that these should be taken at the end of a student's program. They are also advised to save all papers and exercises since they will be required to submit a portfolio as part of the Social Science 193 and that their portfolio will include samples of work from several courses.
4. Students are then expected to take identified program courses in a reasonable order. The program includes eleven specific courses, three courses each chosen from a restricted list of two or three options and finally six courses each taken from more extensive lists. All courses have been selected to meet specific requirements identified in the Social Science Standards.
5. All the program history courses (students take at least eight) require graded writing as do most other program courses. Almost all program courses require writing assignments that demonstrate students' ability to analyze and synthesize material. All program courses require reading.
6. Many program courses may include graded oral work but two courses require it: The History 192/193 and the Social Science 193.
7. Social Science 193 formally introduces the history of the Social Sciences and the most common systems of knowledge and analysis employed by them and by history.

ASSESSMENT PLAN AND INSTRUMENTS

Presented in the order typical student experience would provide the raw data.

1. Monitor performance of majors on the WPE. Since many majors plan to be teachers and since California needs teachers with strong literacy, program students on average should do at least as well on the WPE as other CSUS students. Further, since students in this program take at least twenty-four units of history, an expectation that they perform about as well as history majors is reasonable. If between the 1999-2000 program review and the 2004-2005 program review WPE results are consistently below WPE results for history majors the program should consider additional writing requirements. If results are consistently below campus WPE averages, additional writing requirements should be begun.
2. Compare performance of history and social science majors in the History 192/193. In its assessment plan the History Department intends to regularly collect a selection of papers from the 192/193 sections "that would indicate the progress toward achieving the History program objectives made by students in their course of study and their level of achievement at the end of their program." Since virtually all students in History 192/193 are either history or social science majors an analysis of papers such as the history department proposes would be useful to the social science program as well. However, one additional step would have a useful if not provocative benefit: Each semester tabulate the average grade of history vs. social science students in the 192/193. If the average grade of social science students is consistently and significantly lower, then the program must seriously consider program modification.
3. Collect, summarize and analyze Curriculum Framework assignment in Social Science 193. Quite by accident we have a marvelous assessment opportunity that comes from an assignment in the Social Science 193. This assignment was designed to assist individual students to assess their preparation for a teaching assignment. Each student is asked to measure her/his preparation against California's Social Science Curriculum Framework. Each student reads the framework paying specific attention to the semester by semester subject matter for grades seven through twelve. The student then estimates what per cent of the framework she or he is prepared to teach. Since this has been a tool of student self-examination we have not organized the assignment in a form suitable for formal program analysis. Prior to Fall, 1999 we will have the assignment reorganized. If students consistently report a perception of under preparation in a subject area the program should adjust the curriculum accordingly.
4. Use the portfolio prepared in the Social Science 193 as the single most important assessment tool. As currently used, the Portfolio presents to a reviewer:

   (a) evidence of completion of a course of study via advising forms, transcripts etc.

   (b) samples of work from classes in at least three disciplines.

   (c) reflective essays that along with work samples provide evidence of writing skill and ability to do analytic work.

   (d) evidence of performance on assignments devoted to the origin and development of the social sciences

   (e) evidence of quality oral presentations in the SS 193

   (f) evidence of quality group work in the SS 193

   (g) evidence of the student's ability to assemble and present a quality portfolio

To this point, the purpose of portfolio evaluation has been to determine if the individual student meets standards of subject matter preparation. However by modifying the portfolio evaluation, that task can still be accomplished while the additional task of program assessment can also be accomplished by reorganizing evaluation according to the Primary Trait Analysis technique. Essentially, each significant component of the portfolio to be assessed will be measured on a five-point scale. For each of these components an average score for all portfolios will be determined. By comparing the average score of the various components the program can determine which areas are strong and successful and which, if any, areas are weak and need revision.

ASSESSMENT PROCEDURE

Each Spring, the program coordinator will assemble materials from the past two semesters and at the beginning of the Fall semester will present the program assessment portfolio the program committee. By the time of the 2004-2005 program review there should be significant evidence for consideration of program effectiveness.

 

September 1999