Continuous Improvement Policy

°Õ³ó±ðÌýUndergraduate Program Continuous Improvement PolicyÌý[1]Ìýspecifies the procedures that the department follows to ensure continuous improvement of its undergraduate degree programs. The department recognizes that in order to provide a high-quality program that meets the needs of its undergraduates, it must make use of multiple forms of feedback and assessment to collect the data it needs to evaluate the current status of the program and to make sound recommendations for improvement.

To guide this process, the department has established a set of program outcomes and program objectivesÌýfor the BS degree program.ÌýOutcomesÌýare statements made about the skills and capabilities of senior BSCS majors at the time they graduate.ÌýObjectivesÌýare statements made about the skills and capabilities of Computer Science alumni approximately three to five years after they graduate.Ìý The truth of these assertions is established via the assessment and evaluation mechanisms described by this policy.

The department’s website is used to make all continuous improvement-related documents available to the public including listing the outcomes and objectives, the continuous improvement policy and the annual report.Ìý An archive of past versions of these documents is also maintained along with a summary of the basis for any changes made.

Two committees, the Undergraduate Committee and the Curriculum Committee, are charged with designing and maintaining the courses and degree requirements for the program in such a way as to ensure the program’s outcomes and objectives will be met.Ìý Duties related to the task of continuous program improvement for each of these committees is described in this document.

The department’s Continuous Improvement Policy is based on the following processes:

  • Assessment of continued validity of the Program Outcomes and Objectives
  • Assessment of student attainment of Program Outcomes and Objectives
  • Course Assessment Policy
  • Ensuring foundation courses stability

ÌýThese process are based on the use of the following:

  • monitoring feedback for and changes made to individual courses
  • monitoring ratings of course instructors
  • seeking feedback from students, alumni and employers via surveys, alumni and employers involved with course offerings, student representation on Curriculum Committee, and Advisory Review Board Meetings. For more specific see Surveys and Advisory Board sections below.
  • conducting annual survey of faculty about student preparedness for their courses
  • student performance on assessments conducted in the CSCI 4308/18 and CSCI 3100

This information is then used by the Faculty, and the Curriculum and Undergraduate Committees to make changes in the program to address problems and improve quality.

Assessment of Continued Validity of the Program Outcomes and Objectives

Each year, during the fall semester, the Curriculum Committee reviews inputs from the following sources in order to assess the continuing validity of the Program Outcomes and Objectives:

  • CEAS conducted surveys of graduating seniors, alumni, and employers contain questions specific to program outcomes and objectives
  • Associate Chairs’ meeting with senior project sponsors each May

If, after reviewing these inputs the Curriculum Committee determines there is a need to modify either or both of the outcomes and objectives they will draft proposed changes and present them to the faculty for review. The Curriculum Committee and Faculty then iterate on these until an approval is granted by the faculty at which time the revised outcomes and objectives are reposted to the department’s website.

During the final year of an ABET cycle the program outcomes and objectives are further reviewed by the department’s advisory board. Any suggested modifications are then fed into the regular department review cycle.

Assessment of Student Attainment of Program Outcomes and Objectives

The department faculty define success criteria for students to meet with respect to both the Program Outcomes and the Program Objectives.Ìý Student success is assessed annually.Ìý

Program Outcomes are assessed in terms of successful completion of specific coursework based on Program Outcome Success Criteria adopted by the faculty.

All students graduating from the BSCS program must have been exposed to course work that provides and understanding of all ABET criteria.Ìý The department’s curriculum is designed to ensure meeting this ABET requirement through the foundation courses and by requiring that all students graduating pass either CSCI 4308/4328/4338 or CSCI 3100, in that case that a non-project capstone option is taken by the student. The foundation courses require student achievement of all criteria except for the professional development related criteria 3.e and 3.g.Ìý The senior project options and CSCI 3100 provide coverage of those topics with the exception of security, which is covered in CSCI 2400.

Because of the importance of ensuring adequate coverage of topics to ensure meeting ÌýABET criteria the content and delivery of the foundation courses, CSCI 4308/4328/4338 and CSCI 3100 are closely monitored and controlled by the Curriculum Committee.Ìý If a student fails to successfully complete one of these options they must either repeat that course or elect another capstone option and complete that during the following academic year.

For students electing to complete the CSCI 4308/4318 capstone option an additional review of their achievement of the program outcomes is conducted via a survey completed by project sponsors during the Sponsor Post-Mortem in May of each year.

In lieu of an exit exam, the Department will review the achievements of select students on assessments designed to measure student achievement of the program outcomes.Ìý The students will be selected at random from each of the capstone options in numbers proportional to the number of students taking that capstone.

After graduation each year student success in achieving the faculty defined criteria for student success in achieving the Program Outcomes is assessed based on achievements by a randomly selected subset of students.

Student success in achieving the Program Objectives is assessed via the Alumni Survey annually and the Employer Survey when available.Ìý These surveys are administered by CEAS.Ìý The Alumni Survey is run annually and sent to students 3-5 years after graduation.Ìý The Employer Survey is run every 3 years.

Upon completion of the success study each year the Curriculum Committee provides a summary report to the faculty for review, which then reviews student success and any plans for improvement.Ìý Upon approval by the faculty the results are published in the department’s annual report.

Course Assessment Policy and Process

All undergraduate Computer Science courses (with the exception of special topics courses, independent study courses, and courses for non-majors) are required to make publicly available on a semester-by-semester basis the course profile.

The Curriculum Committee will review the course profile for all foundation courses annually.

Course Profile

The course profile includes the following information:

  • course syllabus
    • course metadata (title, number, credit hours, coordinator, URI)
    • catalog description
    • textbook
    • course outcomes
    • relationship of course outcomes to program outcomes
    • prerequisites by topic
    • major topics covered
  • assessment plan for the course
  • pointer to course repository (usually moodle but can be bitbucket)

The course profile is used to track important information about a course and represents a contract as to what is taught in the course and what the department can expect from students that successfully complete the course.

The Curriculum Committee must review any changes to a course profile before these changes become official. This review ensures that consistency across course offerings is maintained and that any changes related to course outcomes do not adversely affect courses dependent on those outcomes.Ìý All externally visible changes must be announced to the faculty of the department.

Faculty Course Questionnaires

Faculty Course Questionnaires (FCQs) are administered by the University every term for each instructor of each class taught. FCQs allow students to rate their instructors and courses and provide information to instructors for teaching improvement, to students for course selection, and to deans and chairs for management.Ìý Detailed FCQ results are available on the CU Faculty Course Questionnaire website, while summaries of FCQ results for Computer Science courses are available at Computer Science Faculty Course Questionnaires website.

The Curriculum Committee will review the FCQ ratings for all courses in October and February of each year.Ìý For any courses for which the ratings fall below a 4.0 the faculty member who taught the course will be required to submit a report documenting the reasons for the low rating and suggesting an improvement strategy.Ìý

Course Archives

To assist in the department's goal of improving the quality of its undergraduate program, the Curriculum Committee is adopting a uniform policy with respect to the collection of course materials from undergraduate courses into course repositories. This information is also useful for maintaining high-quality course dossiers that must be presented to ABET reviewers during an ABET site visit.

Course repository is maintained either on moodle or in bitbucket. It must include the following items:

  • course syllabus
  • instructions for labs/demos conducted in class
  • examples of graded assignments/lab assignmentsÌý (with examples of student work from low, medium and high performing students and student identities concealed)
  • examples of graded quizzes and exams, including mid-terms and finals (with examples of student work from low, medium and high performing students and student identities concealed)
  • copies of (or pointers to an archival location) course materials used in instruction (e.g., workbooks, textbooks, handouts, etc.)

These materials must be archived within 1 month of the close of the semester in which the course was taught.

Because of the different nature of the courses, Senior Thesis and Project courses follow different guidelines.

Senior Thesis

The final thesis document of an undergraduate senior thesis must be submitted to the Main Office Administrator in electronic form to be stored online.

Project Team-Based Courses

In project team-based courses, artifacts such as documents and source code often undergo multiple changes throughout the semester.Ìý Rather than collecting each version of each artifact, the course coordinator should collect only the completed project for each team for archive.Ìý Projects completed by three project teams -- representing low, medium, and high performing teams -- should be submitted.Ìý

In some sponsored project courses it is not possible to collect all project artifacts. At a minimum the archive of a project must contain sponsor evaluations, peer evaluations, team advisor evaluations, team posters, and project summaries, and class presentations.

The Curriculum Committee is responsible to ensure that materials are collected for each instance of an undergraduate course and to assist in the process of collecting material from each course that was taught during the previous semester. In particular, a reminder asking for course materials will be sent at the end of the last week of classes. A second reminder will be sent at the start of the first week of classes of the next semester. Finally, a third reminder will be sent starting on the third week of the semester after a course has been taught, with weekly reminders sent until the required materials have been acquired.

Ensuring Foundation Courses Stability

In order to ensure that all ABET criteria are satisfied by BSCS graduates stability of the content and delivery of the foundation courses, CSCI 4308/4328/4338, and CSCI 3100 must be assured from semester to semester.Ìý In order to achieve this goal, faculty members are provided with the courses syllabus two months before the beginning of each semester. If a faculty member wishes to change the course outcomes or textbook the suggested changes must be reported to the Curriculum Coordinator. If the course is a foundation course or CSCI 4308/4328/4338 or CSCI 3100 the suggested changes must be reviewed by the Curriculum Committee, which then will iterate with the faculty members until agreement is reached.Ìý When the course changes are approved the syllabus is revised, the old version is archived for future reference, and both are made publicly available on the department’s Courses web page.

Surveys

Each year, the College of Engineering and Applied Science administers a number of surveys to collect information from various college stakeholders – students, alumni and employers – which can be used to assess the quality of the College's departments and programs. These surveys include theÌý

  • Freshman Survey –Ìý administered to students at the end of their first year in the College
  • Senior Survey –Ìý administered to students in the month before they graduate
  • Post-Graduate Survey – administered to students roughly six months after they graduate
  • Alumni Survey – administered to alumni the fourth year after they graduate
  • Internship Survey – administered to students who take internship positions with companies during the Summer
  • Employer Survey – administered to companies that hire our graduates

The Curriculum Committee will make use of the data from these surveys as one point of independent feedback concerning the quality of the BS degree program and whether or not the goals of the program as stated by the program outcomes and program objectives are being met.

In particular:

  • The Undergraduate Committee will review the results of the Freshman, Senior, Post-Graduate, and Alumni surveys on an annual basis and transform the knowledge gained into recommendations on how the BS degree program can be improved.
  • This information will be captured in the Department's annual assessment report and will also be submitted to the Chair of the Department and the Executive Committee for review and guidance on how to prioritize the various recommendations made.
  • The Undergraduate Committee will also review the questions asked on these surveys and recommend updates to the College in time for the updated questions to appear on the next instance of each individual survey.

Advisory Review Board

The board's job will be to provide external evaluation and feedback on the Department's goal of continuously improving its undergraduate program.

  • The Chair of the Undergraduate Committee will ensure that the review board has reviewed a significant percentage of the undergraduate program by the time of ABET review of the department, which typically occurs every six years after accreditation is received.Ìý The student work shown to the independent review board will be drawn from the work gathered as part of the collection of course materials as outlined by the course assessment policy noted above.
  • Finally, the Chair of the Undergraduate Committee will be responsible for presenting the "state of the undergraduate program" to the Department's Advisory Board and soliciting their feedback and evaluation of the current program and recommendations for improving the program. These recommendations will also be captured in the Department's annual assessment report. It will be the responsibility of the Curriculum Committee to review the board's feedback and enact changes in response.

[1] Revised Feb 2014, Apr 2015, Nov 2015