Using assessment data for decision-making

Transkript

1 Using assessment data for decision-making Break-out session for curriculum facilitators, FSSM summer workshop 2007 Dr. Erica Lembke Overview 9:45-11:00 Assessment data what do we have and how do we use it? 11:00-11:15 Break 11:15-12:00 Case studies, utilizing data 12:00-1:15 Lunch 1:15-2:30 District planning for utilizing assessment data 2:30-2:45 Break 2:45-3:30 Wrap up, questions Lembke, FSSM, June

2 Assessment data what do we have and how do we use it? Types of assessment data Two large categories Norm-referenced tests (formal tests) Progress monitoring measures, teacher-made or curriculum-made tests ( informal tests) Lembke, FSSM, June

3 Types of assessment data Norm-referenced State MAP, for example District SRI, DRA, or Terra Nova, for example School individual cognitive or achievement tests administered to specific students Critical questions that are answered how does my student compare to a national or state sample in a particular subject area? How are students in our school performing compared to other students on a state and national level? PROS CONS Types of assessment data Progress monitoring measures, teacher-made or curriculum-made tests ( informal tests) Progress monitoring using curriculum-based measurement Should be standardized and can be norm-referenced Used on an on-going basis for screening or monitoring students overall proficiency in a subject Critical questions that are answered what is this student s overall proficiency in a subject? Is my teaching effectively meeting this student s needs? Can also answer the question of how a student compares to his/her peers, either locally or nationally PROS CONS Lembke, FSSM, June

4 Types of assessment data Unit tests, chapter quizzes, assessments associated with a curriculum Used on an on-going basis to monitor whether students are learning the content that you are teaching (aka mastery monitoring). Can also be used diagnostically. Critical questions that are answered is my student learning what I have taught? What skills do I need to re-teach? PROS CONS Types of Assessment Data Teacher-made tests Used on an on-going basis to monitor whether students are learning the content that you are teaching (aka mastery monitoring). Often used diagnostically (can use task or error analysis). Lembke, FSSM, June

5 Task analysis Analyzing a task by breaking it down into the smallest steps or skills Used with a task where the student experiences difficulty If the task is part of the standard curriculum, the teacher may have to access the scope and sequence charts that describe the task s subcomponents The teacher can determine the important task components Find out where the student is doing poorly and provide intensive remediation. Error Analysis Discovering patterns of errors by analyzing student s work samples Goal is identification of error patterns Work sample is scored All errors are noted An attempt is made to sort the errors into meaningful categories Lembke, FSSM, June

6 Types of Assessment data Teacher-made assessments, cont. Critical questions that are answered is my student learning what I have taught? What skills do I need to re-teach? PROS CONS Categorize your districts assessments Questions to answer: What is the assessment we re using? In what content area? How often? At what grades? What type of information is the assessment providing us? (screening, diagnostic, etc.) What is the technical adequacy of the assessment? How are we using the data? (i.e., national or state comparison, diagnostically) Is this the way the data was intended to be used? (was the test researched to be used this way?) Are there assessments that we could reduce (# of times administered) or eliminate? Lembke, FSSM, June

7 How assessments work together Norm-referenced tests Keep teachers and schools and districts in touch with how their students are doing on a state and national basis CBM Provides a technically adequate tool for determining which students are in need of intervention (can also compare to national benchmarks) Provides a technically adequate tool for determining whether instruction and intervention are effective for students Quizzes, unit tests, teacher-made tests Provides diagnostic information regarding what skills need to be taught or re-taught Providing a rationale for the use of CBM as a form of screening and progress monitoring How can the addition of CBM to a school or district assessment plan strengthen your current model? Utilizes a technically adequate model that is backed by research Time efficient and easy to implement Provides frequent data that is graphed and utilized for decision-making Can be used at the district, school, grade, class, or individual levels Lembke, FSSM, June

8 Providing a rationale for the use of CBM as a form of screening and progress monitoring Data can be used to monitor the AYP of specific subgroups Problem-solving teams have graphed data to use to make educational decisions Data can be collected and graphed on the effects of prereferral interventions Graphs are easy for parents, students, and other stakeholders to interpret Research suggests that when teachers use CBM in data-based decision-making, student achievement increases Case Studies, data utilization Lembke, FSSM, June

9 Examining and discussing data general format using the problem-solving model Define the problem Develop an assessment plan Analysis of the assessment results and goal setting Develop and implement the intervention plan Case example examining screening data All students in a school are screened The principal has access to all school data Each teacher gets a graph of data for his/her class Students are placed in Tiers 1-3 based on this screening data See handouts for examples of screening data at the school and class-wide level Lembke, FSSM, June

10 Suggested steps for discussing screening data school-wide data Assessment plan: School-wide data overall Define the problem: Are there particular grade levels that are not achieving as we would expect? Analysis of the assessment results: What could be the cause of this lack of performance? Curriculum? More support needed for novice teachers? Large numbers of students at-risk in a particular grade? Management skills needed? Develop and implement the intervention plan: Are there resources that need to be allocated for this grade level (personnel, professional development, curriculum)? How will the plan be put into place? What decision-making rule will you put in place to assess the effects? Analyze data at next screening point? Sooner? Define the problem: Are there particular grade levels that are attaining superior performance? Analysis of the assessment results: What could be leading to this superior performance? Teaching and/or management skills that could be modeled for others? Suggested steps for discussing screening data school-wide data, specific skills Same questions as on the previous slide, but specifically related to particular skills (nonsense-word fluency or oral reading fluency, for example) Develop and implement the intervention plan: In the case of nonsense word fluency, are there particular skills that need additional instruction in a particular class so that decoding and use of the alphabetic principle (which will lead to increases in NWF) are bolstered? What decisionmaking rule will you put in place to assess the effects of this intervention? Lembke, FSSM, June

11 Suggested steps for discussing screening data individual class data Assessment Plan: Provide grade level teams with an overview of their class data for each skill Questions to discuss: Define the problem: Are there particular students that are not achieving as we would expect (especially students in Tiers II and III)? Do we need to follow up with an additional screening task at this time? Analysis of the assessment results: What could be the cause of this lack of performance? Curriculum? More support needed for novice teachers? Language issues? Management issues? Develop and implement the intervention plan: What resources/interventions/strategies can be implemented for individual students or groups of students at this time? Monitor effects using progress monitoring. Decisionmaking rule? Suggested steps for discussing individual progress monitoring data (See Jim s graph) Assessment plan: How often has the student been progress monitored? What was the student s median baseline score? (present level of performance) What short term objective (growth criteria) or long range goal (benchmark) did you decide on for the student (i.e., number of words or digits gained per week)? Why did you decide on this? Define the problem/analyze the assessment results: What does the data indicate so far? Is the student on track to meet, not meet, or exceed his/her goal? Does it appear the instruction that you are using for this student is working? Why or why not? Develop and implement the intervention plan: Is an instructional change needed at this time? Have you determined what intervention you might implement with this particular student? If yes, what are some you ve thought of? If no, brainstorm with the group to come up with some. How will you determine if the intervention is evidence-based? Lembke, FSSM, June

12 Discussion of the process How did the format work? Pros/cons? Additions/changes? How can discussion of screening and progress monitoring data enhance what you re already doing? District planning for utilizing assessment data Lembke, FSSM, June

13 Needs assessment of your assessment! HNS chart What do you Have? What do you Need? What are your next Steps? How can you close this gap? What do you Have? First, look through the list of assessments you compiled earlier Discuss quality of information. Are these assessments providing you with the information you need to make educational decisions about students? Discuss additions or changes that need to be made Lembke, FSSM, June

14 What do you Need? With respect to additions or changes that need to be made in your assessment process, what tools might fulfill your needs at this point? Are you currently getting all of the assessment information that you need for all students: On a frequent basis? That you are confident in? That you can use for educational decisionmaking (big and small)? What are your next Steps? List your next steps or action planning items. Include specific and measurable steps, who is responsible, resources needed, and the date to be accomplished. Reflect on the resources you have, including staff, current assessment tools, and other supports What additional support/resources will you need to accomplish your goals? Lembke, FSSM, June

15 Discussion, questions What additional questions, comments, needs do you have at this point? How can I support you in your implementation? Thanks and let me know how I can be of assistance! Lembke, FSSM, June