Methodology

__Research Design from Kristina__ please, edit away! Suggestions are welcome!  A. Description Through the valuable use of quantitative and qualitative methods, this correlational (quasi-experimental ?) research study will determine th e effects of various support interventions on the number of office disciplinary referrals teachers and administrators write for students participating in the study. Because of the different treatments implemented at different schools and a control group, there would be three separate research designs using a pre- and post-test group design and the control group design. //(The group also would like to use a qualitative approach by creating three case studies within each of the schools. This would allow us to track one student using each PBS intervention and document potential positive changes in the student’s quality of life.) I think this is adding more work now that I am writing this out … we had talked about the emic/etic perspectives and talking to teachers/students/etc but can we do this more with questionnaires and not individual case studies and observations in the field?// Can it be a quasi-experimental, correlational study? -- Kristina

B. Threats to internal validity, steps to minimize Teachers and administrators write office disciplinary referrals for students they feel are breaking school rules by fighting, possessing weapons or drugs, insubordination, classroom disruption, and other reasons. But faculty members have various degrees of tolerance, and those forbearances can change for the same person from day to day; consequently, the number of referrals written may not accurately reflect what has occurred. As Vavrus and Cole noted, “the language or behaviors that a teacher defines as a disruption vary depending on the persons in a particular class and their social interactions” (2002, p. 90). As a result, it may be difficult to erase these threats because a person’s tolerance level is difficult to standardize. However, we can encourage schools to be as specific as possible when detailing behaviors and consequences. Because of individual interpretation, it will not be possible to fully eliminate bias. Other threats to internal validity could be teacher turnover, a change in the student population and "seasonal variability;" Luiselli, Putnam, Handler and Feinberg (2005) discovered during their research that "seasonal influences such as post-vacation return to school" (p. 195) affected the number of referrals teachers had written. As a result, we will consider their recommendation to examine the data on a year-to-year basis and not on a straight linear foundation. This is a great section, Kristina! C. Limitations to generalizability Positive Behavior Support interventions by and large have constructive outcomes, but most of the studies conducted have been at the elementary level. Our study aims to implement various interventions at high schools in urban settings. Because of the challenges high-risk students in urban secondary school settings face, this study should be able to be generalized to other schools and districts with similar populations. Potential limitations include the parent and community support variables; it is challenging to generalize how well those stakeholders will become involved in their local schools.

D. Qualitative component? If so, criteria to judge credibility/trustworthiness in our design The qualitative component gives the study a more human interest perspective. It would consist of a series of interviews conducted during the initial planning phase, the first two years of implementation and during the final year. //(Again, let me know if we want to go there …. One concern would be: What if the student moved?) Could we include 2 students (more work) to ensure some qualitative data for each intervention?//

(Cody)- I think it turned out ok. Not too much extra work... For qualitative research purposes, the student population will be classified into three subgroups (e.g. no-risk, moderate-risk, high-risk) according to the number of office discipline referrals issued to each student. Stratified random sampling will then be used to draw 1 participant from each subgroup in all four high schools. This sampling method will help ensure that students from all three subgroups are represented in the sample population used for the case studies. Furthermore, purposeful sampling via critical case rationale will be used to draw additional units of analysis from each subgroup. The additional unit of analysis will consist of 1 student from the high-risk subgroup in each of the four schools. The additional units of analysis from the high-risk subgroups will be used to obtain supplementary detailed information about the effectiveness of PBS interventions on students identified to be high-risk and will be used as disconfirming case analyses to strengthen study findings. These additional units combined with the four units drawn via stratified random sampling, will yield a total of 8 units of analysis for qualitative analysis. Informed, written consent from parents or legal guardians will be obtained prior to contact of any case study participants. This study will require the use of volunteers to complete the school climate questionnaire. The anonymous questionnaires will be sent to parents or legal guardians and will include additional information (not included in site administered questionnaires) including demographic information, income and level of education of parents or guardians. To maximize return-rate, teachers will be provided with incentives to offer students. This information will increase generalizibility of results from the study’s sample population.
 * Sampling** (-Cody)

Population The population and sampling structure for this study will consist of students from 4 New York City high schools. The sample population will include all students in grades 10, 11 and 12 enrolled in the 2009-2010 school year (grade 9 students are excluded due to the inability to collect baseline data for these students). The most current enrollment data for the selected schools is: Beacon High School- 1085, Benjamin N. Cardozo High School- 3902, Christopher Columbus High School- 1554, DeWitt Clinton High School- 4392. Warren et al. (2003, p.2) describes urban schools as “those characterized by severe poverty, community violence and high rates of problem behavior”. New York City was chosen as the sampling base because it holds many schools that fulfill this definition of “urban”. The schools chosen in this study fit this definition but also contain a very diverse student body with a valid population size that will enable results to be generalizable to populations with similar characteristics. In an effort to minimize study costs and increase consistency of community support interventions, the sample will be taken from a regional and not national setting. For qualitative research purposes, the student population will be classified into three subgroups (e.g. no-risk, moderate-risk, high-risk) according to the number of office discipline referrals issued to each student. Stratified random sampling will then be used to draw 1 participant from each subgroup in all four high schools. This sampling method will help ensure that students from all three subgroups are represented in the sample population used for the case studies. Furthermore, purposeful sampling via critical case rationale will be used to draw additional units of analysis from each subgroup. The additional unit of analysis will consist of 1 student from the high-risk subgroup in each of the four schools. The additional units of analysis from the high-risk subgroups will be used to obtain supplementary detailed information about the effectiveness of PBS interventions on students identified to be high-risk and will be used as disconfirming case analyses to strengthen study findings. These additional units combined with the four units drawn via stratified random sampling, will yield a total of 8 units of analysis for qualitative analysis. Informed, written consent from parents or legal guardians will be obtained prior to contact of any case study participants. This study will require the use of volunteers to complete the school climate questionnaire. The anonymous questionnaires will be sent to parents or legal guardians and will include additional information (not included in site administered questionnaires) including demographic information, income and level of education of parents or guardians. To maximize return-rate, teachers will be provided with incentives to offer students. This information will increase generalizibility of results from the study’s sample population.

 __Variables__ from Kristina A. Quantitative (independent and dependent) In Schools A, B, C, and D, students receive office discipline referrals for inappropriate behavior. School A is the control population. School B students receive token reinforcement/reward tickets (independent variable) for appropriate behavior (dependent variable) School C students receive counseling services (independent variable) to help manage their behaviors (dependent variable). School D students receive community/parent support (independent variable) to help manage their behaviors (dependent variable). School climate also will be measured (dependent variable) through questionnaires and surveys about the interventions in place (independent variable). B. Qualitative aspects of cases on which data collection and analysis will focus (Cody) Qualitative analysis will focus on the effectiveness of PBS interventions on reducing the case study participants’ levels of problem behaviors, levels of recidivism and number of ODRs.

-What do you all think? I wanted to write more but every time I tried, the ideas ended up not being related to the qualitative aspects...  __Methods of Data Collection__ from Kristina A. How to measure variables Before the PBS interventions are implemented, administrators at each school will distribute student handbooks outlining the school’s disciplinary procedures, and teachers will review the policies with students in homerooms. Researchers will gather data during that baseline year to document the number of office discipline referrals written for that timeframe. During the first year, teachers will undergo training for their school’s specific intervention program and will participate in surveys or questionnaires to assess their beliefs about student discipline, positive behavioral support programs, community support, or counseling. Once the interventions are in place, researchers will monitor the number of referrals written and survey the teachers about their beliefs in the program’s use and effectiveness. This will continue during the duration of the program and the year after the program has been phased out.

B. Validity and reliability of measures The reliability should be considerably high because data collected from questionnaires and surveys will measure thoughts and opinions about the process before, during and after it occurs. The validity of the data gathered with the number of office discipline referrals should remain steady if teachers issue them in a consistent manner. Based on prior research, the number distributed should decrease as the intervention year progresses. Schools using the token reinforcement model will monitor student positive behaviors with tickets (The 2005 Luiselli et al. study called them “Caught in the Act” or CIA slips, p. 188). Previous studies have discussed how the reinforcement component is assessed, and concerns have arisen with the validity of using tickets. In both the Luiselli et al. (2005) and Lane et al. (2007), studies, students were required to drop their reward tickets in a labeled container for weekly or monthly future raffles. However, Lane et al. noted that “it is possible that students could have received tickets but not turned them in to the lottery system” (p. 17). This threat to the external validity can be avoided if teachers maintain classroom containers where they immediately drop the students’ reward tickets or if schools use raffle tickets that are two-sided.

C. Qualitative Perspective of data collection (Cody)

In order to comprehensively assess the effectiveness of PBS interventions on suspension rates and recidivism of identified “high-risk” students, qualitative measures will be conducted. These measures include interviews that will be presented to staff using an ongoing member-checking system (Lincoln & Guba, 1985) to increase credibility, document reviews and comprehensive field notes that will be used to help inform intervention structures during planning and modifications to interventions during implementation.

__Data Analysis Procedures__ (Cody) To analyze qualitative findings, interpretational analysis will be implemented through the use of manual procedures and the computer software program, Ethnograph. Also, interpretational analysis will be used when handling interview transcripts, field notes and document reviews. To provide for replication procedures and increase reliability, complete documentation of all data collection and analysis procedures will be created and coding checks will be performed by an external auditor. Also, a contact summary form with themes/aspects codes relevant to coding categories will be used to summarize all collection procedures. Example coding categories are: positive statements about PBS interventions, negative statements about PBS interventions, progress/status statements about PBS interventions and results statements about PBS interventions. To increase reliability of the qualitative methods and findings, additional steps will be implemented: all qualitative data will be examined by the field recorders and then by one additional team member, statements will be reviewed with interviewees to ensure completeness and accuracy, and following interviews, periodic checks will be conducted with a selection of school staff members to ensure findings are representative of the school sites from which they are obtained. Finally, all members of the research team will conduct and record self-reflective statements following each interview and data collection period to provide etic perspective as well as to identify potential researcher bias. __Ethics and Human relations__ (Cody)

A. What threats does the study pose for participants? Study participants will be required to complete and return signed parental/legal guardian consent forms. All parental/legal guardian informed consent forms will be collected at separate times from surveys to ensure strict confidentiality and anonymity of participants. For schools that fail to maintain PBS interventions there is a possibility that negative results can occur in students. This study’s plan will include sustainability measures such as incorporating school staff in the PBS planning phase, revising school ODRs to match PBS behavior expectations and establishing in-school staff members to act as part of the research team. These staff members will help review data, educate colleagues and serve as primary leaders for PBS intervention methodologies.

B. How to gain entry into research settings? HELP!? The schools selected for this study serve student populations that are representative of those who benefit most from PBS interventions (Warren, et al., 2003). High rates of disruptive behaviors have been shown to be detrimental to both students and staff; alternatively, very low achieving schools have been rewarded with state recognition for achievement following PBS interventions (Shore, 1997). PBS interventions are shown to have positive effects on achievement, student and staff efficacy as well as the overall school climate. This information will be presented to the primary stakeholders of the target schools with complete information related to the study and assurance that full support will be provided in creating, establishing and implementing effective, sustainable PBS interventions tailored specifically to the school(s) in question. Cody, we could even improvise and say one or two of the schools has asked for help in curbing the number of referrals written. That is what happened in one of the articles I read, so it would be possible! Then we could say we are going to the other high schools with this information and asking if we can conduct the survey. OR we can say one high school has a strong community/parent base that is involved and that is how we chose that school for that intervention, etc etc. --Kristina __Timeline__

I will have a look at it tomorrow and try to tie all this together as much as possible.- Cody
 * I am going to try to work on this tonight and tomorrow. If anyone has any free time (ha ha) and can throw some ideas my way, that would be great. I am beginning to reread chapter 6! Kristina**