Professional Documents
Culture Documents
Developing An Early Warning System: Identifying Factors That May Predict High School Completion by 6th Grade Students
Developing An Early Warning System: Identifying Factors That May Predict High School Completion by 6th Grade Students
Prepared by
Prepared For
Executive Summary
Purpose
The primary purpose of this study was to identify specific academic and social factors
that may be used as “flags” to identify Duval County Public Schools (DCPS) 6th grade students
who are at high risk of not completing high school. A secondary purpose was to understand
how these factors may or may not be aligned with those identified by Neal, Balfanz, and other
researchers (Neal and Balfanz, 2006).
Four questions guided our 6th grade analysis: (1) Are significant numbers of students
showing the signs of disengagement previously identified in our examination of 9th grade
students who fail to complete high school? (2) Can we identify a set of indicators that flag sixth
graders who have high odds of failing to complete high school? (3) From this set of indicators,
can we identify a practical set of indicators routinely collected and reported at the individual level
that flag sixth graders who have odds of failing to continue in school until graduation? (4) Can
we then identify a set of those indicators that may be influenced by school and community
programs?
Data
The data used in this analysis were provided by the DCPS Department of Instructional
Research and Accountability. The data were gathered from the district’s data warehouse and
provided to the research team in the form of several Excel spreadsheets. The data included
selected demographic, academic, attendance, and disciplinary information from each student
enrolled in grades 6–12 during the 2001–2002 through the 2007–2008 school years.
Approximately 60,000–65,000 students were enrolled in grades 6-12 during each school year
with each grade’s enrollment averaging between 10,000–12,000 students.
Since the focus of this analysis was 6th grade students, we began by identifying all 6th
grade students for the 2001–2002 school year and then tracking that cohort of students through
each school year through the 12th grade (2007-2008). Ten thousand ninety-three (10,093)
students remained for inclusion in the analysis after the elimination of withdrawals, transfers,
and incomplete data sets.
Early Warning System 3
Once the data set had been cleaned of cases with missing data or incorrectly coded
data, an initial factor analysis was conducted to determine whether variables fell into coherent
categories (i.e., factors) as predictors of completion. No one piece of data, such as attendance
or mathematics standardized test scores, has enough weight to serve as a valid predictor.
Therefore, this process of combining various data sets was repeated until a consistent set of
categories was identified. Once a coherent set of predictors was identified, linear regression
(stepwise) and logistic regression analyses were conducted on each identified factor in order to
estimate the strength of each factor.
Results
A significant number of 6th grade students displayed the signs of disengagement, such
as poor academic achievement and excessive absences, previously identified in our study of 9th
grade students and in studies conducted by other researchers. Five factors proved to be of the
greatest value in identifying members of the 2001-2002 6th grade cohort who did not graduate
on-time from high school in 2007-2008. Each flag was a statistically significant flag (p< .05)
even after controlling for the other flags. The five flags listed below, individually and in
combination, identified 81.52% of the non-graduates.
Standardized Math Test Scores: 69% of the 6th grade students who failed to graduate
on-time scored below Level 3 on the Florida Comprehensive Assessment Test.
Standardized Reading Test Scores: 56% of the 6th grade students who failed to graduate
on-time scored below Level 3 on the Florida Comprehensive Assessment Test.
Failing Course Grades in Mathematics: 30.2% of the 6th grade students who failed to
graduate on-time failed a math course.
Failing Course Grades in English/Language Arts: 28.1% of the 6th grade students who
failed to graduate on-time failed an English/Language Arts course.
Excessive Absences: 12.2% of the 6th grade students who failed to graduate on-time
missed more the 20% of the scheduled school days.
The number of Class 3 and 4 Code of Conduct violations occurring at the 6th grade among
the study population was not very large (3%). As a result, this flag only identified 6.7% of the 6th
grader who failed to graduate on-time. However, our tracking of students indicated that the
number of Class 3 and 4 violations increase significantly as students get older and as more
Early Warning System 4
students become over-age for their assigned grade. Consequently, the conduct violations from
subsequent years have more predictive power than those from Grade 6. While the yield for this
factor was relatively small as a single factor and no greater when considered in combination
with test scores or grades, we nevertheless consider this to be an important flag to monitor as it
becomes a key indicator in later grades.
2. We can, with a reasonable degree of validity, identify a set of indicators that flag sixth
graders who have high odds of failing to complete high school.
3. From this set of flags, we can identify a practical set of indicators routinely collected and
reported at the individual level that flag sixth graders who have odds of failing to
continue in school until graduation.
4. These indicators are factors that may be impacted by school and community programs.
In conclusion, as we have learned from our previous studies of first-time 9th grade
students and focus groups with students who had previously dropped out of school, students do
not graduate from high school for a multitude of reasons. Many of those reasons are not linked
directly to academic achievement, attendance, and behavior at school. However, our research
supports the work by Balfanz (2006) and others (Balfanz, Hertzog, Neild, and MacIver, (2007))
that there is ample, readily available data for early identification of students with the greatest risk
of failing to graduate high school.
Early Warning System 5
Table of Contents
Page
INTRODUCTION ……………………………………………………………………………...… 7
BACKGROUND……………………………………………………………………………..…... 8
Guiding Questions
Outcome Variable
Data Source
FINDINGS………………………………………………………………………………………... 17
Attendance
Course Grades
Discipline
Status Variables
SUMMARY……………………………………………………………………………..………… 26
References…….…………………………………………………………………………………. 30
Table 3. Double Warning Flag Combinations and Counts for all Duval 6th Grade 19
Students in 2001–2002 Who Failed to Graduate in 2007-2008
Early Warning System 7
Introduction
Jacksonville Florida (CF) designed to examine issues related to Duval County Public School
(DCPS) students who fail to graduate from high school. The first study, Where Have All the 9th
Graders Gone? (Wilburn, Weaver, and Wilburn, 2008), was a descriptive study of a cohort of
first-time 9th grade students. The study was designed to provide the local community with a
more detailed understanding of the non-completion problem in Jacksonville. The second study,
Kids Say the Darndest Things (Roush and Wilburn, 2008), was a qualitative analysis of focus
group data which had been collected by The Community Foundation and their community
partners in the Learning To Finish Project. These focus groups were conducted with 16–24
year old students who had been identified by the school district as being of high risk of dropping
out, currently enrolled in a school or community sponsored dropout prevention program, or had
dropped out of school and were incarcerated in the Duval County jail for a criminal offense. As
with the descriptive study, the primary goal of the effort was to gain a richer understanding of
students who have dropped out of the public school system. This study used the information
learned in the first two studies to develop an early warning system that could be used by the
community to identify the students, at the end of the 6th grade, who are the most at risk for
• Background. This section provides a review and summary of the 9th grade descriptive
and focus group studies and how the outcomes from those studies were used to inform
• Methods and Procedures. This section describes the methods and procedures used to
• Findings. This section reports the results from the predictive analysis and their
the study.
• Summary. The final section reviews the main points of the study.
Background
beginning the process of developing flags for identifying 6th grade students with a high
The first study was designed with the overall goal of developing a better understanding
of issues surrounding the graduation crisis and the dropout problem found in DCPS. Simply
stated, the design of the study was to follow three cohorts of students from the time they first
entered the 9th grade through their 4th or 5th years of high school. Using data provided by the
DCPS Department of Research and Accountability, we were able to select a study population
that met our criteria for inclusion in the analysis. As recommended by Balfanz (2007), we sought
to study cohorts of 9th grade students who had been enrolled in DCPS during the previous
grades, were enrolled in the 9th grade for the first time, and were engaged in an academic
program for which a standard high school diploma was a reasonable expectation. Additionally,
we targeted a study population that was reflective of the overall 9th grade population in gender,
Graduation and Dropout Rates. The analysis indicated that by the end of the 3rd year of
high school, approximately 488 students (2.5%) of the original first-time 9th grade cohort had
received their diploma. By the end of the 4th year, the number of students who had graduated
increased to 11,842 students (60.74%), and by the end of the 5th year, the students who had
received a standard high school diploma was 11,901 (61.04%). While approximately 61% of the
first-time 9th grade students eventually graduated, 95% of those did so in the customary 4th year
of high school. Only a small percentage (4.1%) of the students graduated early, and an even
smaller percentage (0.005%) graduated after a 5th year of high school. In addition to tracking
graduates and non-graduates, we also identified the number and characteristics of students
labeled “dropped out” (i.e., did not withdraw or transfer but just failed to show up the next school
year). Our calculations indicate that the number of students who dropped out increased with
each succeeding cohort. The 2002 cohort had an estimated dropout rate of 25.5%, followed by
29.5% for the 2003 cohort, and rising to a high of 31.3% with the 2004 cohort. When examined
as a whole, the estimated dropout rate for the combined three student cohorts was
approximately 28.2%. What this means in terms of students lost is that over the 4 years of high
school almost 2,000 students left each cohort without receiving diplomas.
In 2007-2008, under the direction of Dr. Shannon Perry, chair of the projects qualitative
focus group, seven focus groups met to explore the dropout situation in Jacksonville.
Information from these sessions were transcribed and provided for analysis to Dr. Connie
Roush, a qualitative research specialist with the Brooks College of Health at the University of
North Florida. In this report, qualitative themes were identified with highlighted responses from
each focus group and quotes provided to illustrate some of the major points. The major themes
identified were participants’ attitudes and beliefs about the dropout situation; the influence of the
family, school, and community environments; specific issues related to peer pressure; the
Early Warning System 10
culture of violence; the need to know and care for students; and accountability for school
attendance. Each group of participants approached the focus groups from a different point of
view or attitude shaped partially by their own beliefs about what spurs school disengagement
and dropout.
Major Themes. Each group reinforced the belief that getting to know a student takes
continuous contact over time and a lot of listening. This lays the groundwork for a trusting and
caring relationship necessary to (as one student said) “…attract us to school and keep us
there.” Listening was a topic most often addressed by the middle school and incarcerated
youth while all groups offered wisdom about the caring relationship and the time that it takes to
evolve. Another theme our analysis identified was the power of the need to “fit in” to school
culture and/or the “wrong crowd.” Whether it is with other students in school or with groups in
the neighborhood, peer pressure is extremely strong. A third theme identified was the “culture
of violence” that is so much a part of students’ lives. Many students described their
disengagement from school as being “sucked in” to using and selling drugs and the violent
lifestyle that goes with it. While the family emerged as a critical theme, it was seen by many
teens as equally as positive and negative as a factor. For many, the discussion centered around
the barriers put up by family to be supportive. In a smaller but equally powerful part of the
discussions, students spoke about the people who motivated and inspired them to stay in
school. In the course of reports on all previous themes, the influence of the school environment
on disengagement and dropout is evident. There was a great deal of concern by all
participating groups regarding the family’s communication with the school, conditions that
prompt children and teachers to stay at a school, and the need for a variety of educational
options. While all emphasized that the lines of communication must be open for parents,
students, and the school to work together, few could provide any positive examples. Most of the
examples were negative in nature describing the problems and consequences of poor
Early Warning System 11
communication. In regard to perceptions of the community, little time in the focus groups
examined this in particular. However, most comments conveyed the message that the
community was not aware and did not understand why students are leaving school and was not
doing anything to address the problem. During the focus groups, participants explored the issue
of school attendance from multiple perspectives. Students described “skipping school” and
being “kicked out” of class; parents were concerned about timely notification of absences; and
all participants discussed the need for accountability. One other topic of concern was the
enforcement of attendance policies and the potentially negative consequences for students with
special circumstances.
Taking Action. Students, parents, and teachers discussed the different programs that
can lead to graduation, the need to “graduate on time,” and the pros and cons of graduation
options (the usual high school diploma, the G.E.D. pathway, and the certificate of graduation).
They also discussed the experience of transferring to an alternative program (specifically the
Pathways program). All three groups agreed that the high school diploma was the most
desirable of the three. According to one teacher, the high school certificate has a major impact
on the community as well as the students because it limits their employment opportunities.
Another teacher stated that it basically indicates, “I went to school for 12 years but I did not earn
a high school diploma.” When comparing the G.E.D. to the diploma, one parent told her child,
“Some places won’t take the G.E.D., so it is better for you to get your high school diploma. It will
open more doors.” From the perspective of a student, programs to help students stay in school
“When we get funding we try to make the kid fit into the grant so we aren’t actually
working on what needs to be worked on. The people who are giving the money or
disbursing the money do not understand the population to which they are giving. Schools
Early Warning System 12
try to make the money fit in the programs but it doesn’t work” (Roush and Wilburn, 2008,
p. 17).
Guiding Questions
identified in our examination of 9th grade students who fail to complete high school?
2. Can we identify a set of indicators that flag sixth graders who have high odds of failing to
3. From this set of indicators, can we identify a practical set of indicators routinely collected
and reported at the individual level that flag sixth graders who have odds of failing to
4. Can we identify indicators within that set that may be influenced by school and
community programs?
Outcome Variable
The outcome variable used was whether or not the sixth graders in the 2001–2002
cohort graduated from high school on time or after one extra year. We chose this outcome, in
part, because of our desire to replicate the study reported by Balfanz, Herzog, Neild, and
MacIver (2007) which previously identified a set of sixth grade predictor variables. In addition,
we selected this variable because in the previous analysis of 9th grade students, the vast
majority of students who earn a diploma did so on time or within one additional year. Therefore,
extending the time frame would be of little significance. It is also important to note that we
Early Warning System 13
selected to use graduating from high school as opposed to dropped out of school since each
student who receives a standard high school diploma is clearly identified with a high degree of
accuracy in the district’s data system; however, students who fail to graduate can do so for a
multitude of reasons (e.g., transfers, withdrawal, failing to report to school) that are not captured
Data Source
As with previous studies, the data used in this analysis were provided by the DCPS
Department of Instructional Research and Accountability. The data were gathered from the
district’s data warehouse by a district staff member and provided to the research team in the
form of several Excel spreadsheets. The data included selected demographic, academic,
attendance, and disciplinary information from each student enrolled in grades 6–12 during the
Since the focus of this analysis was 6th grade students, we began by identifying all 6th
grade students for the 2001–2002 school year and then tracking that cohort of students through
each school year including 12th grade (2007–2008 school year). We created a longitudinal
dataset designed to follow the performance of students enrolled in Grade 6 during the 2001-
2002 school year. The dataset included attendance, demographic components, math and
reading course grades, and math and reading standardized test scores per year. These
students were traced through the 2007-2008 school year, the year during which students
normally complete Grade 12 and graduate or finish a program. Those still enrolled as DCPS
students at the end of the 2008 school year were categorized as either “Graduated with
In our data file 10,516 students were identified as being in Grade 6 for the 2001–2002
school year. From this initial population, 423 students were deleted due to incomplete data sets
which resulted in a data set of 10,093 students for our study population. This sample of
Asian-American (2.8%), and Multiracial (1.3 %) students. Slightly more students were male
(51.5%) than female (48.5%) and slightly more than eight percent (8.3%) of these sixth graders
required services as English Language Learners. For the purpose of this study, students in the
two most severe classifications of special education (those who were not expected to graduate
or to earn a certificate of completion) were eliminated from the sample. The remaining ESE
students (those who spent at least part of the school day in regular education classes) were
No data about free or reduced lunch eligibility were available for about half the students
in the Grade 6 population. Data for 5,015 of the students indicated that 94% qualified for free or
reduced lunch (about half of the total Grade 6 sample). Of the complete study population
(10,093), only about 1% was younger than expected in the Florida public schools, while 41.5%
were of “normal” age for the grade level and 58.4% were over the expected age indicating that
they had been retained at some earlier grade level. At the end of the 2008 school year, 66.7%
(6,733) of the students graduated by 2008 while 33.3% (3,360) did not.
Table 1 provides an overview of the study population in regard to the school related
characteristics between those 6th grade students who graduated from high school and those
who did not. For example, for those students who were enrolled in a dropout prevention
program, only 12.4% graduated high school while 87.6% of those who were not in the program
earned their diploma. Among students who committed a serious conduct violation, a much
larger percentage did not complete high school. There were also significant differences
between completers and non-completers on the Developmental Scale Scores of the Florida
Early Warning System 16
Comprehensive Assessment Test (FCAT). When compared over the four years students were
tracked, those who completed high school had higher reading scores each of the four years and
higher math scores two of the years. While the differences were not always great enough to be
significant, the final course grade point averages (GPA) in language arts and mathematics were
also higher for high school completers for each of the four years.
Once the data set had been cleaned of cases with missing data or incorrectly coded
data, an initial factor analysis was conducted on the 23 different variables available in the Duval
dataset to determine whether variables fell into coherent categories (i.e., factors) as predictors
of completion. One of the reasons for beginning with this data reduction procedure was to
determine if one piece of data, such as attendance or mathematics standardized test scores,
had enough weight to serve as a valid predictor or would combinations of variables serve as
stronger predictors. The process of combining various data sets was repeated until a consistent
set of categories was identified. Once a coherent set of predictors was identified, linear
regression (stepwise) analyses were conducted on each identified factor in order to estimate the
The factor analysis yielded eight variables strong enough to be included in the regression
converted to one of five individual levels in each student record, and the five levels were
2. Although the nature of the courses varied from student to student, end-of-sixth-grade
course grades in English and math were included. Letter grades were converted to
3. Students who had incurred serious infractions of the conduct code (major violations or
violations of zero tolerance policies) were identified and labeled dichotomously from
4. Absences during Grade 6 were calculated and included. No distinctions were made
among reasons for absence: a simple total of days missed was initially used. Based on
indications from previous studies, in subsequent analyses, absences were divided into
two categories: less than 20% of the school days in 2001-2002 and more than 20% of
5. Graduation status for each year up to and including the routine six years of high school
6. Participation in dropout prevention programs during Grade 6 only was included. Duval
Justice, and neglected or delinquent. Students were labeled by the specific program in
7. Demographic variables such as race, gender, and date of birth were included. Dates of
birth were subsequently converted to age in months and then divided into three
categories: under expected age, normal expected age, and over expected age.
8. The need for services as English Language Learners was identified and coded
Findings
As we worked through the identification procedure, it became apparent that the most
valuable warning flags from an educational standpoint would be those that could be impacted by
the school and/ or community based educational programs. These are (1) measures of
academic learning, (2) school attendance, (3) student behavior, and (4) program placement.
Consequently, some demographic variables (e.g., race, gender, primary language) were
Table 2 shows the yield of each of the eight preliminary predictors. Seven of these
independent flags met our test of being within the influence of the educational program and
accounting for at least 10% of the students who failed to graduate, but one did not.
(8) Receiving a serious (Code 3 or 4) disciplinary referral (This flag only identified 6.7% of
the non-graduates. It is possible that this flag has some value but failed to meet the two
pronged test because of the low number of students in the overall population who met
Early Warning System 19
One of the lessons learned from our qualitative analysis of interviews with students who
had dropped out of school was that often there was no single factor that prompted students to
leave school. As Balfanz has reported, “examining the occurrence of multiple flags and their
impact on students’ graduation chances provides additional insight into the process and impact
of student disengagement at the start of the middle grades” (Balfanz, 2007, p. 229). With this in
mind, in our next step we set out to determine how our predictors might be used in combinations
to better flag potential 6th grade non-graduates. Using the three strongest flags (i.e., being over
age, FCAT scores, and course grades) we reanalyzed the data to examine the power in
identifying non-graduating students when using two factors. These results are presented in
Table 3.
Table 3. Double Warning Flag Combinations and Counts for all Duval 6th Grade Students in
2001–2002 Who Failed to Graduate in 2007-2008
FCAT Scores
Early Warning System 20
Course Grades
Absence
Attendance
Our analysis confirmed the information previously reported by Balfanz (2007) that
attending school less than 80% of the time increases the chance that students will not complete
high school. In our study population, approximately 11% of the 6th grade students who failed to
graduate were absent from school for more than 36 days. However, as reported in our previous
study of 9th grade students (Wilburn and Weaver, 2007), attendance was not as strongly
associated with failing to graduate as academic factors. While useful, in combination with other
factors, independently attendance does not provide a meaningful yield as a predictor variable.
Early Warning System 21
Consistent with findings from other studies, academic achievement identified the highest
number of 6th grade students who failed to graduate on time (Balfanz & Boccanfuso, 2007;
Balfanz 2007). However, unlike other studies, in this case standardized test scores were the
most reliable predictor of failure to graduate. When used independently, having a low (Level 1
and Level 2) Florida Comprehensive Assessment Test score in Reading or Math accounted for
approximately 62% percent of the 6th grade students who failed to graduate. While being over-
age identified the highest number of future non-graduates among the 6th grade study population,
in most cases the reason that a student is overage is because that have been retained for failing
to make an acceptable score on the FCAT. Consequently, being over-age may simply be a
proxy for poor FCAT performance over time. In addition, it takes more than one year to be
Course Grades
In our work with the data provided by the school district for our descriptive study of 9th
grade graduation rates and this study, we found many inconsistencies in the course grade data
sets. In our efforts to identify early warning flags for 6th grade non-completers, we found the
data to be so inconsistent that we initially removed the course grade variable form our analysis.
However, because it is information that is readily available, we have included it in this analysis.
When taken independently, failing one 6th grade math or English course identified approximately
36.2% to 28.1%, respectively, of the students who eventually failed to complete high school.
Because of inconsistency in course grades from year to year, we believe that standardized test
scores provide the most valid and reliable predictor of high school completion for sixth grade
warning flag for identifying students. This point may be illustrated by the fact that course grade
Early Warning System 22
is the only academic factor that is stronger when combined with other factors. For example
when failing a math course is combined with the low FCAT math score factor, the number of
students identified increases from a math course alone yield of 30.2% to combined yield of
48.7%. A similar gain is also realized when the two factors of a failed English course and a low
FCAT reading score are combined. For this reason we do not recommend using course grades
as an independent predictor.
Discipline
The discipline codes used in this study were those that represented the most serious
(i.e., Class 3 and 4) offences that a student could receive. For example, students’ behaviors that
involve physical assault, weapons and the use and/or possession of illegal drugs fall into these
categories. In the vast majority of cases, when a student receives this type of disciplinary
referral, he/she is automatically suspended from school and/or removed from their home school
and assigned to one of the District’s alternative schools for a period of 45 to 90 days.
Consequently, the number of Class 3 and 4 Code of Conduct violations occurring at the 6th
grade was not very large. As a result, this flag did not identify a large number of the 6th graders
who eventually failed to complete high school. From our work in tracking the 6th grade student
as well as our previous analysis of 9th grade students who failed to graduate, we know that the
number of Class 3 and 4 violations increase significantly as students get older and as more
students become over-age for their assigned grade. Consequently, the conduct violations from
subsequent years have more predictive power than those from Grade 6. While the yield for this
factor was relatively small (6.7%) as a single factor and no greater when considered in
combination with test scores or grades, we nevertheless consider this to be an important flag to
monitor.
Being either in the District’s Dropout Prevention Program (DPP) or over-age for the 6th
grade significantly reduces the probability that a student will successfully complete high school.
Our analysis indicates that of the 858 6th grade students enrolled in one or more of the District’s
Drop Out Prevention Programs, 341 (40%) graduated from high school on time while 60% failed
to complete high school. Obviously, for a student to be placed in one of these programs as early
as the 6th grade, he/she exhibits a number of our warning flags such as poor attendance, low
FCAT scores, and poor course grades. Consequently, the probability for completing high school
is low (i.e., 4 out of 10) for this student group. While the value of this factor as an early warning
flag for identifying large numbers of 6th grade students is limited, 15% of the 6th graders who
failed to complete high school were in the DPP. We believe it would be useful to examine the
program as to a better understanding of what initiatives may be successful in working with high
risk students.
On the other hand, being over-age for sixth grade appears to account for a large number
on non-high school completers. In our study population, 72% of the 6th grade students who
were over-age failed to complete high school. While this initially appears to be a strong flag for
identifying those who will not graduate from high school, as indicated in Balfanz’s 2007 study,
this is primarily because a high percentage of over-age students scored at Level 1 or 2 on the
FCAT, failed a 6th grade math or English course, and attended less than 80% of the time. The
few over-age students who did not exhibit any of the other flags tend to graduate at the same
rate as other 6th grade students. Consequently, being over-age is not the issue.
Our final action in the identification of variables (i.e., predictive flags) was to narrow the
number of flags to those that independently or in combination with other flags provided the
strongest practical tool for early identification of 6th grade students who had the highest
Early Warning System 24
probability of not graduating from high school. As discussed above, some of the variables, such
as being over-age and/or enrolled in a dropout prevention program, were actually products of
other variables such as poor academic achievement or attendance. While a case could be
made for including the discipline code flag in the final selection as it seems to be such a reliable
predictor in later grades, it also was very duplicative in that over 90% of the 6th grade students
who had a Class 3 or 4 code of conduct violation also had one of the other flags such as poor
attendance or failing course grades. Therefore, we limited our final selection of flags to five
By using these five factors and eliminating duplicate records, we were able to identify
In order to estimate the validity of our final five flags while controlling for the other flags,
we conducted a multivariate logistic regression to estimate the predictive power of each flag.
The analysis showed that, all else being equal, 6th grade students who missed more than 20%
of school attendance days were 6.07 times more likely not to graduate than students who
missed less than 20% of school attendance days (confidence interval (CI) = 5.03 to 7.32).
Students who failed English/Language Arts in the 6th grade were 2.86 times more likely not to
Early Warning System 25
graduate than students who passed Language Arts (CI = 2.15 to 3.17). Sixth grade students
who failed a math course were 2.38 times more likely not to graduate than those who passed
their math courses (CI = 2.15 to 2.62). Those 6th grade students who scored below 3 on the
FCAT reading test were 1.62 times more likely not to graduate than students who scored 3 or
above (CI 1.49 to 1.77). Finally, those 6th grade students who scored below 3 on the FCAT
math test were 1.71 times more likely not to graduate than those who scored 3 or above (CI =
1.56 – 1.88). Each flag was a statistically significant flag (p< .05) even after controlling for the
other flags. Our five flags, individually and in combination, identified 81.52% of the non-
graduates. Among the 10,093 total study population, 49% (4,975) graduated with none of the
five flags while 17% (1,706) had no flags but still did not graduate.
It is important to recognize that, as with all investigations of this type, these numbers
provide a description of the 2001-2002 6th graders who had not graduated by the end of 2008.
In the course of our data analysis, we conducted both linear and logistic regressions with the
common wisdom being that a logistic regression is the better test to measure predictors on a
dichotomous variable, resulting in the graduation variable being yes/no on graduation. Though
none of the 4,975 students in our 10,093 member 6th grade cohort had any of the five flags and
82% of the non-graduates had one or more of the flags, it is not valid to state that the
possibilities, not actual counts. This is evidenced by the fact that 1,706 of the 6th graders had
no flag but had not graduated by the end of the 2008 school year.
In generalizing our results to other populations, it is also important to note that we did not
control for race in the regressions that we ran. Since our 6th grade population was divided so
evenly between Black and White, there seemed no need to do so. It is common practice to only
control for such a factor if there are large imbalances inside the group. Consequently, caution
Early Warning System 26
should be exercised when applying our results to populations that have racial imbalances within
Summary
Purpose
The primary purpose of this study was to identify specific academic and social factors
that may be used as “flags” to identify DCPS 6th grade students who are at high risk of not
completing high school. A secondary purpose was to understand how these factors may or may
not be aligned with those identified by Neal, Balfanz, and other researchers (Neal and Balfanz,
2006).
Data
The data used in this analysis were provided by the DCPS Department of Instructional
Research and Accountability. The data were gathered from the district’s data warehouse and
provided to the research team in the form of several Excel spreadsheets. The data included
selected demographic, academic, attendance, and disciplinary information from each student
enrolled in grades 6–12 during the 2001–2002 through the 2007–2008 school years.
Approximately 60,000–65,000 students were enrolled in grades 6-12 during each school year
Since the focus of this analysis was 6th grade students, we began by identifying all 6th grade
students for the 2001–2002 school year and then tracking that cohort of students through each
school year through the 12th grade (2007-2008). Ten thousand ninety-three (10,093) students
Early Warning System 27
remained for inclusion in the analysis after the elimination of withdrawals, transfers, and
Once the data set had been cleaned of cases with missing data or incorrectly coded
data, an initial factor analysis was conducted to determine whether variables fell into coherent
categories (i.e., factors) as predictors of completion. No one piece of data, such as attendance
or mathematics standardized test scores, has enough weight to serve as a valid predictor.
Therefore, this process of combining various data sets was repeated until a consistent set of
categories was identified. Once a coherent set of predictors was identified, linear regression
(stepwise) and logistic regression analyses were conducted on each identified factor in order to
identified in our examination of 9th grade students who fail to complete high school?
2. Can we identify a set of indicators that flag sixth graders who have high odds of failing
3. From this set of indicators, can we identify a practical set of indicators routinely collected
and reported at the individual level that flag sixth graders who have odds of failing to
4. Can we identify indicators within that set that may be influenced by school and
community programs?
Results
A significant number of 6th grade students displayed the signs of disengagement, such
as poor academic achievement and excessive absences, previously identified in our study of 9th
Early Warning System 29
grade students and in studies conducted by other researchers. Five factors proved to be of
greatest value in identifying members of the 2001-2002 6th grade cohort who did not graduate
Standardized Math Test Scores: 69% of the 6th grade students who failed to graduate
Standardized Reading Test Scores: 56% of the 6th grade students who failed to graduate
Failing Course Grades in Mathematics: 30.2% of the 6th grade students who failed to
Failing Course Grades in English/Language Arts: 28.1% of the 6th grade students who
Excessive Absences: 12.2% of the 6th grade students who failed to graduate on-time
The number of Class 3 and 4 Code of Conduct violations occurring at the 6th grade among
the study population was not very large (3%). As a result, this flag only identified 6.7% of the 6th
grader who failed to graduate on-time. However, our tracking of students indicated that the
number of Class 3 and 4 violations increase significantly as students get older and as more
students become over-age for their assigned grade. Consequently, the conduct violations from
subsequent years have more predictive power than those from Grade 6. While the yield for this
factor was relatively small, as a single factor and no greater when considered in combination
with test scores or grades, we nevertheless consider this to be an important flag to monitor.
identified in our examination of 9th grade students who fail to complete high school.
Early Warning System 30
2. We can, with a reasonable degree of validity, identify a set of indicators that flag sixth
3. From this set of flags, we can identify a practical set of indicators routinely collected and
reported at the individual level that flag sixth graders who have odds of failing to
4. These indicators are factors that may be impacted by school and community programs.
In conclusion, as we have learned from our previous studies of first-time 9th grade
students and focus groups with students who had previously dropped out of school, students do
not graduate from high school for a multitude of reasons; and many of those reasons are not
linked directly to academic achievement, attendance, and behavior at school. Even though the
issues is complex and no one single factor or group of factors can account for every child’s
decision to leave school, we believe that it would be unconscionable to ignore the early warning
References
Balfanz, R., Hertzog, L., Neild, R. C. and Mac Iver, D. J. (2007). Preventing student
223-235.
Collaborative.
Roush, C., Wilburn, K. T. and Weaver, D.M. (January, 2009). Learning to finish focus group
StatSoft, Inc. (2007). Electronic Statistics Textbook. Tulsa, OK: StatSoft. WEB:
http://www.statsoft.com/textbook/stathome.html.
Wilburn, K. T. and Weaver, D. M. (November, 2008) Where Have All the 9th Graders Gone? A
Descriptive Study of Three First-Time 9th Grade Student Cohorts. Jacksonville, Florida:
Appendix