Main Idea/Feature Selection

To start, we need to understand what data we are going to use. We will begin by selecting these factors from the report card data:

1. School Name
2. % Student Enrollmen - White
3. % Student Enrollment − Black or African American
4. % Student Enrollment − Hispanic or Latino
5. % Student Enrollment − Asian
6. % Student Enrollment − Low Income
7. Student Attendance Rate
8. High School Dropout Rate − Total
9. High School 4−Year Graduation Rate − Total
10. % Graduates enrolled in a Post secondary Institution within 12 months
11. # Student Enrollment

Then we will reduce our set of schools to high schools in Chicago. Our goal is to separate schools into bins with similar demographics and then use simple statistics to describe the different bins. We can achieve this by using what is known as K-means Clustering ([2], p. 460). Using this method, we can partition the schools into separate bins of similar demographics. In particular, we will be clustering schools based on factors 1) through 6). This means that each school in our clustering algorithm can be represented as a point in \(\mathbb{R}^6\). Later we will see how these clusters have an affect on academic “success”. In the context of this paper, we define academic success to be the factors 7) through 10).

Mathematical Setup for K-means Clustering

In the preliminary step of the K-means algorithm we randomly select K points \[\{p_1^{(0)}, p_2^{(0)},\dots, p_K^{(0)}\}\] to be the starting means or centroids. Let \(S \subset \mathbb{R}^6\) be the set of schools. With these random starting points, we will be able to construct a partition given by \[ \{ S_1^{(0)}, S_2^{(0)}, \dots, S_K^{(0)} \} \] where \[\bigcup^K_{i = 1} S_i^{(0)} = S\] and \[S_i^{(0)} = \{s\in S : d(s,p_i^{(0)}) < d(s,p_j^{(0)}) \; \forall \; j\neq i\}\] where the metric \(d:\mathbb{R}^6\times\mathbb{R}^6 \to \mathbb{R}\) is given by euclidean distance. The intuition behind the math here is that we drop \(K\) points in space that each have 6 randomly generated proportions for our school demographics. Then we partition schools into \(K\) subsets where the \(i^{th}\) subset, \(S_i^{(0)}\), is the collection of schools that was ``closest" to the \(i^{th}\) point, \(p_i^{(0)}\). The superscript \(^{(0)}\) denotes that this is the preliminary step. Moving forward, we recalculate our \(K\) points, \[\{p_1^{(1)}, p_2^{(1)},\dots,p_K^{(1)}\}\] where \[p_i^{(1)} = \frac{1}{|S_i^{(0)}|}\sum_{s\in S_i^{(0)}}s\] Which means that \(i^{th}\) point, \(p_i^{(1)}\), is now the mean demographics of the schools from \(i^{th}\) subset, \(S_i^{(0)}\), from the previous step. Then we create another partition given by \[\{S_1^{(1)}, S_2^{(1)},\dots,S_K^{(1)}\}\] using the same strategy as in the preliminary step. This algorithm will repeat in this fashion until the means converge to a point where the clusters don’t change on the next iteration. Thus we will have our final clusters given by \[\{S_1, S_2,...,S_K\}\] Once we have our clusters, we can now label each of the schools with the cluster it belongs in. For each cluster, \(S_i\), we can give summary statistics. We present the implementation with \(K=3\).

Results from K-means Clustering: 2019-2020

From figures 1, 2, and 3 that cluster 0 was predominately black or African American, cluster 1 was predominately Hispanic, and cluster 2 was a mixed demographic cluster with a high amount of non-low-income students. These results, intuitively, give rise to some powerful concerns. The important aspect of the construction of these clusters is that the algorithm only used demographic factors to partition the schools yet was able to spot a group of schools with significantly higher academic ``success" in cluster 2. Now it must be understood that how we are defining academic success is restricted to only 4 pieces of data. In reality, basing an individual student’s academic success on these 4 factors would be preposterous. However, making a judgment about an entire school on these factors makes more sense.

Figure 1. Summary Stats of Cluster 0

Figure 2. Summary Stats of Cluster 1

Figure 3. Summary Stats of Cluster 2

These clusters allow us to analyze even more about the school system in Chicago. They allow us to see the level of segregation in Chicago and how that is woven into the school system. By plotting these clusters, we can create a visual of this.

Figure 4. Mapping clusters over Chicago neighborhoods

It is clear to see that not only do these clusters separate schools efficiently on the basis of academic success (without even trying to) they also seem to be an efficient tool in splitting up the areas of Chicago.

Mathematical Setup for Stable Matchings

The idea of a matching is a rigorously studied concept in discrete mathematics. However, the concept is rather simple. In the context of our problem, we have a bipartite graph as follows

Figure 5. Implementation of stable matching

The lines across are the edges of the graph and they represent an element of the matching. For example, what these lines represent for this problem is that Cluster 1 is most like Cluster 3 and so on. These edges on the graph were determined by the similarity preferences. How can we be sure that such a matching exists? In addition, how can we find such a matching.

In 1962, David Gale and Lloyd Shapley proved that for any set of preferences on a bipartite graph there exists a stable matching ([3], Theorem 2.1.4). It is also worth noting that in 1984 Alvin E. Roth observed that the same algorithm had been used for practical purposes in the early 1950s. The algorithm’s history dates back to matching medical students to hospitals for their residency. Each hospital and student had a list of preferences so how do we make all parties happy? The algorithm is presented below

Initialize all \(a\in A\) and \(b \in B\) to be free
while \(\exists\) free cluster \(a\) who has a cluster \(b\) to match to, do:
    \(b:=\) first clusters on \(a\)’s list to whom \(a\) has not yet matched to
    if \(\exists\) some pair \((a^\prime, b)\) then:
        if \(b\) prefers \(a\) to \(a^\prime\) then:
            \(a^\prime\) becomes free
            \((a,b)\) become matched
        end if
    else:
      \((a,b)\) become matched
    end if
 repeat

First of all, what is a stable matching? In the context of the medical students, say Hospital A wanted medical student B and vice versa. However, in the matching, medical student B was matched to Hospital B and Hospital A was matched with medical student A. This creates an unstable edge and thus the matching would be unstable. A stable matching is a matching where this situation does not occur. It is worth noting that even in a stable matching a hospital or medical student may not be matched with its preferred choice. However, that choice will not prefer it more than what it is already currently matched to.

Result of Cluster Matching

One factor we are interested in viewing is the proportion of low income students over time. From figure 6, we can see that the gap between clusters is immediate. Cluster 2 has significantly less low income students as compared to clusters 0 and 1. Not only this but the gap seems to be getting larger each year. Cluster 0 went from 91 to 85 percent low income students and cluster 1 went from 95 to 89 percent low income students while cluster 2 went from 68 to 55 percent low income students. This relative change is 3 times larger than in clusters 0 and 1. What this means is that the K-means algorithm is identifying that the schools with wealthy student bodies are getting wealthier over time.

Figure 6. Proportion of low income students over time

Now we will turn our attention to the college enrollment rate of each cluster. From figure 7, like in figure 6, the differences are immediate. Cluster 2 has a much higher proportion of students enrolling in college compared to the other clusters. We can also see how the COVID-19 pandemic has affected schools. Over time we have seen more graduates move on to postsecondary institutions but in the 2019-2020 school year this growth comes to a screeching halt. More specifically, we can see the effect that the pandemic has on schools that are predominantly black or African-American students with cluster 0 having the largest drop in postsecondary institution enrollment.