Can institutional arrangements improve student performance?

In 2007, a series of reforms were implemented in the education system in Ceará, Brazil. This research aims to explore data on the education quality index to find out if it suggests these reforms had an impact on student achievement. In order to do so, panel data is analyzed through spatial plots, time series and scatter plots.


Around 2007, the government of Ceará, a state in the northeastern part of Brazil, started a series of reforms in its education system. These reforms comprised five main pillars: fiscal incentives to municipalities based on outcomes; technical assistance for municipalities; more autonomy and accountability for local systems; regular monitoring of learning in order to collect actionable data, and elimination of political influence in the choice of principals and teachers. In the analysis that follows, I employ data collected by the Brazilian government concerning the quality of education (IDEB - Índice de Desenvolvimento da Educação Básica) and data that might impact student outcomes, like socioeconomic level of the school/region. The objective is to describe how the education quality evolved through time, get insights about the data and associations between the variables.


The data used in this analysis is made available by the Ministry of Education of Brazil and the Brazilian Institute of Geography and Statistics. Bellow, there is a representation of how data sets connect to each other. In order to tidy data, I employed the function **gather()** that creates a data frame in which each observation has its own row. In order to join data sets, I employed **left_join()** using the keys highlighted in the scheme bellow.

Distribution of the education index

As a first analysis, I explore the distribution of education quality index in 2005, before intervention, for Ceará and for Brazil (excluding Ceará). In order to do that, I use density plots. Observing the graph bellow, as expected, Brazil has greater standard deviations from the mean because it is composed by 26 different states. Ceará has a much smaller variance in its distribution. The mean results of Ceará are lower compared to the Brazilian average. The distributions are centered around 3.

Below, the distribution of indexes in 2019 can be seen (after intervention). Both indexes improved considerably in the period. Now the distributions are centered around 5. The average index of Ceará, however, improved more compared to the average in Brazil. The probability of finding an index higher than 7, for example, is a lot higher in Ceará data.

OLS: Ceará vs. Brazil

Here we can observe the results of a linear model that shows the association between being the state of Ceará and the quality of education index both in 2005 and in 2009. We can observe that initially, in 2005, being located in Ceará meant a decrease of -0.35 in the performance of a school. In 2019, this association changed direction and increased considerably: being located in Ceará is now associated with a 0.61 increase in the education index. If we control for human development index (HDI), being a school in Ceará in 2019 is associated with an increase of 0.89 in the quality of education compared to the rest of the country. This means that when we compare schools with similar HDIs, there is a stronger positive association of being located in Ceará and the education quality index. We cannot interpret, however, that the reforms in Ceará caused a 0.89 increase in the quality of education, because there are more confounders affecting this relation.

Table 1 - OLS Model: Education Quality Index per group.
Dependent variable:
Education Quality Index
Note:p<0.1; ⋆⋆p<0.05; ⋆⋆⋆p<0.01

Spatial Data: Ceará vs. Brazil

The following maps show the geographical area of Brazil colored according to the average education quality index of each municipality. Blue is associated with a good index and red with a poor one. We can tell that the country had an overall improvement from 2005 to 2019 both for primary and lower secondary school. Ceará is highlighted at the top right of the map and it is visible that this state has improved more compared to other states, specially its neighbors.

Time Series: Ceará vs. Brazil

In this section, I explore how the education quality index evolved through time before and after the intervention in Ceará. The averages of Ceará are compared to the average index of all other Brazilian states (except Ceará). Before intervention, the trends follow a parallel development although the curves are not perfectly parallel. Unfortunatelly there are only two observation before intervention: 2005 and 2007.

Education Quality vs. socioeconomic level

In this section, I analyze the education quality index as a function of the socioeconomic level of the school for Ceará and for the rest of Brazil. What can be seen is that schools in Ceará have a lower socioeconomic level, but a higher education index compared to the rest of the country. The best primary and lower secondary school in 2019 are located in Ceará.

NAs analysis

Here I present the proportions of missing values (NA) relative to the number of observations for each state regarding the primary schools. Some states have more than 50% of missing values. It appears that poorer states have a bigger proportion of NAs. In the second table, the same information is presented, but regarding the Lower Secondary Schools. Again, it appears that poorer states have a bigger proportion of NAs.

NAs in Primary Schools: 2005 - 2019
State Number of Observations Number of NAs Percentage of NAs
1 DF 3280 738 22.50
2 PR 23120 5550 24.01
3 MS 6480 1610 24.85
4 SP 58232 15993 27.46
5 MG 46216 13350 28.89
6 ES 8504 2525 29.69
7 SC 18456 5783 31.33
8 RJ 28712 9531 33.20
9 GO 15288 5129 33.55
10 RS 31712 10734 33.85
11 MT 9304 3322 35.71
12 RO 5120 1830 35.74
13 AC 2544 983 38.64
14 TO 5944 2345 39.45
15 RN 11032 4452 40.36
16 AM 11072 4594 41.49
17 AP 2552 1085 42.52
18 PE 24696 10548 42.71
19 AL 11152 4837 43.37
20 CE 27968 12236 43.75
21 SE 7760 3450 44.46
22 PB 14280 6475 45.34
23 PA 28368 12979 45.75
24 BA 48984 23334 47.64
25 MA 30048 15114 50.30
26 PI 14464 7541 52.14
27 RR 1816 994 54.74

NAs in Lower Secondary Schools: 2005 - 2019
State Number of Observations Number of NAs Percentage of NAs
1 PR 15320 3689 24.08
2 MG 35744 9408 26.32
3 SP 46392 12837 27.67
4 SC 14224 4072 28.63
5 GO 11480 3578 31.17
6 RJ 18864 6019 31.91
7 ES 6864 2211 32.21
8 MS 5224 1738 33.27
9 PE 15360 5447 35.46
10 DF 1784 633 35.48
11 AL 5888 2289 38.88
12 RO 3864 1507 39.00
13 TO 4360 1710 39.22
14 RS 27504 10988 39.95
15 SE 4648 1872 40.28
16 MT 7936 3286 41.41
17 PB 9096 3808 41.86
18 RN 7232 3206 44.33
19 BA 29656 13364 45.06
20 AP 1464 660 45.08
21 CE 24504 11154 45.52
22 AC 1800 874 48.56
23 AM 8368 4192 50.10
24 PI 10824 5477 50.60
25 PA 18088 9386 51.89
26 RR 1256 666 53.03
27 MA 23576 12661 53.70


As just presented, the reforms in Ceará appear to have had a positive impact in Ceará. The spatial data shows a blue island of good education indexes in the treated state. The time series show a much more inclined line of increase for Ceará, compared to Brazil. Estimating the true impact is, however, more complex because we must account for any source of endogeneity that might be affecting both treatment assignment and the outcome of interest. Predictors of education quality index should also be balanced between control and treatment group. One possible approach to continue this investigation is to apply a fixed effect model. It is also important to investigate whether NAs are associated to poor performing municipalities and how NAs develop before and after intervention. To sum up, this was an informative descriptive analysis of the reforms in the education system of Ceará and a first step to analyze its impacts in student achievement.