Thread by @cvangeyn

I've arrived at the Ontario Court of Appeal for the hearing today in the appeal of a case called Petrucci v Ontario. It is an appeal of a Divisional Court decision that found legislation requiring teachers to pass a mandatory math proficiency test was unjustified discrimination

gov: different universities have different standards and requirements for teacher candidates.

gov: whether you have a test or a course, there will be variation on pass rates

gov: gov's policy choice in imposing this test is entitled to deference

gov: the test is proportionate. The practical impact of the math test were signiciant. The objective of the test is to improve student achievement. Even within early stage the math test was achieving this by having math avoidant teacher candidates taking steps to improve their skills

gov: in response to the math test, some universities, like Lakehead, offered a new math course for skill improvement and had more teacher candidates enrolling in math courses. These candidates are now better prepared to teach math to students

gov: the math test prompted universities to provide resources for teacher candidates to improve their math skills. Even outside of new courses. Brock and lakehead offered virtual sessions, for example. Ottawa had workshops and courses for test prep.

gov: windsor offered math tutorials. York provided support through peer mentoring. Teachers organizations helped students improve proficiency too.

gov: teachers federation offered a weekly online program through zoom to ask questions about the math test and get help from a teacher

gov: the gov also provided practice exams online

gov: this was available online for candidates for free. The law society makes you purchase prep materials and free questions are limited. And if you fail the LSO test you have to buy those prep materials again

gov: div court erred in finding the math test would have a negative impact on the diversity in the teaching profession. This cant be concluded. The most deleterious you could say is that there are candidates who may have to take the test again. The evidence is success improves with retakes. You cant conclude those who didnt pass the exam could never pass the exam and would be barred from the profession

gov:all teacher candidates who get their degree are capable of passing this math test

gov: the goal is for all teacher candidates have what they need to pass, regardless of their starting point

gov: the time a teacher candidate spends preparing for the test is a benefit

gov: any cost associated in passing the test is small, in the form of time, and is small compared to what costs they already incur to get their teacher certification

court: is there a charge?
gov: maybe down the line for retakes but we dont know when or how much

(charge for retakes, that is)

now the intervener the CCF is up, George Avraam for 15 minutes

CCF: we focus in on what arbitrariness means under second step of the s 15 test.

CCF: there has been caselaw all over the map on whether arbitrariness is at s 15 or s 1, and Sharma settles that

CCF: we outline some criteria we hope helps court determine what is arbitrary

CCF: first we will consider how arbitrariness has been applied, and then some factors under s 1 minimal impairment can also be considered

CCF: debate on when arbitrariness is considered has been settled.

CCF: in Fraser, court in para 80

CCF: then pulls up SCC Sharma. Court there identified under arbitrariness the test is applied at step 2 of the s 15 test. Historically the court has assessed arbitrariness in s 15 framework. As early as Andrews

CCF: it hasnt always been consistent. sometimes applied at step 1 of 15. Sometimes at step 2. Sharma settles it

CCF: some factors help in assessing arbitrariness factors in s 15 analysis. Did you look at alternatives. Did you accomodate broadly members of impacted groups. First assess if decision was arbitrary. If not arbitrary, no discrimination, even if there is a distinction

CCF: it doesnt have to come down to s 1

CCF: our position is arbitrariness is a factor the court can and should look at under s 15 and the factors provided in factum (and crown relies on) are things court should look at.

Court: without getting into the revised numbers, how do numbers help us on arbitrariness? The numbers on the test score first time, those numbers are on their face suggesting a difference. Does that help us consider if this is an arbitrary distinction?
CCF: I can speak broadly. If youre looking at a disparity in numbers, and gov has done nothing else in assessing impact, bias, accomodation, cost, timing, thats what you look at. Pure numbers may help finding a distinction. But if youre looking at arbitrariness you must look at much more than numbers.

That wraps up CCF's submissions, we are now on lunch break until 2 pm

We are back now with counsel for the candidates. Starting with their submissions on the test for s 15, as well as responding to the gov's arguments on s 15. As a preview, candidates key points are:
1. failure on first test of the test IS the disproportionate impact. There is a disproportionate impact on multiple attempts, but even failing the first attempt is a disproportionate impact.
court: if you're wrong do you lose?
candidates: no, because there are multiple points with disparate impact
2. the all attempt data still shows a disparate impact for racialized communitiies

candidateS: when the gov says the data shows racialized candidates are being excluded from the profession, that isnt true on the facts
court: it doesnt show anyone is being excluded from the profession
candidates: lets start there then

candidates: gov has relied the incomplete data set the div court relied on, and then full data set after all attempts. If you look at rate of success for all demographic groups, 2 groups stand out: black and indigenous. I take your point from a statistical perspective the indigenous numbers are low. But the passing rate for white candidates is 97%. For every 10 candidates, 3 fail
Court: no, for every 100...
candidates: oh, yes.

candidates: the failure rate for black candidates is higher (note: I missed what it is - i think he said 90%). There is a slow accretion of black candidates
court: you assume these numbers are final. They're quite clearly not. The candidates can try again.
candidates: look at the raw numbers.
Court: but those who attempted more than once, only 12 failed. 333 passed. It's over 90%. There's another 25 who only took it once, and what we know that if you take it more than once your success rate goes up.

court: there is good reason to believe if we can persuade those 25 who took it once, to retake it, some significant number would pass. Isnt that right?
candidates: I dont take issue with the premise. But this ignores the key point that the actual fact they have to take the test again is disparate impact
Court: we're not there yet in the argument though. These aren't final numbers for success rate on the tests.

candidates: there is a practical reality here. The gov says people can retake an unlimited number of times. In a practical sense thats not a good answer. The test is administered 3 times a year and the bachelor takes 2 years. That's 6 times
Court: no people could write 8 times and there is evidence of that
candidates: that's an anomoly and beside the point. There comes a point for racialized candidates who have to keep retaking it, and they see their white classmates passing, thats the discrimination. They have to give up jobs, time with family, to prepare for the test. There are job opportunities that go to someone else because theyve passed the test and you havent. Yes they can keep trying, but what the test is doing is putting a hurdle in those candidates way that is higher than for white candidates. And while maybe they can get over it eventually, it's a larger hurdle than for their white candidates
Court: even if the pass rate is 97%? Even for racialized candidates?
candidates: yes. And it is human nature to eventually just give up rather than keep trying and failing.

candidates: when you parse our the data by race, you see the rates of abandonment are higher for all racialized groups than for the white group

candidates: for white candidates the rate of abandonment is just over 2%. Just over 2% of white candidates who started the test quit.

Court: there's no information for these black candidates on why 24 of them decided not to take it again or if they really were not going to take it again. We dont know if they were going to take it again
candidates: the SCC has said the burden on claims for discrimination should not be unduly onerous
Court: but you need evidence. We dont know whether these people didn't want to try at some other point to retake the test
candidates: If we look at the difference in the numbers, that point may land if the numbers were very low. But on the evidence the numbers are so far apart in terms of percentage of the different demographic groups quiting, we can draw the inference there is something effecting black candidates
Court: but 90 white candidates didnt retake the test. Why arent we concerned with them
candidates: but as a proportion they are very small. There was a much larger starting group.
Court: but it seems like the sample size for black candidates (6%) is very small. Youre taking that small sample group and extrapolating it for a whole community. Your ability to take a sample size and apply it depends on if that sample size is responsible. If you're just polling say, 10 Canadians, no one thinks that reflects what Canadians think. You need to look at the sample size
candidates: this isnt a sample. This is the actualy data of the people who wrote the test and answered the demographic survey. There's no extrapolation needed. It's not a statistical significance problem. This is the real raw data.

Court: it's such a small group there could be all kinds of idiosyncratic reasons why people may have quit. Someone moved, someone got sick
candidates: but it's still higher than the white candidates
Court: if you jjust take a few out of the group who quit, you get a very different quit rate

candidates: Im having trouble understanding the court's concern with these numbers
Court: because it's got nothing to do with discrimination

Court: isnt the assumption of a statistical analysis that it cant be explained by one-offs? You've got like 24 people here. You have to come to grips with that there could be idiosynchratic reasons to explain the differences

candidates lawyer pulls up Fraser.

candidates: if the issue is quality with statistical data, from an expert perspective, that's in the government's hands to put that into this record. If the gov wants to say there is something unreliable about this data they need to put that in and explain why we can't extrapolate
Court: no theyre just saying these stats dont get you where the div court went

Candidates: SCC said the burden should be low on the claimants. Claimants should not have to undertake massive demographic studies to make claims of discrimination, especially for low population groups like indigenous canadians

Candidates: I want to refocus on the failure rate at first attempt. This is a more important factor for us.

Candidates: im confused why the gov says the div court relied on preliminary data. I think they may mean incomplete. Because there is nothing hesitant about this data, its just the first half of the administration of the test

Candidates: we can see the racial disparity is much wider on first attempt. Im not sure we can make qualitative arguments about reliability of data. Black candidates pass at 74%, indigenous at 68%, white at 97%. These difference form the basis of disproportionate impact.

Court: question at div court was not whether first attempt vs multiple attempts was different. It was about whether the math test would impact the diversity of the profession.

Court: And that focused on final attempts. After 3 attempts, for example. None of the experts put forward the theory you have that we only need to look at the first attempt. All the studies cited by the experts say we only look at the first attempt. We arent looking at how many times the test was taken, we were looking at the ultimate makeup of the profession. There is no evidence that suggests first attempt is what we evaluate the test on.
Candidates: there isnt expert evidence in record talking about initial vs subsequent attempts. But it does talk about the impact of these types of barriers on the teaching profession

Candidates: these tests slowly erode diversity in the teaching profession. Maybe 90% to 97%. I dont think the legal test requires the claimant to wait that process out in order to prove discrimination. And I will note this argument about first attempts vs subsequent, this is a new argument by the gov. The Div Court considered it and rejected it.

Candidates: thrust of div court decision is the fact that racialized candidates hae to write multiuple tests is itself a burden the white candidates dont bear
Court: but those who succeed still become teachers. The retakers who succeed aren't barred from becoming professions, so it doesnt impact diversity of profession
Candidates: as it becomes known there are systemic barriers to getting certified as a teacher, and even if the court sees them as small, they add up. And it discourages racialized candidates from attempting to become teachers.

Candidates: and lack of diversity in the profession means racialized students do more poorly. It has knock on results. It's systemic, not just the statistics you have on the page in front of you

Candidates: standardized testing reduces diversity across the board.

Candidates: in the step 1 part of s 15, this is the first time the gov has characterized the test as a diagnostic tool. If it were, it would not have final consequences. The test is a "high stakes" test, even if the gov resists this test. All the literature calls it that, because its either pass or fail

Candidates: gov is saying unless math test can be shown to admit fewer teachers of colour into profession, it doesnt have disproportionate effect. Thats doing what SCC warned against in Sharma. The gov is blending steps 1 and 2 of s 15. Step 1 is to act if one group is treated differently by another, it doesnt ask magnitude of difference
Court: Div court said first question is doest math test have a disproprtionate impact on diversity of profession. Are you not satisfied by that characterization?
Candidates: there is more too it than that.
Court: Has the div court framed the first part of s 15 test correctly by talking about entry into the profession, regardless of # of times test is taken?
Candidates: I think they have, in that passage at div court they were looking at partial data that had stronger disparities. My submission is when you look at full data, the disparity is still there. Div Court was alive that barriers created by math test is a disproportionate impact in and of itself.

Candidates: step 1 just asks if the conditions between 2 groups is different. Step 2 asks if that difference is caused by the government action

Candidates: div court ruled fewer racialized teachers would become teachers if the test is in place. Its harder for racialized candidates to pass than for white candidates.

Candidates: div court also talks about the failure of the government to follow up on those candidates who abandoned the test. Who never retook it.

Candidates: The fact the gov has not followed up causes concern, because of higher failure rate of racialized candidates, according to Div Court
Court: did Div Court say how that should be done? Do they want the gov to send the authorities to knock on peoples doors and ask why they didnt take the test again? Its hard to ask gov to do something when you dont say how. How are they supposed to follow up, given privacy rights? These people many would not answer a questionairre on their background.
Candidates: What you are hitting on is the flaw in the model of teacher assessment and skill development. The gov has said the math test value is it encourages you to improve your skills and by the time you reach the barrier your are ready. But we say that method will always result in discriminatory effects. The record says if the province wants teachers to be better at math the real method is to ensure bachelor of education programs gives teacher candidates the courses they need to learn math
Court: but wouldnt that course have tests?
Candidates: yes but its not a faceless test, in a classroom setting there is closer interaction. Thats why the experts advocated for course work not a math test
Court: but there is nothing stopping people from taking those courses anyway

Candidates: if the gov wants to legislate the solution they can legislate it as part of the curriculum for the program . That would be an alternative
Court: but the gov declined because they didnt want to interfere with the autonomy of the teacher colleges. And experience suggests universities resist that.

Candidates: on one hand it could ruffle some feathers about what is required in the curriculum. On the other it is hollowing out the diversity of the profession. I dont see how those are equal options.

Candidates counsel now discussing one particular canidate. There is lots to be gained from looking at his expereince (will call him Mr N)

Candidates: Mr N has 2 masters in sociaology and a PhD. He deals with sociological stats, hes from cameroon but studied all over and speaks many languages. Mr N has skills that will benefit the ontario education system, but it took him 3 tries to pass the math test.

Candidates: If you look at his results, keeping in mind the test has both a math skill and pedagogy component... only 14% of the math pedagogy component has to do with math.... On skills he got above passing grade. But he failed pedagogy by 4% the first time.

Second time he failed pedagogy again, this time by a worse time. 61% on second attempt on pedagogy. But passed math again.

On third try, he passed with 71% in the pedagogy.

Someone like Mr N, it indicates a lack of math skills.

Court: no, it's pedagogy. You can have a PhD in math but if you cant communicate with a grade 6 student its no good. He knew the math, but based on this he didnt know how to teach it. Its a different thing, and very important.

Candidates: the gov says this is an objective test of grade 3, 6 and 9 math. And reviewed for bias. Here Mr N passed math with flying colours, but he failed pedagogy
Court: are you saying it doesnt matter if he doesnt know how to teach math?
Candidates: my point is its not a math test
Court: no, it also is about teaching math
Candidates: but only 14% of the pedagogy questions are about math. Its otherwise general pedagogy
Court: is that not imprtant?
Candidates: my point is the test is not a test of math knowledge. The problem is also language. We have a confluence of problems.
Court: He took 2 tests a week a part. And then within 5 weeks took it 3 times. And was successful. We dont know what he did to improve, but presumably he figured it out. He did something, it wasnt random, to improve his skills. You said they should take courses to improve skills, but he did something here to improve his skills. Why isnt that a good thing?
Candidates: this ignores his circumstances. He has 3 sons. His wife is studying to be a nurse. He has a part time job. Why should he keep studying when his white classmate passes the first time? He passed, because he's smart, but he had to jump through additional hoops. How many others are out there like him, who will be discouraged from continuing to their third attempt.

Candidates: we have statistics that show black candidates abandon their chosen career path at a higher rate.
Court: we dont have evidence of that. All your evidence is that a small number chose not to retake the test at a particular point in time.

Candidates: The court is entitled to infer, in the limited universe of the math test when it existed, a greater rate of candidates who didnt retake, the court is entitled to infer at least some of them simply made that choice just to give up.

Candidates: the fact that the numbers are so different across demographics suggests something is happening here

Court: the final success rate for French speakers vs English it is almost the exact same.
Candidates: Dont get hung up on this
Court: well youve said it was unfair to Mr N because he speaks french
Candidates: I take your point overall, but the data shows those who failed on pedagogy were more likely to fail
Court: were the French speakers taking it in English?
Candidates: No, they were taking it in French. But there are a constellation of factors that can be discriminatory. That are not responsive to the personal characteristics of individuals.

Court: i dont know where you are getting this link that French speakers were less successful in pedagogy, the difference is 95% vs 97%. There is no meaningful difference
Candidates: This is on all atempts not first attempts.

Candidates: the test was biased. It was written by people who did not consider particular characteristics of minorities.
Court: this is like Alice in Wonderland. What are they supposed to do? Check with people who speak 200 languages?
Candidates: they must do more than nothing
Court: but the test is in english. If you dont speak or read english you wont do well.
Candidates: the government must meet people where they are and take into account the situations of people impacted by the test
Court: a black guy walks in the test and doesnt speak english. what do you do with the test so he's treated fairly?
Candidates: we can assume there must be some competence in English. Its not an onerous requirement that they speak english or french. I regret bringing up the language thing because everyone's getting hung up on it. My point is the province had fair warning the test would be discriminatory. The EQAO literature review is clear that all standardized tests bear the risk of discrimanatory tests. Given that, the gov was obligated to do something more than nothing

Candidates: when gov says they reviewed the math test for bias, its just a bald assertion. They didnt consider the richer review from EQAO and no questions were removed
Court: didnt div court find the EQAO already reviewed for bias
Candidates: they say they did
Court: no, but the Div Court found they did in fact
Candidates: theyre taking the affidavit from the evidence at its word as a sworn statement. The div accepted it. But on a s 1 Charter analysis, gov owes a burden more than just saying they did it. That's not enough to meet the burden on s 1.

Candidates: what we have here is a test that on the facts show a disproportionate effect on either first or subsequent attempts that has a disproportionate effect. This has the effect of perpetuating disadvantage. It takes people already disadvantaged, and heaps on an additional burden. On s 1, the gov is obligated to put in some kind of evidence about why it chose this measure over others. And given the measure they chose, they did everything they could to reduce discriminatory effects. And evidence shows multuiple other options could have achieved the govs goal without discriminatory effects. Prov has put in no evidence of review of math test to ensure it was minimally impairing of s 15 rights.

Teacher candidates co-counsel now up to address fresh evidence motion and to respond to arguments by CCF

Im not tweeting the arguments on fresh evidence but will summarize the teacher candidates arguments responding to @CDNConstFound

@CDNConstFound Candidates: CCF set out a list of nonexhaustive factors to consider gov responsiveness to arbitrariness.

Candidates: while these questions should be considered, it would be unduly burdensome to require them under s 15 because it would require claimaints to lead evidence of government consultation.. It cannot be logical to place such a burden on claimaints and that's not consistent with the development of s 15 case law

Candidates: if these questions or factors are to be incorporated, it should be under s. 1. The CCF also said gov pointed to factors that may be considered accomodations. The gov has discussed and identified these factors under s 1 and they should not be considered accomodations. Accomodations must limit discrimination, otherwise what value are they

@CDNConstFound Candidates: one factor is the ability to re-write tests. This is the discrimination, not an accomodation. Forcing candidates to retake the test multiple times does not alleviate the burdens of the test, including time, stress and stigma

@CDNConstFound Candidates: this is the very thing that needs to be accomodated, and doesnt constitute accomodation itself

@CDNConstFound Now up will be 2 interveners, the Ontario Human rights tribunal and the OSSTF

@CDNConstFound OSSTF making submissions about why teachers should have to take these tests if they are not required to teach math. Court interjects to say no one in this case is saying teachers must teach HIGHSCHOOL math - this case is about elementary school.

@CDNConstFound OSSTF says the gov has misstated things to say all teachers must know how to teach math, although it is true teaching math could be required for lower teachers.

@CDNConstFound Court: you should have applied to be a part of the proceedings below. This is not the role of the intervener. This is a whole new argument. You have 1 minute.

@CDNConstFound OSSTF: it must be through mutual agreement for a teacher to teach outside their qualification.

@CDNConstFound OSSTF: this circumstance - on rational connection - the gov did not adequately look at the system when they created this test.

@CDNConstFound Now gov is back for reply

Before gov up, court asks if Div Court overreached by finding unconstitutional the portion of the teachers act that allows for the creation of a math test. Teachers candidates says they do not think this was overreach
Court: can gov put any test into the statute?
Candidates: not a discriminatory one
Court: that's not what Im asking.

Candidates: my submission is, it is so difficult for any kind of high stakes standardized test to avoid discrimination it is unconstitutinal for the gov to impose that
Court: your own experts dont even say that
Court: and didnt you previously acknowledge that a test could hypothetically be possible and not discriminatory?
Candidates: yes hypothetically, but there are knock on effects
Court: ok thank you

@CDNConstFound Now gov up for reply. Going to be clarifying a few things.

@CDNConstFound gov: candidates say all step 1 of s 15 requires is a difference. No, that's not right, the claimaint must prove it causes a widening of a gap.

@CDNConstFound gov: the candidates have said the burden is on us to show the test is not discriminatory. No, it is trite law that the burden is on the claimant.

court: in terms of evidentiary burden... there is significant evidence that this kind of testing carries with it a discriminatory potential. Is that fair?
gov: the evidence there is largely about testing in the US and in the US there has been an resulting impact on diversity on teaching in the US. Because of that literature, the EQAO did additional vetting of testing questions
Court: so you wont agree with proposition that in the literature and among experts this kind of testing carries a significant risk of discrimination?
gov: im being careful about how you characterize it as "this kind of testing"
court: black and white pass or fail. Weve all done tests like that
gov: the literature is not evidence in the proceeding, there are experts. Ontario's evidence is in the area of math there is positive correlation between student achievement and teacher testing. I agree in the US there has been an experience of testing resulting in less diversity, but I resist the characterization that it is this kind of test. And the fresh evidence bears that out

court: so your answer is Ontario acknowledged this risk and thats why they undertook a review for bias.
gov: yes
court: so lets just consider a 70%/90% difference. Doesnt that satisfy the prima facie case the plaintiff has to make?
gov: it doesnt satisfy step 1 of s 15 because it doesnt show the math test caused or contributed to the difference. It doesnt satisfy step 2 because it doesnt look at it within the regulatory context and all the steps taken to prevent an adverse impact on diversity in the profession

@CDNConstFound court: everyone seems to recognize there is a risk of discrimination. Here it is white / black 90/70. at the initial test. Is that enough?
gov: no, because the difficulty is the evidence on success rates doesnt tell you anything about impact on diversity in the profession

@CDNConstFound court reserves judgment and we are adjourned!

@UnrollHelper

@CDNConstFound @UnrollThread