Thursday, April 16, 2015

Hey, Stupid, We Suck at Math, and it's Your Time to Shine

A Statistics Lesson

Let's say you're the dictator of a small island.  You've got 10000 healthy males.  To aid in building your army, you hold an annual shooting competition to determine who enters your royal guard.  Using a complex algorithm, your scribes assign every entrant a shooting score between 0-100 based on a normal distribution curve.  You select the top 50, and are delighted that you can field a royal guard where everyone has scores in the 90s.

Two years later, a deadly plague has swept the island.  You only have 5000 healthy males.  When you hold your annual shooting competition this time around, your scribes use the same algorithm to allocate scores between 0-100 on a normal distribution curve.  Now, your royal guard is - gasp - taking people with 85s!

When asked to explain, your royal statistician concludes the following:

A) The best shooters were less interested in serving in the royal guard
B) Because of the results of the competition, the royal guard is in peril
C) It's worrisome that the number of scores in the 50s isn't falling as fast as the scores in the 90s
D) The 85s now aren't as good as the 90s from 2 years ago.

Do you:

1) Agree with everything he says; it makes intuitive sense! or
2) Roast the moron over hot coals and cook a suckling pig over his charred ashes

If you chose door number (2), you're likely too sharp for the utensils in this drawer.  Go breed a cancer pill or something.

But if you chose door number (1), you're potentially an expert at Law School Statistics!

Law School Statistics

Yes, Law School Statistics, the ever-expanding field of knowledge that presents claims without rudimentary skepticism for the too-far conclusions being made based on the observed data.  This week, we feature Jerry Organ and the nonsense surrounding LSAT scores.
Fewer people with high Law School Admission Test scores are applying to and enrolling in law school, and less-qualified students are filling their slots, new research shows.
Of course, the real problem is that fewer people are applying to law school, period (or, that law school seats aren't declining as fast as applications), which means schools are having to take more marginal candidates from the left side of the bell curve.  But in Law School Statistics-land, we have no problem assuming that a smaller population means a less qualified or interested population at the higher end.  There's really no support for this provided, but fuck it, it's research!
The disenchantment with law school on the part of the people most likely to get in shows up in the classroom head count, too: Around 5,400 people with the highest scores will enroll in law school this year, down from 9,400 in 2010, according to Jerome Organ, the University of St. Thomas School of Law professor who parsed the numbers.
You mean if you cut the population in half, the amount of members above x percentile is virtually cut in half, too?  Did you peer review that?  The actual story here is that the matriculation rate for high scorers is actually increasing, which means the smartest people in the group taking the LSAT are actually more interested in law school.  And "most likely to get in" seems a bit odd, since you're more likely to get in with a smaller population in competition for seats.  But, shit, we've got more alarming research to give you!
“The top is eroding and the bottom is growing,” says Organ, adding that schools risk churning out graduates with less of a shot at becoming lawyers.
Actually, the bottom (meaning 120s and 130s) is almost certainly shrinking as rapidly as the 170s are.  But of course, in Law School Statistics-land, "bottom" means 150, and "growing" means that law schools are shifting the acceptance curve leftward.  In fact, the second chart on this article clearly shows that the "bottom" isn't growing at all; it's shrinking rapidly in the aggregate.  But the yield rate for that section of the curve has skyrocketed, which is what shows law schools' desperation to find warm bodies.

LSAT scores tell us almost nothing about the applicant pool except that it's smaller.  You could just as easily conclude that the dumbest people are opting out of law school because sub-150 applications are way down.  Sub-140 applications are likely dropping much faster than 145-155 applications.

Anyone with a rudimentary understanding of normal distributions could have predicted this exact result with any commensurate drop in total applications without a similarly-radical drop in the number of law student spots.  But in Law School Statistics-land, the drop in LSATs can lead to alarming conclusions.

There may be a brain drain going on.  But it ain't in these numbers.

Lemming Opportunity

Why does this statistical particularity matter?

Is it because if the law schools can't get this basic stats 101 stuff right how can you believe anything that they say?  Well, yeah...

But also because lemming, baby, you've got an opportunity as green as Glen Ross Farms.  See, the real problem here is that too many law schools are still taking in too many students.  Are smart people really opting out of Yale and Harvard?  No, of course not, you fuckwit, and they know that.  But the purpose of these articles is three-fold.

First, the headline is attention-grabbing, good old fashioned scare-journalism

Second, it places causal responsibility on individual actors (students) rather than institutional actors (law schools taking in more students than the market can bear).

Third - and this is key - it relies on assuming a constant amount of selections.  Consider a world where law schools limit enrollment to 25,000 students.  Suddenly the LSAT-based "brain drain" would be instantly reversed, like magic!  Instead, we start with the assumption that the law schools' amount of seats is valid and a constant that is useful in determining year-over-year comparisons of LSAT scores to determine whether a "brain drain" is going on.

So, Lemming, the obvious approach is simple.  As long as law schools refuse to contract the amount of seats available, your mediocrity has never been more coveted.  You'd practically be dumb not to take the LSAT and get on their scholarship target grid.

But here's a more advanced one.  Get everyone to take the LSAT at least twice.  Make a pact that on the first time around, everyone bombs it on purpose.  This will allow everyone taking it on the 2nd and 3rd try to get around the median or above, while even some of the 1st timers may sneak up there by dumb luck.  Then repeat the process.  By loading the bottom scores on purpose, LSAT scores will rise, everyone will get scholarship money, and everyone will be on pace to pass the bar!  No more "brain drain!"

You might say, "but LSTC, that's basically a pyramid scheme" or "but won't schools pick up on the manipulation going on?" 

To you I say:  eat a dick and learn yourself some Law School Statistics.  These schools are so desperate for lemmings with a 151 LSAT they won't care whether you bought it on the black market with two kilos of coke, a truckload of Kord 12.7 machine guns, and Raphael's Portrait of a Young Man.

1 comment:

  1. Thanks for writing this. When I read your post, I am forced to think of class size of class, quality of class, and price as separate, distinct issues.

    This: "Second, it places causal responsibility on individual actors (students) rather than institutional actors (law schools taking in more students than the market can bear)."

    Also, this: "we start with the assumption that the law schools' amount of seats is valid and a constant..."

    In a fantasy in which every one of 40,000 LSAT-takers scores a 170+, the problems of lack of demand in the job market and price still exist.

    Clearly schools feel they have no duty to reduce their class sizes to be commensurate with, say, the number of graduates who landed in JD-required, bar-passage-required, full-time employment in the last 5 years - or any metric that reflects labor market conditions.

    On the other hand, if class size were reduced commensurate to labor market conditions, but all LSAT-takers scored 120, the price problem still exists and probably some graduates [there will be graduates, law schools would never fail out their entire class] will never pass the bar.

    Where do we go from here? What's the desirable policy?

    Policy is going to be very resistant to a duty on the part of schools to respond to labor market conditions by reducing class size.

    The most that is done is to mandate that schools publish consumer information, and allow students (and the government) to bear the risk. Schools are the only source of the relevant consumer information that is absolutely necessary for a prospective student to begin to understand risk.

    If there were truly reliable, fully explained, non-disputed data on the following, how many lemmings would jump?

    How many people are landing jobs as lawyers? How much do those jobs pay? Are the jobs open-ended employment contracts or time limited?

    From the perspective of a casual observer, I find the disconnect between BLS non-growth in the Legal Services Sector, and the aggregate number of graduates supposedly being placed in JD-required, bar-passage-required, "long-term" employment, jaw dropping.

    ReplyDelete