Genius Interjected: What the Hell is this NELS Stuff? (Part V)

We continue our exploration of Simkovic & McIntyre's Economic Value.  You could call this a special summer serial, but it's actually just an LSTC that's too lazy to come up with stories in the deadest month of the year.

By this point, we have established that SIPP is a poor method through which to obtain samples of the JD-holding population in the present for useful evaluation.  We are not exactly novel in pointing some of these things out.

But genius is unflappable.  When it has one clunky government survey that's designed for an entirely different purpose and insufficient for the conclusions at hand, genius doubles down and finds a totally unrelated clunky second one from which to draw different, related conclusions.

In Economic Value Simkovic & McIntyre suddenly invoke the National Educational and Longitudinal Study ("NELS") of '88.  What the Dept. of Ed did with NELS is start with a survey of ~25k 8th graders (or thereabouts) in 1987-88 and then it used a variety of methods to follow them up to the year 2000.

Note the obvious limitations here:

• It appears to be one age cohort who would have started graduating law school circa 1999, and therefore entirely non-representative of the population in SIPP, or in reality absent some assumptions that don't work.
• When one reads the associated guides and materials, it becomes apparent it's not even a truly representative sample of the population of students for that age group.
• The sources of information in the survey are not even consistent.
• Using this argument/analysis is basically an admission that inputs matter significantly in the process and that the idea of a unified "premium" for law school is inane.
• It seems to ignore that the designation of majors/fields of study is inherently corrupted by the presence of post-graduate options.  A student who might otherwise study engineering and be fully capable to completing the engineering curriculum (leading to engineer money) may opt to major in liberal arts and study business or law; grouping such a student with the terminal bachelor's liberal arts students is disingenuous.  In fact, the primary study that almost certainly inspired using longitudinal studies and evaluating "ability sorting" (http://public.econ.duke.edu/~psarcidi/arcidimetrics.pdf, cited in Economic Value) focused on people who graduated in the 1970s and last reported earnings in 1986 and ignores the role of graduate school, which is likely more forgivable for students from the early 70s than students who went to school in the late 90s.

Nonetheless, Simkovic & McIntyre seemingly use these survey results - from literally one window in 1990s time - to draw relatively sweeping conclusions about who goes to law school and what factors influence it, including what college majors they have, whether they went to private school, and a slew of other variables, and how those various factors change future earnings.

To some extent, then, Economic Value - whose authors wanted to look at the lifetime JD premium and who stress that first-year salaries are not reflective - seem to suddenly use "terminal" earnings reports from 27-year olds (the last panel of NELS was done in 2000) to calculate differences in future earnings among various factors.  So there's no accounting for the teachers with 30 service years and a fat pension or the restauranteur who toils for a decade and then opens their own successful bar, no consideration that these folks follow that mythical, exponential career arc that eventually shoots $35k new lawyers to the economic heavens.

They then used those numbers - from a not-necessarily-representative sample derived from one segment of their target population - to compare it to the regression results from their SIPP data to conclude that the "ability sorting" variables they could identify in the NELS were not significant in changing their SIPP-based conclusions.

In other words, they used NELS (again, an unrelated survey from one cohort) to rule out a variety of alternative theories about the "premium" identified in the SIPP pool coming from sources or characteristics other than a JD - a time span that covers like 50 years of JDs and evaluates nothing at all after 2009.

Given that NELS is limited to one age group, its use to make any relevant conclusions at all about their other non-representative sample pooled from SIPP is stacking bullshit on top of bullshit and coating it with a thick layer of bullshit.

In other words, it's art.  When a mediocre artist or scholar has something discardable and unusable, they throw it away.  But Simkovic & McIntyre refused to sweep their deformed scholarly child into the trash.  Instead, like many of postmodernism's fine artists faced with trashy art, they made trash itself art.  Likewise, piling junk statistics onto junk statistics creates a masterpiece of legal scholarship.