A Florida-based AI firm has released some very interesting revelations about lawyers and their rates of prevailing in lawsuits. The firm, Premonition, has determined that lawyers — even those on ‘top lawyer lists’, or considered by peers to be among the best — generally have average results. The company also discovered that lawyers and law firms don’t actually keep track of win/loss data: they mostly track fees and billing proportions. One indicative statistic kind of jumps out as a condemnation of the industry:

In a study of the United Kingdom Court of Appeals, it found a slight negative correlation of -0.1 between win rates and re-hiring rates, i.e. a barrister 20% better than their peers was actually 2% less likely to be re-hired.

The industry is so self blind that it actually penalizes lawyers that perform better.

Toby Unwin, the co-founder of Premonition says that the only thing that correlates with win rate in court cases is a history of winning. ‘The only item that affects the likely outcome of a case is the attorney’s prior win rate, preferably for that case type before that judge,’ he stated in the company’s press release.

The Premonition system is a web crawler that applies AI to scrape court results so that win/loss data for lawyers can be analyzed. This is a function of many factors, such as the relationship between the lawyer and the judge. This is like the example in Moneyball — the story about Billy Beane’s Oakland A’s, where the team analyzed how batters fared against specific other pitchers, or how frequently they got on base in general — except Premonition is analyzing how lawyers do when in front of specific judges and specific kinds of litigation.

Most interesting is that Premonition states that they have found no correlation between win rate and billing rate. The best results appear to come from small firms and soloists that they call ‘strip mall superstars’.

I make no claim about this technology, but what I find interesting is that the lack of transparency that makes it basically impossible for the average person or business to make informed judgments about which litigator to hire for a court case. It seems obvious that AI/analytic tools like Premonition should be of immense value, and therefore the marketplace for such tools should lead to a Moneyball like disruption in the field of law. This is also a condemnation for the larger law firms, who aren’t apparently tracking their lawyers in a sensible way relative to what matters to their clients.