When launching a new product and in particular, when that product is disrupting an incumbents rather cosy world, then pushback is inevitable indeed, if you don’t get pushback from the incumbents then you’re doing something wrong.

And so it is with Premonition, litigation analytics and the conclusions we’re able to draw from building the world’s largest litigation database.

The question of how we calculate “win rates” is a regular one and often delivered within an attempt to dismiss litigation analytics model and their future role within the legal profession.

Occasionally we’re forced to dismiss the clearly silly objection.  During one meeting in the UK we’re asked “how do you know about matters that haven’t gone to trial?” to which the answer was simple, “we can’t, because we are data scientists, not magicians”

But there are legitimate questions about our methodology and how the algorithm works.  Whilst members of the magic circle are prohibited from revealing their magic secrets, here at Premonition we’re a more open bunch and always happy to disclose how we arrive at “win rates”;

Determining Wins

The first and most important point to make, is that Premonition data is not an absolute, it’s indicative. Other factors such as lawyer availability, price, personal chemistry, etc. should always affect final selection but, for an industry that works around reputation and word-of-mouth, performance metrics is a powerful tool to add to the mix.

At the heart of our methodology is the determination of an attorney’s “win rate” and what constitutes a win is often hotly debated.

If you call both lawyers on a closed case, both will claim to have “won”.

When launching Premonition, we consulted with lawyers and judges asking them about the general winners and losers for various dispositions (we have over 100 types in the system). There was broad agreement that the fairest approach is to consider judgements as a “win” for plaintiffs, dismissals as a “win” for defendants and settlements should be categorized consistent with local court rules, which generally categorize a settlement as a “dismissal.”

While the system is not perfect, it creates a “level playing field” for which reasoned judgments can be made of a group of attorneys operating in the same judicial environment.

Thus, comparisons between local attorneys operating on the same types of cases in the same jurisdiction can be more “data based” and less subjective.

This is the ultimate goal of Premonition.  Premonition is not a “rocket science” system where we strive for 100% unarguable analysis of every case; rather its function is to spot trends and outliers.

As we’re adding >40,000 cases every day in the US alone, the power of big data becomes apparent as data is “smoothed out” over many 1,000s of cases.

Case Difficulty

We also understand that every case differs in difficulty, so we don’t try to assess the difficulty of a case.

Some attorneys claim to “only take the toughest cases” when explaining their low win rates. Occasionally this is true. However, our data indicates that as an overall excuse this is a false foundation as it too is “smoothed out” over a large number of cases.

It is also worth noting that in some jurisdictions such “cherry picking” is prohibited. For example, the United Kingdom has a “cab rank rule” preventing Barristers from cherry picking easy cases and our results are similar to the United States.

While Premonition is not designed to identify the “surgeon” attorneys that only work the hardest cases, it does group cases with those of similar difficulty by using case type, sub type and parties. For example, the Tax Court is notorious for its low win rates against the IRS, however top performing attorneys still stand out, not by their wins, but because they lose less.

Cases grouped by court type (Appeals Court being a more rigorous black letter law venue than small claims), and by Judge where certain cases, e.g. complex business claims are assigned to certain Judges. Over time, case difficulty variations are also smoothed out over large numbers of cases and Attorneys who only take easy cases will de-select themselves when approached.

Multiple Attorney and Firm Only Cases

Premonition looks at the first listed attorney for each side and ignores the others. This not only makes the system easier to use but it also recognizes the practical reality that the lead attorney usually has the greatest impact on the result.  In courts where the individual attorney names per party are not identified specifically, Premonition uses an algorithm to predict which individual attorney likely represents which party, then selects the 1st one for each side to analyze. However, co-counsel can be analyzed if specifically requested by the client.

Does it help the buyer of legal services?

Ultimately we’re driven by one thing, does our data on lawyer performance put the consumer of legal services in a better position than they were before Premonition?

Is it better for the consumer and the market, that lawyer selection is driven by metrics, rather than peer review?

By developing a Proprietary Algorithm for determining lawyer win rates, we provide the consumer with unparalleled insight into lawyer performance, so that as part of their selection process, they can now judge metrics, as well as recommendations.

We’re able to shine a unique spotlight onto which lawyers, win which cases and in front of which judges, leaving even the most jaundiced of our legal friends, hard pressed to argue against the need and benefits of transparency.