LLM4Law - Legal LLM Benchmarks

Legal Task Benchmarks

Model	Version	Contract Analysis	Case Prediction	Legal Research	Document Drafting	Overall
Llama 2 Meta	7B-chat	78%	65%	52%	82%	72%
Mistral Mistral AI	7B-v0.1	85%	78%	68%	88%	80%
GPT4All Nomic AI	Falcon-7B	72%	70%	58%	80%	70%

Carefully curated legal tasks representing real-world scenarios including contract analysis, case prediction, legal research, and document drafting.

Each task is evaluated on accuracy, legal reasoning, citation quality, and practical applicability by a panel of IP attorneys.

All models tested on identical hardware (RTX 4090, 64GB RAM) with standardized prompts and temperature settings for fair comparison.

Added comprehensive comparison between the two leading 7B parameter models on contract analysis tasks.

Posted: June 15, 2023

Refined our evaluation criteria for legal research tasks to better assess citation accuracy and relevance.

Posted: May 28, 2023

View All Updates →