Open-Source vs Open-Weight LLMs: The OSI Test

Open-source and open-weight LLMs are not the same thing. Open weight means you can download the trained parameters. Open source AI, under the Open Source AI Definition (OSAID) 1.0, also requires the freedoms to use, study, modify, and share the system, backed by the code and data information needed to make meaningful changes.

That distinction is not academic. It decides whether your team can reproduce the system, redistribute a derivative, pass a license review, or move away from the original vendor.

Our position: if a release gives you Apache-2.0 weights but withholds the training code and data information, call it permissively licensed open weight. Calling the whole AI system open source overstates what was released.

This article focuses on that disputed line. For the shorter definition, see the open-source LLM glossary. For software licenses rather than model releases, use the open-source software guide.

What Is the Difference Between Open Source and Open Weight?

The cleanest way to compare model releases is to separate access to weights from openness of the full system.

The middle row causes most confusion. A permissive license can make the weights easy to use while the system still falls short of OSAID because its training pipeline is not available.

Open weight is still useful. It can enable self-hosting, fine-tuning, quantization, and vendor independence. It simply answers a different question from “is the full AI system open source?”

What Does the Open Source AI Definition Require?

OSAID applies the open-source idea to an AI system rather than treating a model as one file. It asks whether a recipient can exercise four freedoms and has the preferred form for making modifications.

The Four Freedoms

A user must be able to:

1. Use the system for any purpose without asking permission.
2. Study how it works and inspect its components.
3. Modify the system, including its outputs and behavior.
4. Share the original or a modified version for any purpose.

A field-of-use restriction breaks the first freedom. A revenue threshold that requires a separate deal can break the freedom to use or share without permission.

The Three Artifact Groups

The preferred form for modification includes:

• Parameters: weights, configuration, tokenizers, and other material needed to run or change the model.
• Code: complete training, data-processing, validation, testing, and inference code under OSI-approved licenses.
• Data Information: provenance, scope, acquisition, selection, labeling, processing, and filtering details, plus where public or third-party data can be obtained.

The OSAID FAQ is explicit that raw training data does not always need to be redistributed. What must be available is enough information and code for a skilled team to build a substantially equivalent system.

That compromise is contested. Some free-software advocates argue that modification is not meaningful without the exact dataset. The practical point remains: OSAID asks for much more than downloadable weights.

Why Is an Apache-2.0 Weight File Not Enough?

Apache-2.0 and MIT are excellent software licenses. They grant broad copyright permissions, and Apache-2.0 includes an express patent grant. But a license only governs the artifact to which it applies.

If a model card marks its weights as Apache-2.0 while the training data, preprocessing code, and full training pipeline remain undisclosed, the license does not make those missing artifacts appear.

OSI does not certify individual models. Treat “OSAID-compliant” lists as an assessment, not an official seal.

OLMo also shows why “everything is Apache” is too simple. Its Dolma corpus has an ODC-By license and source-specific terms. A transparent stack can still contain multiple licenses that legal and data teams must review.

How Can You Audit a Model in 60 Seconds?

Use this four-check audit before you read benchmarks or start a deployment.

Check 1: Open the Actual License

Do not stop at a marketing page or a repository badge. Open the LICENSE file and the model card for the exact checkpoint.

An SPDX identifier is a machine-readable name, not proof that a license is OSI-approved. The SPDX License List marks OSI approval separately and also includes many non-OSI licenses.

Check 2: Map the Released Artifacts

Write down whether the release includes:

• weights and configuration;
• inference code;
• training and evaluation code;
• data-processing code;
• data information or a dataset card;
• training recipe, checkpoints, and logs.

Do not infer a missing artifact from the license on another one.

Check 3: Search for Conditions

Look for these terms inside the license:

• acceptable use;
• monthly active users;
• annual or monthly revenue;
• non-commercial or research-only;
• derivative naming and branding;
• redistribution and notice requirements;
• patent or copyright litigation termination.

Conditions do not automatically make a license bad. They tell you which label and review path are honest.

Check 4: Record the Version You Approved

Save the model ID, commit hash, model-card revision, license text, and review date. Model families mix licenses, and vendor terms can change between releases.

The decision record should say what your team approved, not “Qwen is open” or “Llama is free.”

Which Model Licenses Need Extra Review?

Custom model licenses often permit broad use while adding one or two conditions that matter only at scale. Those conditions still keep the release outside a strict OSAID label.

OpenMDW illustrates another subtlety. Its definition covers artifacts provided under the agreement; it does not promise that every artifact needed to rebuild a model was provided. License breadth and release completeness are separate checks.

What Changes When You Self-Host an Open-Weight LLM?

Downloading a model changes who owns the operational work.

You Gain Control

You can pin a checkpoint, keep prompts inside your environment, tune the model, and move between serving providers. For regulated or offline workloads, that control can decide the architecture.

You Take On the Infrastructure

There is no per-token API bill, but GPUs, batching, autoscaling, upgrades, and incident response are now yours. At low or irregular volume, a hosted API can still be cheaper.

Serving engines such as vLLM make deployment easier, but they do not remove capacity planning or model-specific compatibility work.

You Still Need a Quality Gate

The model card was not measured on your customer traffic. Quantization, prompts, tool schemas, and retrieval context can all change the result.

Openness improves inspection and portability. It does not guarantee factuality, safety, latency, structured output, or task completion.

How Should You Compare Candidate Models?

Use a fixed evaluation set before moving a model into production.

1. Collect 25–100 real inputs that represent normal cases and costly failures.
2. Define pass criteria for task accuracy, grounding, tool calls, and output format.
3. Run every candidate with the same prompt, temperature, context, and serving precision.
4. Measure quality with latency, GPU memory, and cost rather than ranking quality alone.
5. Review the failures by cohort before choosing a winner.

For a UI-based workflow, FutureAGI’s Evaluate platform lets a team manage datasets, evaluators, and eval runs in one place without building a code-first harness for the comparison.

The July open-source LLM release tracker applies the license half of this process to current releases. Your workload-specific evaluation supplies the other half.

The Label Is a Starting Point, Not a Deployment Decision

The useful distinction is no longer “closed or open.” It is:

• What can we download?
• What can we legally do with it?
• Can we study and rebuild the system?
• Does it work on our task at an acceptable operational cost?

Call a model open source only when the full release earns that label. Call downloadable parameters open weight when that is what shipped.

Then test the model. The most open checkpoint can still be the wrong production choice, and the best benchmark score can still hide a license your team cannot accept.