How We Build the CVE List

Introduction

The process of building the CVE List is divided into three stages: the initial submission stage, the candidate stage, and the entry stage. MITRE is solely responsible for the submission stage but is dependent on its data sources for the submissions. The CVE Editorial Board shares the responsibility for the candidate and entry stages, although the entry stage is primarily managed by MITRE as part of normal CVE maintenance.

Stage 1: Submission

For the CVE project, MITRE has a content team whose primary task is to analyze, research, and process incoming vulnerability submissions from CVE's data sources, transforming the submissions into candidates. The team is led by the CVE Editor, who is ultimately responsible for all CVE content.

Conversion Phase

During the submission stage, MITRE's CVE Content Team, which consists of MITRE security analysts and researchers, collects raw information from various sources, e.g. the various Board members who have provided MITRE with their databases, or publishers of weekly vulnerability summaries. Each separate item in the data source (typically a record of a single vulnerability) is then converted to a "submission," which is represented in a standardized format that facilitates processing by automated programs. Each submission includes the unique identifier that is used by the original data source.

Matching Phase

After this conversion phase, each target submission is automatically matched against all other submissions, candidates, and entries using information retrieval techniques. The matching is based primarily on keywords that are extracted from a submission's description, references, and short title. The keywords are weighted according to how frequently they appear, which generally gives preference to infrequently seen terms such as product and vendor names and specific vulnerability details. Keyword matching is not completely accurate, as there may be variations in spelling of important terms such as product names, or an anomalous term may be given a larger weight than a human would use. The closest matches for the target submission (typically 10) are then presented to a content team member, who identifies which submissions are describing the same issue (or the same set of closely related issues) as the target submission.

Once matching is complete, all related submissions are combined into submission groups, which may include any candidates or entries that were found during matching. Each group identifies a single vulnerability or a group of closely related vulnerabilities. These groups are then processed in the next phase, called "refinement."

Refinement Phase

Typically, a content team member is assigned a batch of 20 or more submission groups, which usually includes both duplicate submissions and new issues.

During refinement, the team member analyzes a submission group and determines whether one or more of the submissions identify an existing CVE item. If so, then the analyst notes any additional references that are in the new submission, but not the existing CVE item, so that the existing CVE item's references can be extended.

If there are submissions from the group that do not describe an existing CVE item, then a team member makes the following assessment:

• Apply CVE content decisions to decide whether any candidates should be created.
• Apply CVE content decisions to decide how many candidates must be created.

For each candidate to be created, the analyst does the following:

• List the associated references using CVE's canonical reference format.
• Create a description.
• Determine if there is vendor acknowledgement.
• Identify any related content decisions.
• Identify other supporting information such as the date the problem was announced, high-level operating system (OS), whether the issue is remotely or locally exploitable, and a few other attributes. This information is used to group sets of candidates later in the process, or to provide tailored voting ballots to individual Editorial Board members.
• Identify any keywords that could help in later submission matching (as well as the CVE Web site's search engine). Typically the keywords include alternate spellings or terms that were not explicitly necessary for the description.
• Identify the rationales for acknowledgement and content decisions in the "analysis" section.

In some cases, an analyst may choose to delay analysis of a submission group (or a portion of the group) when an issue is unusually complex or if other individuals need to be consulted.

Submission refinement is a bottleneck because deep analysis is sometimes required to understand the reported problem, apply the content decisions, find vendor acknowledgement, research the references, and write the descriptions. Refinement is especially difficult for new analysts because there is a large amount of detail and background knowledge that is required before the analyst can be comfortable and productive in doing refinement.

For each action that the content team member undertakes—whether identifying a duplicate, rejecting a submission, or suggesting the creation of a new candidate—a "refinement group" is produced. One or more refinement groups are produced from the original submission group, depending on how many separate issues were in the original submission group.

Editing Phase

After refinement, the CVE Editor reviews the work of the analysts, occasionally making modifications to follow CVE style, ensure that CVE content decisions are being followed, or to do advanced research. Occasionally, submissions may be added or removed from the refinement groups. The Editor provides feedback to the analyst for the purposes of training or to raise certain issues. Since the submission matching may not always find all related submissions, typically due to spelling inconsistencies across submissions, the Editor may merge multiple refinement groups that were produced by different analysts.

The Editor then processes the resulting refinement groups. New candidate numbers are assigned to the groups that identify new issues (the "Assignment Phase" in Stage 2, below).

After candidate assignment, each data source is provided with a backmap from their submission to the associated CVE items (whether newly created candidates, or existing candidates or entries). The backmap can reduce the amount of effort that the data source needs to perform to maintain a mapping to CVE. After the backmaps for the candidates are generated, the associated submissions are removed from the submission pool. In addition to backmaps, a "gapmap" is also provided to the information source. The gapmap identifies the newly created candidates that were not found in the data source's original set of submissions, which may make the source aware of additional security problems that they had not seen previously.

In some cases, the submission stage may be entirely bypassed. This usually happens when an individual or organization reserves a candidate number in order to include it in the initial public announcement of a vulnerability.

Stage 2: Candidates

Assignment Phase

Candidates are normally created in one of three ways: (1) they are refined by the content team using submissions from CVE's data sources; (2) they are reserved by an organization or individual who uses it when first announcing a new issue; or (3) they are created "out-of-band" by the CVE Editor, typically to quickly create a candidate for a new, critical issue that is being widely reported.

Proposal Phase

The CVE Editor organizes candidates into clusters of 20 to 50 candidates. For new issues, the clusters are typically grouped by the initial public announcement dates of the candidates. For legacy issues, the clusters may be created according to other criteria that make the clusters more manageable for Editorial Board members to work with, such as UNIX vendor advisories. The candidate clusters are then proposed to the Board for review and voting.

Voting Phase

Editorial Board members review proposed candidates, registering their feedback with a vote and optional commentary. Votes include ACCEPT, MODIFY (signifying the need for a minor change), REJECT, RECAST (signifying the need for a major change), REVIEWING (member is actively reviewing the candidate but does not have a vote ready), and NOOP (no opinion). A Board member may ACCEPT or MODIFY a candidate if (1) it has been acknowledged by the vendor, (2) if the issue has been replicated by the voting Board member, (3) if the issue has been reported or replicated by someone whom the member trusts, or (4) if there is independent confirmation from another party. MITRE is considering whether (4) is sufficient to establish the veracity of a candidate. One issue that has not yet been resolved is how to deal with "permanent" candidates that may be real, but never receive enough positive votes to be accepted as official entries.

Modification Phase

The candidate may be modified based on feedback from Board members. (See "Modification" in Stage 3, below).

Interim Decision Phase

The CVE Editor decides when the review of a candidate is complete or has come to a standstill. The Editor casts a single ACCEPT or REJECT vote, then gives Board members a "last call" opportunity to post any final comments or objections (at least 4 business days). If there are extensive comments or objections that require additional voting, the candidate may be returned to the Modification phase.

Final Decision Phase

If the CVE Editor determines that no sufficient grounds exist for changing the vote made in the Interim Decision, the decision becomes final. If the candidate is ACCEPTed, the Editor announces to all Board members that the candidate will be placed into CVE, and identifies the CVE name(s) that will be assigned to the new entry. If the candidate is REJECTed, the Editor notes the reason for rejection.

Stage 3: Entries

Publication Phase

If the candidate has been ACCEPTed, the candidate is converted into an entry by changing the name from CAN-YYYY-NNNN to CVE-YYYY-NNNN and removing the voting record. The new entry is then added to the next version of the CVE List.

Modification Phase

The entry may need to be modified in simple ways, e.g., to clarify the description or add more references.

Modifications and Deletions in the CVE List

Modification

Most candidates and entries are modified by adding more references (such as additional vendor advisories), or through small changes to descriptions (such as fixing typos and clarifying the issue). Candidate modifications are normally not explicitly presented to the Editorial Board or the public, due to the number and frequency of changes that take place. For entries, the Editorial Board is notified of basic modifications at least four business days before the new CVE version is targeted for creation.

For CVE users who want to track modification in the CVE List, MITRE provides "version difference reports" that detail which entries have changed, and how they have changed, between two versions. In addition, the Cassandra project being led by CERIAS/Purdue University offers a change monitoring report called "CVE Change Logs" that allows you to obtain daily or monthly changes to CVE identifiers. The tool is a feature of CERIAS' Cassandra incident response database service, which is listed on the CVE-Compatible Products and Services page.

Some modifications may be substantial. For example, a candidate may need to be split into multiple items, or multiple candidates may need to be merged into a single item (RECAST). This will happen if a content decision was not applied properly when the candidate was first created, or if new information forces such a change. In some cases, a RECAST may be required for entries. The procedure for recasting candidates and entries has not been completely defined, because most of these changes are due to content decisions that have not been finalized yet. However, it is certain that the procedure will cover including forward pointers from any recast item to the correct items.

In other cases, a description and/or the set of references may be vague enough that the item could appear to describe more than one different vulnerability. This happened more frequently in the early days of CVE when the utility of references in deconflicting similar issues, and the importance of having necessary details in the descriptions, was not fully understood. Vague descriptions and missing references can lead to mapping errors in CVE-Compatible Products and Services. Vendor security advisories with vague information present a special challenge; the issue is likely to be real (otherwise the vendor would not have reported it), but the issue could already be identified in a different CVE item. Consultation with the vendor may clear up any ambiguity, but it is not always possible or feasible.

Deletions

There may be several reasons why a candidate or entry should be "Deleted" from its associated list:

• If it is a duplicate of another CVE identifier.
• If further analysis shows that the vulnerability does not exist (e.g., the original report was incorrect).
• If the item needs to be recast.

Since any number of CVE-compatible products and services could be using older CVE identifiers, it is important to keep a record of what happens to each item that must be "deleted." A candidate is deleted by rejecting it. An entry is deleted by deprecating it. The process is the same for candidates and entries, and includes the following:

• An announcement is made to the Editorial Board that the item will be rejected or deleted.
• At least four business days are allowed for Board members to raise any questions (for candidates, this takes place in the Interim Decision phase).
• A Final Decision is made to Reject or Deprecate the item.
• All references for the item are deleted.
• The description is removed, and replaced with a statement that says that the item has been Rejected (for candidates) or Deprecated (for entries).
• A short reason for the action is included in the description.
• If the item is a duplicate, a reference is made to the correct CVE item(s).
• The change is noted in the next CVE difference report.
• The item remains in its associated list, so that there is always a record of what happened to it.

The references and descriptions are removed so that (1) it is clear to everyone that the item is no longer identifying the original vulnerability, and (2) the item is not returned as a result of keyword searches.