[Date Prev][Date Next
][Thread Prev][Thread Next
[CVEPRI] Increasing numbers and timeliness of candidates
Later tonight, I will be proposing about 330 candidates to the
Editorial Board. Note that this number of candidates was generated in
6 weeks. We have significantly increased our rate of candidate
January 1 - May 31, 2001: 496 candidates
January 1 - May 1, 2002: 820+ candidates
The yearly totals are as follows:
2000 - 1189 candidates
2001 - 1460 candidates
2002 - 2460 candidates (estimated)
I expect that MITRE will be producing more content at a faster rate
now, because (a) the number of issues being reported is rapidly
increasing, (b) the content team is becoming better and faster at
creating candidates, and (c) MITRE is making a commitment to
generating content more frequently and in a more timely fashion. For
all practical purposes, (c) means that I am personally generating more
content than I was in late 2001.
123 of today's candidates are "leftovers" from the late-2001 backlog
of candidates. These are combined into 3 MISC-2001-xxx clusters.
Other "leftovers" from 2001 will continue to trickle in as MITRE
processes the more complex or vague issues. However, that backlog is
basically gone (though the more complex and daunting legacy backlog
Note that some of these 2001-era candidates were complex enough that
only consultation with the vendor was sufficient to iron out the
details. Mark Cox of Red Hat has helped out greatly in this area! I
have found that in some cases, a painstaking attention to detail, a
high level of expertise, and a detailed discussion with the vendor is
the only way to generate a usable CAN.
The other 208 candidates being proposed tonight, represent some
portion of the issues that have been announced this year, the bulk of
them before March. So far, we have 100+ candidates for January 2002
and 150+ for February.
Now that we have overcome the most recent backlog, the CVE content
team can concentrate on becoming more timely. This may include major
modifications to our process and tools for generating content. We
have already made significant changes in recent months, including (a)
better error checking during refinement, (b) enhancing incoming
submissions to collect and pre-format CVE style references, which
reduces refinement time and reduces the number of duplicates, (c)
better feedback mechanisms from the Editor (me) to members of the
content team, (d) improvements to the refinement GUI, and (e) better
As CVE becomes more timely, I believe that this may have some
noticeable impact on the initial quality of the candidates, in the
1) Application of content decisions - having a 1-2 month "perspective"
on multiple, closely related vulnerabilities helps us to use the
right level of abstraction for candidates. With less time for the
community to find all closely related vulnerabilities for an issue,
or for details to be "leaked out" after an initial delay, we are
more likely to make abstraction errors. In the past year and a
half, the content decisions have been modified so that they are
less "brittle" with respect to the amount of information that is
available at any one time, but they cannot be expected to handle
imperfect or incomplete information, especially without clear
vendor acknowledgement of the issue.
The result may include: (a) an increase in the number of RECAST
operations that would be required to SPLIT or MERGE items, which
may increase database maintenance for CVE compatible vendors, as
well as confusion for CVE end users; or (b) merging newly
discovered issues with existing CANs if the CDs already dictate it,
sort of a "soft recast" to use parlance from an old Board meeting.
Some Board members may argue that content decisions related to
abstraction are not that essential, and that we should tolerate
some error in abstraction. However, I believe that as product
liability and security becomes more important and quantifiable,
good metrics will become more important. Content decisions ensure
that CVE-based metrics are as reliable as possible. This will help
ensure that comparisons between products - whether software
products or security products - remain as fair as possible.
2) CANs will be proposed with fewer references, since not every data
source will have created the references at the time the candidate
is created. This is especially the case with vendor bulletins.
3) We will begin proposing more CANs before vendors have provided a
patch or other acknowledgement (typically between 7 and 45 days, if
the issue was released before a fix was available). This in turn
will impact the vendor acknowledgement field that we provide to
voters. Subsequently, this impacts the level of confidence that a
voter may have in an issue; without vendor acknowledgement, a voter
may be more likely to NOOP an issue.
This will result in more "permanent" candidates and/or delays in
moving candidates to the entry stage, because there will likely be
fewer supporting votes, overall.
There will be several ways to combat this:
(a) more frequent voting by more Editorial Board members
(b) notifying voters when the vendor acknowledgement status has
changed, in case some want to change their NOOP.
(c) figuring out a method that allows voters to do a "conditional
acceptance" if one or more Board members replicate the issue.
It is likely that we had all 3 of these problems to some degree in
late 1999 and 2000, when things were a little more timely. With the
perspective of having extensive and mature data (i.e. the backlog), I
am better able to understand the impact of *not* having all that data
Although there will be some growing pains, increasing CVE's timeliness
and capacity is an important goal. As MITRE modifies its processes
over the coming months, we will be considering many factors, such as
the ones I outlined above. We will try to balance speed, correctness,
and the maintenance costs for voting Editorial Board members, CVE
compatible products, and CVE end users. Really though, it's just
another day in the life of CVE ;-)