[CVEPRI] June 29 Editorial Board teleconference summary

["Steven M. Christey" <coley@linus.mitre.org>]

Below is the summary for last week's teleconference.  I apologize for
taking so long to write it, but a short vacation got in the way ;-)
Other participants may wish to add their own comments.

- Steve



CVE Editorial Board Teleconference - June 29, 2000
--------------------------------------------------

Below is a writeup of the Editorial Board review teleconference that
took place on June 29, 2000, from 1 to 4 PM Eastern time.


Participants
------------

The following Board members participated in the teleconference:

Ken Armstrong
Scott Blake
Kelly Cooper
Andre Frech
Dave Mann
Pascal Meunier
Craig Ozancin
Mike Prosser
Adam Shostack

MITRE participants included Margie Zuk, Gary Gagnon, Steve Christey,
and several members of the CVE content team: Dave Goldberg, Barbara
Pease, and Matt Wojcik.


CyberCrime Treaty Letter
------------------------

Gary Gagnon of MITRE answered questions and provided additional
details regarding MITRE's new approach toward the cybercrime treaty
letter, specifically MITRE's strategy to educate the drafters via
legal channels.  This effort is parallel to the signed public
statement that is being prepared by participating Board members.

An issue was raised with respect to how various organizations should
use the letter at this time.  For example, some have begun providing
the letter to others as part of an opening dialog on the treaty,
including some organizations who are also addressing the issue via
legal channels.  Participants agreed that because the letter is now on
a web site (albeit under a non-published URL), the letter should not
be treated with such secrecy.  Thus organizations should not be
restricted from moving forward and using the letter in their own
educational or legal activities.  However, as the letter has not been
formally released to the public, organizations should be prudent in
determining whom they contact regarding the letter.

Some participants raised concerns that the activities related to the
letter may have distracted too much from the core CVE-related
activities for which the Board is responsible.  MITRE briefly
described the impact that non-CVE activities have had on some other
CVE-related activities.  For example, during the week of June 26,
MITRE delayed moving some candidates to Interim Decision because MITRE
was reacting to Board member concerns about its change in direction
with respect to the letter.

However, various Board members agreed that the Editorial Board can
serve as a useful cross-section of varied interests in the security
community.  Previously successful non-CVE activities were discussed as
positive examples.  It was decided that a separate mailing list would
be created for discussing non-CVE issues with other Board members.


Editorial Board
---------------

Some of MITRE's role in the Editorial Board was reviewed and
clarified.  MITRE may serve as a "tiebreaker" or decision maker in
situations in which consensus is not achieved, but it does attempt to
reach consensus on all issues.  MITRE guides the direction and
activities of CVE, but input from Editorial Board members plays a
significant role in these activities.  MITRE actively promotes CVE to
the general public, e.g. through conference booths or talks.

It was suggested that MITRE should be able to make its own decisions
with respect to how it participates in activities that are not
directly related to CVE, as these activities may touch on other
aspects of MITRE's business and require consultation outside of
MITRE's own CVE personnel.

On a related note, Steve Christey observed that in some cases,
significant issues were raised on the Editorial Board mailing list
which Steve could not directly answer without further consultation
with other MITRE personnel, e.g. management.  This in turn caused
delays in day-to-day CVE maintenance; for example, Steve was not
comfortable in moving about 100 candidates to the Interim Decision
phase before some major issues related to the cybercrime treaty letter
were addressed.  In the future, if an issue requires a broader MITRE
response than Steve can provide directly, Steve will post an
acknowledgement of the issues, but continue to pursue day-to-day CVE
activities uninterrupted.

In some cases, it is difficult for MITRE to gauge the level of
agreement on a particular topic.  For most discussions, only a few
members actively participate in the mailing list.  Other members may
agree with the active participants, but do not have anything further
to add, and consequently do not post followups to the mailing list.
The Board has now become too large for MITRE to consult each
individual via phone.  To prevent this problem, MITRE will
specifically poll all Board members for feedback on critical issues.
This will help "silent" Board members to know when their feedback is
necessary, and MITRE can be more certain that all (or most) Board
members' viewpoints have been heard.

Recruitment of additional Board members was also discussed.  In recent
months it has become clear that having significant representation from
major software vendors would serve a number of useful purposes for
CVE, e.g. by providing detailed feedback on CVE candidates or entries
that require "deep analysis."  The participation of David LeBlanc and
Casper Dik has shown that vendor involvement helps CVE significantly.
Board members were requested to provide contacts from other vendors
(e.g. Linux vendors) who could be represented on the Board.  In
addition, various participants agreed that it would be useful to have
a list of official vendor contacts, even if that vendor is not
directly represented on the Board.  An open question is how to
determine fairly recruit vendors for membership.  At this time, the
informal requirements for all members include active community
participation and established commitments to information sharing.
Thus MITRE's recruitment activities often involve seeking out the most
visible individuals or organizations, then determining if they have
technically skilled people who can spend some time to contribute to
the effort.

Some concerns were raised with respect to risk that the Editorial
Board could become too large to manage.  However, the Board as a whole
is not processing candidates fast enough, especially considering that
there will be an additional influx of legacy candidates.  Since Board
members are heavily involved in other activities outside of CVE,
additional Board members will be brought into the process until there
is sufficient voting to handle the volume of candidates.  In addition,
MITRE occasionally reviews the level of participation of each
individual member; if the Board becomes too big in the future, less
active members may be asked to leave, with close attention paid to
organizations with multiple Board members.  However, the Board has not
become too large to consider such an action yet, and Board members
participate in a number of different ways that could make it difficult
to establish a "baseline minimum" level of participation.


CVE Content Update
------------------

A brief update of CVE content was provided.  As of June 29, there were
700 CVE entries (version 20000602), 722 active candidates, 54 active
candidates that were not proposed yet, and 9960 submissions, most of
which came from the vulnerability databases that various Board members
provided to MITRE in May.  The CVE content team is actively working on
submission matching - they are matching the submissions against
existing candidates, entries, and other submissions.  Once matched,
the submissions are refined into candidates.  The sheer number of
submissions will produce better candidates, as there will be a richer
pool of information to draw from, but refinement is a labor intensive
process.

Approximately 300 candidates are affected by unresolved content
decisions, approximately 100 are ready to be moved to Interim
Decision, and an additional 100 need only one more vote.

The concept of reference maps was introduced.  In the near future,
MITRE will publish mappings from each reference to the associated CVE
name(s).  This will help people who are mapping vulnerability items to
CVE, because knowing the reference name (e.g. a specific CERT advisory
number) will help the mapper to more quickly locate the associated CVE
names.

A short presentation was also made regarding the progress of
pre-publication candidate assignment (i.e. providing candidate numbers
to people who include them in their initial public announcement of a
new security problem).  MITRE will continue to provide candidates to
those who request them, but it is not fully publicizing this
capability because it is not prepared to provide rapid response on a
large scale yet.

The timeliness of the CVE process was also reviewed at a high level.
A more detailed report will be posted separately.  In general,
candidates for new problems have taken an average of 1.7 months to
move from Proposal to Final Decision (and an average of 2.05 months
from initial public announcement to Final Decision).  That average
appears to be declining.  Almost 60% of all CAN-1999-xxxx candidates
have reached Final Decision, as well as 30% of all CAN-2000-xxxx
candidates.  In some cases, an entire cluster may not be voted on by
enough Board members, which has contributed to the relatively low
finalization rate of 30% for new candidates.  The rate has also been
affected by unresolved content decisions.  And in some cases, a
candidate may receive a large number of votes, most of which are
NOOP's.

It was noted that since early 2000, the precentage of active
candidates to "finalized" candidates consistently remained near 50%
(informally, additional candidates are being proposed as quickly as
old ones are being finalized).  While it is uncertain as to what an
appropriate "finalization rate" would be, more active voting is
critical to reduce the backlog of legacy candidates.


CVE Content Issues
------------------

Board members discussed the extent to which MITRE should perform "deep
analysis" of candidates and submissions before moving them to the next
stage.  CVE content team member Dave Goldberg briefly described the
extent to which he had to conduct an analysis he performed with
respect to security issues in lpd (CAN-1999-0061) which were
discovered several years ago, but re-discovered in other Unix
distributions earlier this year, as reported by L0pht (no CAN's
assigned yet, pending the final analysis).  Activities included
reviewing source code, Bugtraq postings, change logs and README's,
patch recommendations, etc., which took about 12 hours to resolve
approximately 5 candidates.  Given the nearly 10000 submissions that
MITRE is processing, analysis of this scale clearly cannot be
conducted on a regular basis.  (It is estimated that there are about
100 security issues that require deep analysis).

Participants suggested that MITRE should conduct some reasonable
minimum of analysis before involving other Board members.  The CVE
content team will work on identifying and documenting the minimum
analysis requirements.  The evolving approach is for MITRE to be less
restrictive about assigning candidates (e.g. to help "label" complex
issues such as the lpd problem mentioned above), perform approximately
1 hour's worth of analysis to try to resolve any potential
discrepancies, then annotate these "complex" candidates.  If
additional analysis is necessary, MITRE could consult the Editorial
Board, vendor contacts, or outside parties who volunteer to support
the effort.  It is presently unclear how deep analysis results should
be made available to those who wish to view them.  While analysis
results could be included as part of the voting record, the voting
record is best suited for listing a small number of short comments.


Voting
------

It was observed that some Board members, when voting, post their votes
to the entire Editorial Board mailing list.  While this makes that
voting information more directly and immediately accessible to other
potential voters, in summer of 1999 it was thought that posting votes
to the list added too much traffic.  The opinions of participants were
mixed as to which approach is better, so the issue was left
unresolved.  The issue may become moot once online voting is provided
to Board members.

A concern was expressed that some Board members, by seeing votes from
other people, may choose to accept a candidate because another Board
member they trust has accepted it.  Most participants agreed that this
is a low risk; however, voting guidance (as recorded in CD:VOTE) will
be updated accordingly.  It was noted that an open voting record may
help Board members to identify voting discrepancies and work with the
associated parties to resolve the discrepancies.

It was suggested that it may be beneficial to some users of CVE to
know how many votes a particular entry has received; this could serve
as a rough measurement of the level of "trustworthiness" of an entry.
One problem with this sort of measurement is that the set of active
voters can vary significantly over each week, so some of the most
well-known vulnerabilities may only get a minimum number of votes
before they are accepted.  In addition, it was noted that the online
voting site will begin to obtain the reasons why a Board member
ACCEPTs a candidate, which "end users" could find very useful as a
confidence measure.  It was noted that much of this information is
already publicly accessible in the Editorial Board archives, but it is
not easy to process.  However, it was not decided whether this
information should be made more easily available or not.  If the Board
decides to formally adopt a more rapid review process than the current
one, such information may become more important to end users, as the
resulting CVE versions would be more "noisy" and less reliable than
they currently are.

One Board member suggested that it may be useful to allow Board
members to record their votes for official CVE entries, to further
bolster confidence in that entry.  Additional technical development
would be required to provide such a capability, as voting information
is only recorded and processed for candidates; however, this could be
folded in with other activities for CVE entries that could benefit
from this sort of "voting record."

Some participants expressed an interest in receiving reminders related
to voting, e.g. what candidates they are actively REVIEWING, or which
clusters they have not voted on yet.  It was emphasized that this
should be an optional service.  Various types of reminders will be
devised and made available to Board members.

Board members were asked about the role that vendor acknowledgement
plays in their acceptance of a candidate.  It was surmised that if a
vendor acknowledges a problem, a Board member may be more likely to
ACCEPT the related entry.  Some Board members said that it helps them,
while others do not use that information much, if at all.  MITRE
sometimes relies on vendor acknowledgement when processing votes, as
CD:VOTE specifically allows vendor acknowledgement to count as an
ACCEPT vote.


Other Topics
------------

There was not enough time to discuss the process for performing
various modifications to CVE entries or candidates, such as candidate
recasts and rejections, or entry recasts and deprecations.  The
process will be developed and discussed at a future time.  Simple
modifications to CVE entries have occurred, most of which involve
adding more references to the entries.

Content decisions were also scheduled for discussion, but they were
also omitted due to time constraints.  It was reiterated that MITRE,
as the maintainer of CVE, should take on the responsibility for
ensuring that CVE items properly satisfy content decisions, thus
allowing voting Board members to concentrate on voting.  Participants
were asked whether content decisions should be expressed less formally
than they currently are.  While reaction was minimal, one proponent
suggested that making the CD's easy to read, and leaving them as
guidelines instead of strict rules, would be beneficial in
streamlining the process, and making it more accessible to others.

A quick summary of upcoming web site enhancements was presented.
Major additions include: reference maps, two mailing lists for general
CVE news and specific content details, CVE version information
(e.g. which entries were added to which version), a "credits" page for
those organizations that provide MITRE with access to their
vulnerability databases or periodic summaries, and the online voting
for Board members.  MITRE plans to release the new web site before the
next Editorial Board meeting on August 14-15.

One of the most popular user requests for web site enhancements
involves providing "hot links" with references.  However, as hot links
would overlap with the capabilities provided by many vulnerability
databases, MITRE has been careful not to provide such links without
consulting with the rest of the Board.  (There are also issues of link
maintenance.)  While hot links are included with most new candidates,
they are deleted when a candidate becomes an official entry; in
addition, any URL's associated with candidates are not made "live."
Teleconference participants were asked whether they thought that hot
links would be a source of competition with their own products.  None
of the participants saw a problem with this; some participants thought
it would be a direct benefit to them.  However, it was suggested that
the owners of online vulnerability databases should be consulted,
since those databases would be most affected by this capability.
MITRE has contacted several such vendors and will adopt or reject this
enhancement accordingly.

At the minimum, MITRE will make a reference key available.  The
reference key provides a basic description of the abbreviation for
each source (e.g. BID, XF, CERT) and a URL for the home page of the
source.  Presently this key is made available upon request, but
eventually it will be accessible from the CVE web site.