|
|
|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [CVEPRI] January 18 Editorial Board Teleconference Summary
CVE Editorial Board Teleconference Summary - January 18, 2001 ------------------------------------------------------------- Participants ------------ Participants in the teleconference were: Ken Armstrong (EWA-Canada/CanCERT) Andy Balinsky (Cisco) Tim Collins (NFR) Andre Frech (ISS) Shawn Hernan (CERT/CC) Scott Lawler (DOD-CERT) Jim Magdych (NAI/PGP) David Mann (BindView) Pascal Meunier (Purdue CERIAS) Larry Oliver (IBM) Craig Ozancin (AXENT/Symantec) Mike Prosser (Symantec) Kevin Ziese (Cisco) MITRE participants included Dave Baker, Bob Martin, Margie Zuk, and Steve Christey. CVE Content Update ------------------ The Board was updated with the latest statistics for CVE content, including estimates for the next CVE version. On January 16, 234 candidates were moved to the Interim Decision phase. 232 of those candidates have since been added to the new version, 20010122. There are now 1309 entries and 815 active candidates, with 185 candidates needing at least 1 more vote before they can be converted to entries. The remaining active candidates are affected by unresolved content decisions, or they have received a REVIEWING, REJECT, or RECAST vote. Statistics were also provided regarding submissions. (See http://cve.mitre.org/board/archives/2000-07/msg00024.html for background information on the submission process). There are currently 10095 submissions, i.e. items from vulnerability databases submitted by various organizations. Most of them were obtained in mid-2000, when 10 organizations provided MITRE with their vulnerability databases. Approximately 1400 submissions have been matched against other submissions, candidates, and entries. 400 of those submissions are being refined into candidates. In the past few months, several new CVE content team members have been added. Ramsay Key and Jeff Taylor, the newest members, join Barbara Pease, Jean-Paul Otin, Dave Goldberg, and Dave Baker. New CVE Content Goals for MITRE ------------------------------- With the current CVE content team, MITRE projected that all legacy submissions will be matched by June 15th. By July 1, submissions for security issues that were publicized in 1999 will be refined into candidates. The remainder of the legacy submissions are expected to be refined into candidates by the end of 2001. Another task is to process the remaining submissions for problems announced in 2000; due to the nature of the process, some submissions have "slipped" through the cracks. By processing all legacy submissions and converting them into candidates, MITRE would significantly increase the comprehensiveness of CVE, allowing CVE-compatible products to map 80% or more of their own vulnerabilities to CVE names. In general, Board members believed that MITRE's projected deadlines for legacy submissions were sufficient for their needs. There was some discussion regarding how long it takes Board members' organizations to research and create their own database entry for an individual vulnerability. The amount of time varies from minutes to days (depending on complexity, accuracy of information, whether a complete exploit has to be constructed, etc.) However, the Board agreed that MITRE's estimated review times seemed reasonable. Since the CVE content team has only solidified in the past few months, the projections may be inaccurate. Some members also expressed concerns that the schedule could slip - or CVE content in general would suffer - if a critical content team member became sick or left the project. While MITRE has fully staffed the CVE content task as budgeted, it may be insufficient for the amount of work that is required to process legacy submissions. Documentation and training is taking place to minimize the impact of the loss of a content team member. Progress will be reviewed at the next face-to-face Editorial Board meeting in March. MITRE's main CVE activities were presented to the Board. In addition to processing legacy submissions, MITRE is also keeping up-to-date with new security issues. Some Board members expressed concerns that it currently takes too long - sometimes more than a month - to assign candidates for newly publicized problems. These delays have occurred for two main reasons: (a) new security problems have been processed entirely by Steve Christey, and are thus affected by his other CVE activities; and (b) there has been a significant increase in the number of new security issues announced per month. In 2000, a at least 1082 new security issues were disclosed, approximately twice as many as there were in 1998. The delays will be addressed in two ways. First, one of the content team members will begin assisting Steve in creating candidates for new issues. Also, it is expected that there will be a continued increase in the number of candidates that are reserved by organizations for inclusion in the first public announcement of a new vulnerability. These two factors should speed up the process of assigning candidate numbers to new security issues. In addition to the creation of candidates for new and legacy problems, MITRE is also increasing the focus on CVE compatibility (by educating the public, finalizing the requirements, and developing an evaluation process). MITRE plans to improve voting support for Editorial Board members, to make it as easy as possible for them to vote. MITRE is also adding software vendor liaisons, who could consult on CVE items related to their own products. Editorial Board Voting Status and Issues ---------------------------------------- It was noted that without specific goals for the Editorial Board, voting on candidates generally is not at the level that is necessary for handling the workload. When major goals are set, the Board as a whole works together to reach those goals, but major goals are only set about twice a year. Overall Board participation was reviewed. Of 38 non-MITRE Board members, 17 had voted on candidates for new issues that had been announced in the past year. 12 of those members had been on the Board for at least 6 months, and 5 of them had voted on less than 50 candidates over that time (which is 10% of the candidates that were proposed). While some Board members play other roles in the Board and are not expected to vote, the percentage of voting members - and the average level of activity - is lower than preferred, as the Editorial Board has been regarded as an organization of highly technical members. In addition, there has been an overall decrease in the number of votes per candidate, and there have been more NOOPs ("No Opinion" votes). This has resulted in increased noise in the CVE entries that are produced; for example, a duplicate CVE entry was inadvertently added to CVE, partially because it only had the minimum number of reviewers. In some cases, some candidates with vendor acknowledgement got a vote from a MITRE Board member (Dave Baker) and only one non-MITRE person. There has also been a lack of consistency of voting for organizations who have 2 members on the Editorial Board. Use of the voting web site has also declined since it was first made available to Board members in Fall 2000. Overall, it is estimated that at least 5 different voters are needed per candidate, to ensure that it is properly reviewed. While statistics are not available at this time, it is believed that the Board is not reaching this level of participation in voting. Some of the newest Board members said that they had not been provided with the supporting documentation for voting, such as information on the voting web site. Those members will be provided with the voting documentation, and it will be included in the support package for any new members that join. Some of the decline in voting is also due to the more stringent voting guidance that was created in the fall, as documented in the VOTE content decision (see http://cve.mitre.org/board/archives/2000-10/msg00000.html). The guidance suggests that a Board member should have good confidence that a reported vulnerability is real. Some Board members said that they were not voting as often because of the more stringent voting guidelines; for example, some Board members were not comfortable in voting on candidates that they did not verify personally. As currently written, the guidance does not include specific criteria for determining when a candidate is real, so different Board members apply those criteria in different ways. Some criteria for establishing the veracity of a candidate were suggested. A Board member could be reasonably confident that a candidate is a legitimate security problem if: (a) they verify the problem themselves; (b) the affected vendor has acknowledged that the problem exists; (c) someone they trust has verified the problem; or (d) there has been confirmation by independent sources. The Board decided to have a more detailed discussion of voting guidelines and practices at the face-to-face meeting in March. Voting-related topics during that meeting will include: (a) formalizing voting requirements (e.g. what percentage of candidates a Board member should be "expected" to review); (b) making Board members' voting records more easily accessible to the public; (c) establishing more consistent guidelines for all Board members to follow when voting; and (d) how confident the Board should be in the veracity of candidates, before they are converted into entries and added them to the official CVE list (this in turn affects how stringent the voting guidelines should be.) Board members also discussed the possibility of providing custom voting capabilities to individual members, outside of the normal process of using the standard e-mail ballots or the voting web site. MITRE will work with interested individual members to develop ad hoc voting processes that make it as easy as possible for them to vote. Content Goals for the Editorial Board ------------------------------------- Participants discussed the next set of CVE content goals for the Editorial Board. Members agreed to expand CVE to 1600 entries by June. It is also hoped that Board members will vote more actively and increase the average number of voters per candidate. Given the changes in voting practices that may take place after the face-to-face meeting in March, it was agreed that setting more long-term goals would not be practical. Confidence Levels ----------------- In November 2000, MITRE proposed the idea of attaching "confidence levels" to CVE candidates and entries. (See http://cve.mitre.org/board/archives/2000-11/msg00006.html and http://cve.mitre.org/board/archives/2000-11/msg00007.html). However, in the teleconference, MITRE stated that they would not extend CVE to include such information. If they had been adopted, confidence levels could have simplified Editorial Board voting practices while making the "quality" of CVE information more explicit to users of CVE. Several Board members were supportive of this approach. However, after offline discussion with some organizations, it was decided that it was not appropriate to extend CVE to include confidence levels. The primary motivator was that it potentially competed with existing or future vulnerability databases. Since CVE must minimize competition (real or perceived) with other efforts as much as possible, MITRE decided not to adopt confidence levels. However, the concept may be proposed to the public in another context, outside of CVE. In addition, to address potential public misconceptions of the process by which CVE entries are approved, the voting rules and guidance could be made more publicly accessible. Confidence levels could be useful to Board members who vote on candidates. For example, one member may have been able to replicate a reported vulnerability; if other members know this, then they might be more comfortable voting to accept a candidate, even if they haven't replicated it themselves. MITRE will investigate ways of making this information more easily available to Board members, though the information will remain private. CVE Entry Deprecation Process ----------------------------- While the CVE candidate review process is designed to minimize the number of entries that are erroneously created, there is a risk that an entry is improperly assigned. For example, a duplicate entry may be added, or an existing entry may need to be split into different entries, or merged with other entries. When such a change is needed, a CVE entry may need to be deprecated. In the face-to-face meeting in August 2000, the Editorial Board had some initial discussions on the deprecation process. Deprecation was not necessary until recently, when a duplicate CVE entry was inadvertently added to the CVE list. The following process for handling deprecation was presented to the Board; it will be followed when an entry must be deprecated. 1) Initially, entries that have been marked for deprecation enter the "Reassessment" phase. 2) The CVE Editor sends an email to the Editorial Board to identify those entries that have been targeted for deprecation. For each entry, the reasons for deprecation will be noted. 3) The Board is provided given a short period of time to review the reasons for deprecation, at least 4 business days. 4) The CVE Editor makes a Final Decision to deprecate the entry (barring any feedback by Board members). 5) The description of the entry is modified to say that it has been deprecated, and it also provides the reason for deprecation. If the entry is a duplicate, then the correct CVE entry or candidate is also identified. The description will contain keywords that allow users to quickly find all deprecated entries. 6) The deprecated entry is noted in the CVE difference report, which compares one CVE version with the previous version. 7) The entry remains in the CVE list, in case CVE-compatible tools still use the deprecated entry. This process will be followed starting with the next version of CVE. A different way of identifying deprecation status would be to add another field to CVE. That approach will be considered if the current approach is not sufficient. CVE Entry Modification Process ------------------------------ Sometimes a CVE entry needs to be modified. For example, the description may contain some minor inaccuracies, or additional references may need to be added. In the past, such modifications were made by the CVE Editor without approval by Board members. However, at the August 2000 meeting, some Board members suggested that a short review period could reduce the possibility that inappropriate modifications would be made to CVE entries. The following process for entry modification was presented to the Board. 1) Initially, entries that have been marked for modification enter the "Reassessment" phase. 2) The CVE Editor sends an email to the Editorial Board to identify the entries that are being modified, providing details of the planned modifications. 3) The Board is given a short period of time to review the reasons for modification, at least 4 business days. 4) The CVE Editor makes a Final Decision to modify the entries (barring any feedback by Board members). 5) The modifications are made in the next CVE version. 6) The modifications are noted in the CVE difference report, which compares one CVE version with the previous version. Candidate Rejection Process --------------------------- In some cases, a candidate may need to be rejected. For example, if a problem is reported, it may be discovered that the reported problem is not real; or, the candidate may be a duplicate of another candidate or entry. The following process for candidate rejection was presented to the Editorial Board. 1) The CVE Editor identifies the candidates to be rejected, and moves them to the Interim Decision phase. 2) The Editor sends an email to the Editorial Board to notify them about the candidates that are slated for rejection. The Editor annotates the voting record to specify the reason for rejection. 3) Board members the normal review period to review the candidates, typically 4 business days. 4) The CVE Editor makes a Final Decision to reject the candidates. 5) If a candidate is rejected, then its description is changed to state that it has been rejected. The description will contain keywords that allow users to quickly find all rejected candidates. If the candidate is a duplicate, then the correct entry or candidate is also specified. 6) The rejected candidates are identified in a separate report, possibly as part of a CVE version difference report. 7) The candidate will continue to be included in the candidates list, as downloaded from the CVE web site. This will allow users to understand the status change. It was suggested that it might be important to keep the description for some candidates intact. For example, suppose a security issue is reported and assigned a candidate number, but it is proven to be incorrect later on. If users search the candidates list for the issue, then they should be able to obtain the candidate name and see that the candidate has been rejected. This issue will be revisited at a later time; currently, the rejection process will only be applied to duplicate candidates. Candidate Reservation --------------------- Candidate reservation is the process by which individuals or organizations obtain candidate numbers so that they can include them in the initial public disclosure of vulnerabilities. The number of reserved candidates has increased steadily. Approximately 65 candidate numbers have been reserved, most of them by leading security organizations. (See http://cve.mitre.org/board/archives/2000-05/msg00178.html for some early rationales for this approach). As the use of candidate reservation grows, it will make candidate numbers available in a more timely fashion. As more and more organizations adopt it, this may encourage other organizations to use candidates in their advisories. While candidate reservation is available to anyone who requests it, MITRE is not actively publicizing this capability yet. To provide rapid response on a larger volume of candidate number requests, members of the CVE content team need to gain additional experience so that they can apply CVE content decisions to determine how many candidate numbers must be reserved. Also, the organizations who currently reserve candidates work closely with the affected vendor before release, and they are able to clearly describe those vulnerabilities. This will change as more people request candidates without providing sufficient information or consulting the vendor. This in turn would increase the amount of noise in reserved candidates, as well as the amount of non-public vulnerability information that is made available to MITRE. It could interfere with the current vulnerability disclosure process and/or force MITRE into a CERT-like role of an intermediary between the discloser, the vendor, and the public. One way of minimizing the amount of information that is provided to MITRE, while increasing responsiveness and decreasing the impact on the current disclosure process, is to link candidate reservation more closely to the organizations that are currently involved in the disclosure process. This could include major software vendors and neutral third parties who act as intermediaries between disclosers and vendors, e.g. CER/CC or SecurityFocus' VulnHelp service. Some major operating system vendors are beginning to include candidate numbers in advisories. MITRE has begun consulting with some OS vendors to determine the best way to make candidates available in their security advisories, including initial public announcements. If vendors or neutral third parties are given a pool of candidate numbers to assign to reported problems, then candidates could be announced without MITRE's direct involvement. However, there is an increased risk that multiple candidate numbers could be assigned to the same issue, or a candidate might be assigned at the wrong level of abstraction. Other complexities exist, such as ensuring that the discloser and all affected vendors use the same candidate number, and reducing the amount of information available to MITRE while reliably producing candidates. Candidate reservation issues will be discussed in more detail at the March meeting. Other Issues ------------ Following are some other issues that were discussed or presented. 1) MITRE will be focusing more on CVE compatibility this year, e.g. expanding the requirements document for new types of products, finalizing the evaluation process, and working more closely with vendors. Bob Martin of MITRE will be leading this task. 2) MITRE is continuing to form the CVE Advisory Council, a group of CIO-level managers who provide funding and high-level guidance for the CVE Initiative. The next Council meeting is on January 25th. 3) MITRE is working to establish liaisons with software vendors. These liaisons will not be a formal part of the Editorial Board; however, they will be available to consult with the Board on technical security issues related to their own products. 4) MITRE is re-evaluating the requirements for Editorial Board membership. In the past, the Board has been an informal organization. As it grows, however, it becomes more important to be precise about the roles and responsibilities, and the expected tasks, for each member. Board membership requirements will be reviewed at the March meeting. 5) Board members discussed possible dates for the next face-to-face meeting. The meeting will be hosted by Cisco in mid-March. In addition to the topics mentioned in this summary, other points of discussion will include CVE compatibility, a progress report on the IDS-specific Common Intrusion Event List (CIEL), and new approaches to critical CVE content decisions.
|
||||
