[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: on the topic of CNA assignments... (was Re: Please welcome Kurt Seifried to the CVE Editorial Board)



Hi Brian,
 
Brian Martin said:
> I alluded to this in less-than-subtle ways in the last year. MITRE ignored
> the comments on the editorial board list.
 
We are not sure to which "this" you are referring, so we can't provide a specific, complete response. Please clarify your statement and MITRE will respond.
 
>  Last year, Will Dormann at CERT developed a tool to discover MitM issues
> in Android applications
> ...
> CERT, being a registered CNA, started issuing IDs for these vulns. About
> three to four weeks in, CVE/MITRE arbitrarily decided that Will should
> STOP assigning IDs for these vulns.
 
Mr. Dormann’s  research results were first published in the CERT Vulnerability Notes database (a CVE “Partial Coverage Source – Other”) and CERT was assigning CVE IDs to their findings in their role as a CVE CNA. MITRE became concerned about the sheer number of IDs to be assigned and discussed our concerns with CERT. After the discussion between MITRE and CERT (which included Mr. Dormann), MITRE and CERT agreed on a way forward, and CERT stopped issuing IDs based on the results of their automated testing of Android applications.
 
Key factors leading to the agreement between CERT and MITRE included:
  • Disagreement between MITRE and CERT as to whether each of the findings noted by CERT represented a vulnerability, requiring further investigation by CERT before assigning CVE IDs.
  • While MITRE agreed that there are valid vulnerabilities in a number of the findings cited, we did not accept the blanket assertion that there is a valid vulnerability in every identified Android app that does not verify a certificate.
  • The CERT Vulnerability Notes Database is a partial coverage source and is a lower priority source.
  • The sheer number of lower-priority results asserted to be vulnerabilities would require taking resources away from handling other, higher priority "must have" CVEs.
  • The consideration that a relative explosion in the number of CVEs published would create problems for downstream consumers.
  • The not-yet-completely-settled discussion of conditions by which CVE would handle results from automated testing.
 
> MITRE, decided on their own, without consulting the board, that a
> CNA should *STOP* issuing IDs to valid vulnerabilities. No valid reason
> was given, not to the public, and I bet a dollar not to Will himself. Just
> "oh god no stop it".
 
MITRE did not agree that all the CVE IDs being issued by CERT were for valid vulnerabilities. Our position was discussed with CERT at the time, and it was agreed that CERT would stop issuing CVE IDs for every finding from their research that had been published as a CERT:VU. This was noted in an email to the Board (2015-04-09), and was discussed at the Board meeting/conference call during RSA last year (2015-04-22).
 
MITRE often makes decisions on its own, but they are based on how we understand the Board wants CVE to be operated. MITRE viewed this specific case as a normal situation arising from a researcher’s results which needed to be clarified and discussed with the issuing CNA. MITRE views our decisions on coverage and prioritization simply as normal practice.
 
> Anyone on the board should be concerned right here. Why does MITRE have
> this absolute authority to stop issuing IDs on valid vulnerabilities?
 
MITRE does not have absolute authority. MITRE operates and manages CVE according to how we understand the Board wants us to operate it.
 
We hope that the above helps clear up our decisions regarding the automated testing of Android applications and answers the questions raised.
 
Best Regards,
The MITRE CVE Team
 
-----Original Message-----
From: owner-cve-editorial-board-list@lists.mitre.org [mailto:owner-cve-editorial-board-list@lists.mitre.org] On Behalf Of jericho
Sent: Saturday, November 07, 2015 2:03 AM
To: Art Manion <amanion@cert.org>
Cc: cve-editorial-board-list <cve-editorial-board-list@lists.mitre.org>
Subject: on the topic of CNA assignments... (was Re: Please welcome Kurt Seifried to the CVE Editorial Board)
Importance: High
 
On Wed, 4 Nov 2015, Art Manion wrote:
 
: On 2015-11-04 09:26, Kurt Seifried wrote:
:
: > One thing I do know is some of the external issues facing CVE (e.g.
: > people asking for CVE's on oss-sec that then take a while), I was
: > wondering if there are specific internal issues/challenges, e.g. are the
: > requests taking a long time/eating up resources because the requests are
: > poor? I often bounce private CVE requests back with "I need more info,
: > specifically X/Y/Z", and have been very firm with fuzzing reports (I
: > need minimized test cases and root cause analysis, not a pile of 100
: > text files that crash a command line app, and as it turns out that's all
: > the do).
:
: We (at CERT/CC) face similar issues in our CNA role.  We get researchers
: asking for a CVE ID for legitimate, but low severity/impact
: vulnerabilities that we otherwise would not handle or publish.  One
: option is to redirect the researcher to cve-assign, which sometimes
: comes back as "I already asked them and didn't get an answer so I'm
: asking CERT."
 
OK, so board... if the above is kind of foreign to you? Go read the
comics, ignore this.
 
For the 2% of you following board traffic, and my kind-of-subtle jabs in
this direction, grab your popcorn.
 
Kurt's mail, and Art's reply, are PERFECT. While MITRE can figure that out
for 'today', and moving forward, I will use this as an opportunity to
bring up the past. And the best part? I will pick on Art!
 
I alluded to this in less-than-subtle ways in the last year. MITRE ignored
the comments on the editorial board list. No one else on the board picked
up on it either, which is beyond discouraging. Because it means they are
likely oblivious to the real world of vuln disclosure.
 
So, OFFICIALLY.... MITRE... SPEAK TO THIS PLEASE.
 
Last year, Will Dormann at CERT developed a tool to discover MitM issues
in Android applications. He called it 'TAPIOCA', not to be confused with
the recent 'TAPIOCA' tool name. Because researchers are largely ignorant
and can't Google for shit.
 
CERT, being a registered CNA, started issuing IDs for these vulns. About
three to four weeks in, CVE/MITRE arbitrarily decided that Will should
STOP assigning IDs for these vulns.
 
Yep, process that for a minute or eight.
 
MITRE, decided on their own, without consulting the board, that a
CNA should *STOP* issuing IDs to valid vulnerabilities. No valid reason
was given, not to the public, and I bet a dollar not to Will himself. Just
"oh god no stop it".
 
Anyone on the board should be concerned right here. Why does MITRE have
this absolute authority to stop issuing IDs on valid vulnerabilities? You
can't argue they are valid or not, because MITRE actually spent the time
to write scripts to import Will's first run of vulnerabilities! CVE
auto-imported the data into the official CVE database, that feeds into
NVD, that is a cornerstone of our industry. In doing so, they missed the
many dozens of entries that had bad data due to the original import
scripting. To this day, we have CVE entries saying software is vulnerable,
with gibberish for the affected version number. Then, MITRE decided, no
more IDs... told Will, at CERT, which is a CNA, to stop assigning.
 
A year later, after alluding to it on the board, and MITRE ignoring
those comments... here we are. Still no explanation as to why that
decision was made, no hiring an extra intern at ~ 20k a year to import the
rest (on a budget of 1mil+)... basically, nothing holding them
accountable. Given MITRE/CVE's mission statement, that doesn't work for
me.
 
If you are wondering what all this means, please ask yourself why. Why
didn't you notice this last year? It was center-stage of the vuln
disclosure world, in many ways. No offense, but if you didn't notice me
bringing it up on list, and didn't notice it happening, are you really
suited to be on the board? 2014 represented a near 8x increase in vuln
disclosures. Yet, that isn't reflected in CVE at all, and it wasn't
brought to our attention, or our vote. Are you really comfortable with a
tax-payer funded VDB hiding ~ 80% of the disclosed vulnerabilities any
given year?
 
: I believe that CVE looks for a reasonably trusted public source of
: information (this might get to the sources and products lists) on which
: to base a CVE ID assignment and entry.  So a researcher publish
 
Yet, CERT is not trusted. See above. If we can't trust CERT to be a CNA...
who can we trust?
 
Certainly not Apple, who refuses to answer mails clarifying dupe CVE
assignments (Oct 2015).
 
Certainly not Microsoft or Adobe, who keep assigning 2015 IDs to issues
found and disclosed to them in 2014.
 
Certainly not IBM, who has released several hundred advisories using the
wrong CVE ID, that clearly states it is vendor-specific, and who has
been told by me and MITRE to stop...
 
Certainly not Cisco, who is cherry-picking which vulns get CVE
assignments, and when asked about public vuln information (that they
published) opt to redact information instead of clarifying it per
request...
 
Is there any wonder I have been asking for, and waiting for CNA guidelines
to be officially published?
 
The entire system is broken from the ground up. No one is policing it. The
rare times some asshole polices it externally, they get ignored over and
over.
 
: themselves or drop mail on full-disclosure might not be enough to
: support CVE ID assignment and writeup (in the case that neither CERT nor
: the vendor publish).
 
Oh god, stop there. F-D or Bugtraq are your 1999 or 2001 examples. This is
2015, you can't use either as an example to disclosure challenges. Look to
oss-sec first, and you will see why I am shocked I had to fight for three
years to get Kurt on board. Then consider the slightly easier sources like
EDB or PS... then for pure nostalgia, look to Bugtraq or FD, which are
front-and-center on the official sources MITRE monitors.
 
: While I support the idea that every vulnerability should have an
: identifier -- ideally a CVE ID -- there are tradeoffs with quality,
: quantity/scope/coverage, assignment speed, and resources.  It may be
: that working policy is that certain vulnerabilities just don't get CVE
: IDs?  Should a CNA (CERT) shunt requests to cve-assign or just say "no?"
 
If a CNA says "no" to requests in their own software, for ANY reason, they
should not be a CNA.
 
End of story.
 
A volunteer effort, run by 3 people in their spare time, doubled CVE's
output for half a decade. Each year, MITRE got 1mil+ from the
government, while these volunteers did it in their spare time.
 
You simply cannot argue about CVE and effectiveness, without addressing
that point. If MITRE's beauracracy is so convoluted and hindering to
the process, we need to consider alternatives.
 
.b
 

Page Last Updated or Reviewed: November 23, 2015