[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Question about robots.txt

Hi Kurt,


We made the choice a long time ago to not allow indexing of the cve.mitre.org web site. At least part of that decision was simply resource constraints – when CVE was in its toddler years, search engine indexers were very resource intensive.


We are currently re-examining this policy and will keep the Board posted.


Chris Coffin

The CVE Team

The MITRE Corporation


From: owner-cve-editorial-board-list@lists.mitre.org [mailto:owner-cve-editorial-board-list@lists.mitre.org] On Behalf Of Kurt Seifried
Sent: Wednesday, December 02, 2015 11:32 AM
To: cve-editorial-board-list <cve-editorial-board-list@lists.mitre.org>
Subject: Question about robots.txt




User-agent: *

Disallow: /cgi-bin/


This means all the CVE's in the database, e.g.



are not really searchable/indexed via any search engine, is this intentional?


Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: 

Page Last Updated or Reviewed: December 09, 2015