[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Question about robots.txt

To: cve-editorial-board-list <cve-editorial-board-list@lists.mitre.org>
Subject: Question about robots.txt
From: Kurt Seifried <kseifried@redhat.com>
Date: Wed, 2 Dec 2015 10:32:18 -0700
Authentication-Results: spf=none (sender IP is 129.83.29.3)smtp.mailfrom=LISTS.MITRE.ORG; mitre.mail.onmicrosoft.com; dkim=none (messagenot signed) header.d=none;mitre.mail.onmicrosoft.com; dmarc=none action=noneheader.from=redhat.com;
Delivery-Date: Thu Dec 3 08:51:10 2015
List-Help: <mailto:LISTSERV@LISTS.MITRE.ORG?body=INFO%20CVE-EDITORIAL-BOARD-LIST>
List-Owner: <mailto:CVE-EDITORIAL-BOARD-LIST-request@LISTS.MITRE.ORG>
List-Subscribe: <mailto:CVE-EDITORIAL-BOARD-LIST-subscribe-request@LISTS.MITRE.ORG>
List-Unsubscribe: <mailto:CVE-EDITORIAL-BOARD-LIST-unsubscribe-request@LISTS.MITRE.ORG>
Sender: <owner-cve-editorial-board-list@lists.mitre.org>
SpamDiagnosticMetadata: 43da9c421be74aed97248473db21fd15
SpamDiagnosticOutput: 1:2

http://cve.mitre.org/robots.txt

User-agent: *

Disallow: /cgi-bin/

This means all the CVE's in the database, e.g.

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0160

are not really searchable/indexed via any search engine, is this intentional?

--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: secalert@redhat.com

Follow-Ups:
- RE: Question about robots.txt
  - From: "Coffin, Chris" <ccoffin@mitre.org>

Next by Date: Re: Upcoming changes for CVE
Next by thread: RE: Question about robots.txt
Index(es):
- Date
- Thread