Go4Expert

Go4Expert (http://www.go4expert.com/)
-   Ethical hacking (http://www.go4expert.com/forums/ethical-hacking-forum/)
-   -   Microsoft Robots.txt (http://www.go4expert.com/forums/microsoft-robotstxt-t25313/)

Cleptography 21Mar2011 22:01

Microsoft Robots.txt
 
Robots.txt file for http://www.microsoft.com

User-agent: *
Disallow: /*TOCLinksForCrawlers*
Disallow: /*/mac/help.mspx
Disallow: /*/mac/help.mspx?
Disallow: /*/mactopia/help.mspx?
Disallow: /australia/mac/help.mspx?
Disallow: /belux/fr/mac/help.mspx?
Disallow: /blacklisted*
Disallow: /brasil/mac/help.mspx?
Disallow: /businesssolutions/community/newsgroups/
Disallow: /Businesssolutions/Community/Newsgroups/
Disallow: /canada/Library/mnp/2/aspx/
Disallow: /communities/bin.aspx
Disallow: /communities/blogs/PortalResults.mspx
Disallow: /communities/eventdetails.mspx
Disallow: /communities/rss.aspx
Disallow: /danmark/mac/help.mspx?
Disallow: /downloads/*/results.aspx?
Disallow: /downloads/Browse.aspx
Disallow: /downloads/browse.aspx
Disallow: /downloads/info.aspx
Disallow: /downloads/thankyou.aspx
Disallow: /events/advsearch.mspx?*
Disallow: /events/eventdetails.aspx?*
Disallow: /events/EventDetails.aspx?*
Disallow: /events/webcasts/calendar/MonthView.aspx?*
Disallow: /events/webcasts/calendar/CalendarList.aspx?*
Disallow: /finland/mac/help.mspx?
Disallow: /france/formation/centres/planning.asp
Disallow: /france/ie/default.asp
Disallow: /france/mac/help.mspx?
Disallow: /france/mnp_utility.mspx
Disallow: /genuine/ajax/
Disallow: /genuine/common/
Disallow: /genuine/Content.aspx
Disallow: /genuine/csr/
Disallow: /genuine/diag/Error.aspx
Disallow: /genuine/diag/error.aspx
Disallow: /genuine/diag/Instructions.aspx
Disallow: /genuine/diag/instructions.aspx
Disallow: /genuine/diag/RunDiags.aspx
Disallow: /genuine/diag/rundiags.aspx
Disallow: /genuine/diag/Support.aspx
Disallow: /genuine/diag/support.aspx
Disallow: /genuine/downloads/AutomaticUpdates.aspx
Disallow: /genuine/downloads/Default.aspx
Disallow: /genuine/downloads/default.aspx
Disallow: /genuine/downloads/Error.aspx
Disallow: /genuine/downloads/error.aspx
Disallow: /genuine/downloads/GenerateHTA.aspx
Disallow: /genuine/downloads/LicenseStoreError.aspx
Disallow: /genuine/downloads/nongenuine.aspx
Disallow: /genuine/downloads/nongenuineVista.aspx
Disallow: /genuine/downloads/PluginInstructions.aspx
Disallow: /genuine/downloads/RunHTA.aspx
Disallow: /genuine/downloads/SuccessfulActivation.aspx
Disallow: /genuine/downloads/TryAgain.aspx
Disallow: /genuine/downloads/Validate.aspx
Disallow: /genuine/downloads/VAMTPrivacyInfo.aspx
Disallow: /genuine/downloads/VistaDisclosure.aspx
Disallow: /genuine/downloads/VLKInstructions.aspx
Disallow: /genuine/hip/
Disallow: /genuine/IPtoGeoMap/
Disallow: /genuine/notify/
Disallow: /genuine/oe/
Disallow: /genuine/offers/Details.aspx
Disallow: /genuine/offers/details.aspx
Disallow: /genuine/office/BrowserAlternative.aspx
Disallow: /genuine/office/CounterfeitReport.aspx
Disallow: /genuine/office/FailureReport.aspx
Disallow: /genuine/office/nongenuine.aspx
Disallow: /genuine/office/OfficeKeyUpdate.aspx
Disallow: /genuine/office/OfficeKeyUpdateCheck.aspx
Disallow: /genuine/office/OfficeKeyUpdateResults.aspx
Disallow: /genuine/office/OfficeNongenuine.aspx
Disallow: /genuine/office/PluginInstructions.aspx
Disallow: /genuine/office/PrintableCounterfeitReport.aspx
Disallow: /genuine/office/PurchaseError.aspx
Disallow: /genuine/office/PurchaseInfoReport.aspx
Disallow: /genuine/office/ReportInstructions.aspx
Disallow: /genuine/office/ResultsReport.aspx
Disallow: /genuine/office/SuccessfulValidation.aspx
Disallow: /genuine/office/Validate.aspx
Disallow: /genuine/purchase/
Disallow: /genuine/rd.aspx
Disallow: /genuine/selfhelp/Error.aspx
Disallow: /genuine/selfhelp/PkuInstructions.aspx
Disallow: /genuine/selfhelp/PostKitPurchase.aspx
Disallow: /genuine/selfhelp/ValidatePID.aspx
Disallow: /genuine/selfhelp/ValidationHelp.aspx
Disallow: /genuine/selfhelp/ValidationHelpResults.aspx
Disallow: /genuine/selfhelp/ViewExample.aspx
Disallow: /genuine/survey/
Disallow: /genuine/test/
Disallow: /genuine/validate/default.aspx
Disallow: /genuine/vista/AEDetail.aspx
Disallow: /genuine/vista/AERepair.aspx
Disallow: /genuine/vista/CounterfeitReport.aspx
Disallow: /genuine/vista/LocationByRegion.aspx
Disallow: /genuine/vista/PrintableCounterfeitReport.aspx
Disallow: /genuine/vista/PurchaseError.aspx
Disallow: /genuine/vista/PurchaseInfoReport.aspx
Disallow: /genuine/vista/ReportInstructions.aspx
Disallow: /genuine/vista/VistaLegalization.aspx
Disallow: /genuine/windows7/
Disallow: /genuine/wizard/
Disallow: /Germany/kleinunternehmen/euga/detail.mspx
Disallow: /Germany/kleinunternehmen/euga/results.mspx
Disallow: /germany/library/images/mnp/
Disallow: /germany/mac/help.mspx?
Disallow: /germany/mnp_utility.mspx
Disallow: /germany/video/de/de/related
Disallow: /hk/mac/help.mspx?
Disallow: /hpc/en/us/supported-applications.aspx?SortField*
Disallow: /ie/ie40/
Disallow: /info/customerror.htm
Disallow: /info/smart404.asp
Disallow: /intlkb/
Disallow: /ireland/mac/help.mspx?
Disallow: /isapi/
Disallow: /italy/products/mac
Disallow: /italy/products/mac/help.mspx?
Disallow: /Japan/DirectX/default.asp
Disallow: /japan/directx/default.asp
Disallow: /japan/enable/textview.asp
Disallow: /japan/mactopia/help.mspx
Disallow: /japan/mnp_utility.mspx
Disallow: /japan/products/library/search.asp
Disallow: /japan/showcase/print/default.aspx
Disallow: /japan/terminology/query.asp
Disallow: /latam/mac/help.mspx?
Disallow: /library/gallery/components/ratingcontrol/ratings.aspx?rurl*
Disallow: /library/toolbar/3.0/
Disallow: /mac/help.mspx
Disallow: /mnp_utility.mspx
Disallow: /netherlands/mac/help.mspx?
Disallow: /netherlands/mnp_utility.mspx
Disallow: /norge/mac/help.mspx?
Disallow: /nz/mac/help.mspx?
Disallow: /resources/casestudies/casestudyimageshow.asp
Disallow: /resources/casestudies/CompanyLogoShow.asp
Disallow: /resources/casestudies/ddi/companylogoshow.asp
Disallow: /resources/casestudies/ddi/showfile.asp
Disallow: /resources/casestudies/FindCaseStudyResults.aspx
Disallow: /resources/casestudies/showfile.asp
Disallow: /security_essentials/market.aspx?
Disallow: /singapore/mac/help.mspx?
Disallow: /spain/empresas/
Disallow: /spain/mac/help.mspx?
Disallow: /spain/medianaempresa/
Disallow: /sverige/mac/help.mspx?
Disallow: /uk/mac/help.mspx?
Disallow: /uk/mnp_utility.mspx
Disallow: /video/en/us/related
Disallow: /windows/compatibility/windows-vista/Browse.aspx?*
Disallow: /windows/compatibility/windows-vista/Details.aspx?*
Disallow: /windows/compatibility/windows-vista/browse.aspx?*
Disallow: /windows/compatibility/windows-vista/details.aspx?*
Disallow: /windows/compatibility/windows-7/*/search.aspx?*
Disallow: /windows/compatibility/windows-7/*/Search.aspx?*
Disallow: /windows/compatibility/windows-7/*/Browse.aspx?type=Hardware&category=*&subcategory=* &os=64-bit
Disallow: /windows/compatibility/windows-7/*/Details.aspx?type=Software&p=*&v=*&uid=*&l=*&pf=*& pi=*&c=*&sc=*&os=64-bit
Disallow: /windows/compatibility/windows-7/*/Details.aspx?type=Software&p=*&v=*&l=*&pf=*&pi=*&c =*&sc=*&os=32-bit
Disallow: /windows/compatibility/windows-7/*/Details.aspx?type=*&p=*&v=*&l=*&pf=*&pi=*&c=*&sc=* &os=64-bit
Disallow: /windows/compatibility/windows-7/*/Browse.aspx?type=*&category=*&subcategory=*&os=32-bit
Disallow: /windows/compatibility/windows-7/*/Browse.aspx?type=*&category=*&subcategory=*&os=64-bit
Disallow: /windows/compatibility/Windows-7/*/search.aspx?*
Disallow: /windows/compatibility/Windows-7/*/Search.aspx?*
Disallow: /windows/compatibility/Windows-7/*/Browse.aspx?type=Hardware&category=*&subcategory=* &os=64-bit
Disallow: /windows/compatibility/Windows-7/*/Details.aspx?type=Software&p=*&v=*&uid=*&l=*&pf=*& pi=*&c=*&sc=*&os=64-bit
Disallow: /windows/compatibility/Windows-7/*/Details.aspx?type=Software&p=*&v=*&l=*&pf=*&pi=*&c =*&sc=*&os=32-bit
Disallow: /windows/compatibility/Windows-7/*/Details.aspx?type=*&p=*&v=*&l=*&pf=*&pi=*&c=*&sc=* &os=64-bit
Disallow: /windows/compatibility/Windows-7/*/Browse.aspx?type=*&category=*&subcategory=*&os=32-bit
Disallow: /windows/compatibility/Windows-7/*/Browse.aspx?type=*&category=*&subcategory=*&os=64-bit
Disallow: /windows/404.aspx?*
Disallow: /windowsmobile/catalog/
Disallow: /windowsmobile/components/referafriend/getcallbackresult.aspx

Sitemap: http://www.microsoft.com/bi/sitemap.xml
Sitemap: http://www.microsoft.com/business/smb/sitemapindex.xml
Sitemap: http://www.microsoft.com/business/success/sitemap.xml
Sitemap: http://www.microsoft.com/canada/sitemap.xml
Sitemap: http://www.microsoft.com/downloads/sitemap.aspx
Sitemap: http://www.microsoft.com/dynamics/sitemap.xml
Sitemap: http://www.microsoft.com/ebs/sitemap.xml
Sitemap: http://www.microsoft.com/en/ca/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/sqlazure/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/appfabric/azure/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/cloud/sitemap.xml
Sitemap: http://www.microsoft.com/france/sitemap_index.xml
Sitemap: http://www.microsoft.com/fr/ca/sitemap.xml
Sitemap: http://www.microsoft.com/germany/kleinunternehmen/gsitemap.aspx
Sitemap: http://www.microsoft.com/infrastructure/sitemap.xml
Sitemap: http://www.microsoft.com/msft/reports/sitemap.xml
Sitemap: http://www.microsoft.com/online/sitemap.xml
Sitemap: http://www.microsoft.com/sbs/sitemap.xml
Sitemap: http://www.microsoft.com/security_essentials/sitemap.xml
Sitemap: http://www.microsoft.com/sitemapindex.xml
Sitemap: http://www.microsoft.com/business/smb/es-es/sitemap.xml
Sitemap: http://www.microsoft.com/unlimitedpotential/sitemap.xml
Sitemap: http://www.microsoft.com/unlimitedpotential/video-sitemap.xml
Sitemap: http://www.microsoft.com/windows/sitemap.xml
Sitemap: http://www.microsoft.com/windowsazure/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/en-us/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/de-de/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/en-au/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/en-ca/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/en-gb/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/en-sg/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/es-es/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/es-mx/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/fr-ca/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/fr-fr/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/it-it/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/en-ww/sitemap.xml
Sitemap: http://www.microsoft.com/windowsphone/es-ww/sitemap.xml
Sitemap: http://www.microsoft.com/windowsserver2008/en/us/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/en-au/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/pt-br/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/en-ca/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/fr-ca/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/zh-cn/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/fr-fr/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/de-de/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/en-in/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/it-it/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/ja-jp/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/ko-kr/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/nl-nl/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/es-es/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/ru-ru/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/zh-tw/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/en-gb/sitemap.xml
Sitemap: http://www.microsoft.com/windows/compatibility/windows-7/en-us/sitemap.xml

kyle 29Mar2011 02:29

Re: Microsoft Robots.txt
 
dude
i really don't get this
i mean what is all this and how/what for, can it be used ?


All times are GMT +5.5. The time now is 12:49.