1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

robot.txt and disallow google bot

Discussion in 'Search Engine Optimization (SEO)' started by duongkhue84, Apr 21, 2016.

  1. duongkhue84

    duongkhue84 New Member

    Joined:
    Apr 21, 2016
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    0
    I have added an seo tool to my forums , and it asked me to added the following robot.txt ,does this robot ok . does it have any problems with bots . am just want to make sure .
    it was just upgrading to the currently used one . in goal to avoid duplicate content on my site was the need to add robot.txt .i want to ask too . does duplicate content in the same website domain cause problems with google .
     
  2. shabbir

    shabbir Administrator Staff Member

    Joined:
    Jul 12, 2004
    Messages:
    15,314
    Likes Received:
    372
    Trophy Points:
    83
    Its not robot.txt but its robots.txt
     
    Satya123 likes this.
  3. Satya123

    Satya123 New Member

    Joined:
    Jan 25, 2016
    Messages:
    10
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Hyderabad, India
    Home Page:
    no, because its not robot.txt , it is robots.txt
     
  4. Satya123

    Satya123 New Member

    Joined:
    Jan 25, 2016
    Messages:
    10
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Hyderabad, India
    Home Page:
    what he said, it's right.
     
  5. Aditya Agarwal

    Aditya Agarwal New Member

    Joined:
    Aug 13, 2016
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Gender:
    Male
    Duplicate content will always be problematic. With robots.txt file you can block the path which is duplicated.
     
  6. kidzone

    kidzone New Member

    Joined:
    Jul 15, 2016
    Messages:
    28
    Likes Received:
    1
    Trophy Points:
    1
    If you have duplicate content in same domain, then it will cause problem. Make use of robots.txt file to block them.
     
  7. Jeff Ronald

    Jeff Ronald Member

    Joined:
    Dec 9, 2016
    Messages:
    58
    Likes Received:
    0
    Trophy Points:
    6
    Gender:
    Male
    Home Page:
    The rel=canonical element, often called the “canonical link”, is an HTML element that helps webmasters prevent duplicate content issues.
     

Share This Page