Robots?

Discussion in 'Help Requests' started by Will Lawrence, Nov 27, 2015.

  1. John Brunner

    John Brunner Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    May 29, 2020
    Messages:
    25,250
    Likes Received:
    37,077
    Try this:
    -Go to something you've posted on Facebook (or on any other forum)
    --Choose something that's been there a while...these things are not instantaneous
    -Copy a short piece of it (a dozen words or so)...hopefully something relatively unique
    -Paste it into a search engine
    -See what results you get

    Your words on a forum are no different than the content of any other website...it's just that the other website stuff is put on the web by somebody else.

    edit to add: I just tried it with extracts from some of your January posts on this forum, and did not get any results that were from your comments. Perhaps they were not unique enough. Perhaps they were not old enough.

    pps: Thanks for giving me Like #25,000!!! Woo Hoo!!!
     
    #46
    Last edited: May 9, 2023
    Tom Galty and Richard Whiting like this.
  2. Ken Anderson

    Ken Anderson Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    Jan 21, 2015
    Messages:
    25,491
    Likes Received:
    45,675
    We have 125 Facebook spiders here right now, which is very odd. Usually, Facebook only sends a few spiders here when someone links to one of our threads from Facebook. I assume they spider the linked thread to determine whether to block the link or warn someone for linking to something that violates their community guidelines. But 125 spiders at the same time?
     
    #47
    John Brunner likes this.
  3. John Brunner

    John Brunner Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    May 29, 2020
    Messages:
    25,250
    Likes Received:
    37,077
    So out of those 140 Robots, 125 are Facebook spiders? Or are they embedded in the Guest numbers? I assume the ability to see whose spiders they are is an Admin function.

    Now I wonder what someone here has said that's so interesting it got linked to Facebook...and who linked it.
     
    #48
  4. Ken Anderson

    Ken Anderson Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    Jan 21, 2015
    Messages:
    25,491
    Likes Received:
    45,675
    No, they are under Robots, where Google, Bing, Yandex, and the Chinese search engine robots are. Facebook spiders normally only go to the page that was linked to. Today, there are everywhere.
     
    #49
  5. Ken Anderson

    Ken Anderson Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    Jan 21, 2015
    Messages:
    25,491
    Likes Received:
    45,675
    Maybe Facebook has simply changed its policies regarding Facebook spiders (robots) because they're all over the forum today, too. They used to come only in response to a link to our forum from Facebook. Whenever I posted a link to a forum thread from Facebook, I could come and find three or four Facebook spiders here, but only in the thread that I had linked to.
     
    #50
    John Brunner likes this.
  6. Yvonne Smith

    Yvonne Smith Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    Jan 21, 2015
    Messages:
    15,771
    Likes Received:
    30,380
    I have noticed that we now never show any pf the spiders/robots searching the forum, since we have had the server update. Is this a good thing or a bad thing, @Ken Anderson ? The spiders are how we have better SEO, correct , and now we have none ?
     
    #51
  7. Ken Anderson

    Ken Anderson Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    Jan 21, 2015
    Messages:
    25,491
    Likes Received:
    45,675
    The robots are still indexing the HTTP version of the site, which is the same as the HTTPS version. Once I get around to editing a hidden file at the root of our server space. the HTTP will be redirected to HTTPS. Meanwhile, no harm is being done. Version is the wrong word to use since they both access the same database, but it conveys the gist of it.
     
    #52
    Last edited: Nov 2, 2024
  8. John Brunner

    John Brunner Senior Staff
    Staff Member Senior Staff Greeter Task Force Registered

    Joined:
    May 29, 2020
    Messages:
    25,250
    Likes Received:
    37,077
    That's interesting.
     
    #53
  9. Mary Stetler

    Mary Stetler Veteran Member
    Registered

    Joined:
    May 30, 2021
    Messages:
    7,387
    Likes Received:
    13,916
    Things have changed. I don't know when it happened but when I would come here before, I think it was http, I was informed that this was not a secure site. I thought that was what the 'S' was for. Making it secure?
     
    #54

Share This Page