My site is not referenced

    • 106 posts
    September 3, 2018 6:48 PM EDT

    Hello,

     

    I don't know if I choose the good words to explain my problem, my site is not referenced in Google, except for its name Netizz if I check the statistics on Google Webmaster Tools.

     

    Also I see a lot of URL errors :

    2 Soft 404
    43 Access denied
    1,088 Not Found

     

    Could you give me more informations about that and how I can resolve these errors ?

    • 378 posts
    September 3, 2018 10:59 PM EDT

    Run a scan such as an SEO Checker (there are dozens out there) - this will give you a better clue as to why.

    • 106 posts
    September 3, 2018 11:56 PM EDT

    thanks for the advice

    • Moderator
    • 6923 posts
    September 4, 2018 5:03 AM EDT

    Those not found errors are a concern. Did you change domains? Did you have any third party plugin with hard coded urls? Are the pages going to private pages that google bots can't access? Google has tips for that last one. It also has tips for how to add a change of address, https://support.google.com/webmasters/answer/83106?hl=en and other tips here https://support.google.com/webmasters/answer/35120?hl=en

    • 106 posts
    September 7, 2018 9:08 AM EDT

    Hello,

     

    I have recently enabled HTTPS in URLs, but even before my site was not referenced, I didn't change domains.

     

    I have bought a lot of plugins to Socialengineaddons.com, so I have asked the question, if they have some plugins with hard coded urls. They reply me I have to explain my requirement more, what can I reply to this ?

     

    There are a few pages that Google bots can't access, those that are used to create a group for example, or any other page to create content.

    • Moderator
    • 6923 posts
    September 7, 2018 12:46 PM EDT

    Let SEAO know that you are trying to find out why you have 1088 not found urls in google. 

    Try the tool that ITLJames mentioned.

    • Moderator
    • 6923 posts
    September 8, 2018 4:23 AM EDT

    So I woke at 3am with this on my mind lol. Odd thing to wake me. Have you checked your sitemap to make sure it doesn't have URLs in them that are blocked in your robots.txt file? 

    Have you checked the URLs that are "not found" to see what they are? 

    Have you set your redirect from non http to https?

    In a former site with issues like that, I checked every URL in the report. I manually updated my sitemap. I double checked my robots.txt. I also set in google some URLs not to check (forgot how I did that).

    • 106 posts
    September 8, 2018 3:32 PM EDT

    Hey thanks for the attention, I have checked my site with Wookrank.com, and I see a lot of errors, only one is maybe associated to my problem :

     

    These sitemaps in your robots.txt file have invalid URL formats

    /index.php/sitemap?format=xml&rewrite=1

     

    Also here are the URLs not found :

     

    ../stories/manage (I have deleted this page, and other similars)
    I also have modified the url of some pages, this one for example 
    blog/3/gagner-des-cadeaux?locale=en&c=117
    ../helpful (I don't remember that I have created or deleted this page)
    login/return_url/64-L2xvZ2luL3JldHVybl91cmwvNjQtTDJ4dloybHVMM0psZEhWeWJsOTFjbXd2TmpRdFRESjRkbG95YkhWTU0wcHNaRWhXZVdKc09URmpiWGQyVG1wUmRGUkVUa05sVjBsNVYyNUNhVkl4VmpKV1JtaHpaVmRHV0ZKdVVsSldNMmgxV1d4a1IyRXdOVFpQV0U1cFRXczFiMWxyWkZaUFZuQllUa2N4V21WcVFqUlVWM0JLU2xST1JRPT0%2FYz0xMjI%3D
    There are a lot of urls not found of the same type, at least 950.
     
    I didn't set redirect from non http to https, how should I do that ? and what should I do with the urls modified and deleted ?
    • Moderator
    • 6923 posts
    September 8, 2018 5:26 PM EDT

    I would suggest to get a new sitemap done and have it skip any login or logout urls. Also make sure it doesn't include index.php in the url or google will have a fit and go to a page not found. Remember, I did mention I had to manually edit the sitemap I got from a free sitemap generator. It is possible that a third party product for sitemaps might do a better job. However, if you don't want a plugin, take the time to manually edit your sitemap. 

    To redirect from non http to https, you can follow the tutorial. https://support.socialengine.com/php/customer/en/portal/articles/2036045-how-to-protect-your-site-using-ssl-and-https?b_id=14386 it has screenshots and details for redirecting. I've used that tutorial for both of my SE sites that I have hosted at BryZar and it worked fine. I also did the www redirect because it said in a google article that it was best to do that too if using www in your urls.

    • 106 posts
    September 10, 2018 9:56 AM EDT

    I have bought the Ultimate SEO / Sitemaps Plugin on Socialengineaddons.com it would be a shame that something supposed to help me in SEO, finally makes problems in SEO. I don't see any option to exclude index.php from urls.

     

    And I have cheched the tutorial to redirect from http to https, it seems to be correctly configured.

    • 106 posts
    September 10, 2018 9:58 AM EDT

    Also about the question "Are there any plugins that can have hardcoded urls in those I bought you ?"

    The SEAO team is asking me : "Can you please explain your requirement more here so that we can assist you accordingly."

     

    What can I reply to this ?

    • Moderator
    • 6923 posts
    September 10, 2018 12:04 PM EDT

    I suggest bringing the issue to SEAO as the plugin appears not to be picking up the urls or perhaps something is up with it. 

    I answered the other question above about explaining the requirement, on the 7th. Please check that post.

    • 106 posts
    September 10, 2018 2:17 PM EDT

    Okay I'm talking with SEAO about that since the last week, they take time to reply. Anyway I will be happy if my issue is corrected.

    • Moderator
    • 6923 posts
    September 11, 2018 5:00 AM EDT

    Point them to this thread so they can see the extra details. 

    • Moderator
    • 6923 posts
    September 11, 2018 9:28 AM EDT

    Is that a certified plugin?

    • 106 posts
    September 13, 2018 2:59 PM EDT

    Does a certified plugin mean that it's present on the Social Engine Marketplace ? We can find it other there.

     

    Also here is the response of SEAO about not found pages :

     

    "Page not Found" message will be coming to all the URL which required the user login in order to access the page.

    If you want, we can add some custom change so that all these URLs so that they should send some response to the user rather than showing "Page not Found" message.

    Please drop us a line if you are ok with the customization. We will share the estimation cost with you then.

     

    I think I shouldn't pay any customization in this case, but please let me know your opinion.

    • Moderator
    • 6923 posts
    September 13, 2018 3:29 PM EDT

    A certified plugin is one that is listed in the certified marketplace and that you purchased in the certified marketplace. If you purchased it elsewhere, that's not a certified plugin as we haven't tested those files ourselves.

    No. You shouldn't need a customization if you set those pages not to be indexed via the robots.txt and most likely via the settings of the plugin. Also, if you want to, on your page not found page, you should be able to add a google search. I've not looked at it myself to know how the page not found is generated though. https://support.google.com/customsearch/answer/4513903?hl=en

    Did you tell them that you don't want the plugin to index private pages?

    I found this info for you about 404 errors. It appears that Google says that they don't impact your SEO and that you can ignore them. https://moz.com/blog/how-to-fix-crawl-errors-in-google-search-console from that article that references the Google guidelines,

    “Generally, 404 errors don't affect your site's ranking in Google, so you can safely ignore them.”

    Perhaps check the google guidelines and don't worry about those 404 if it's not stopping your members from accessing them. If the members also get 404 errors, that's something to look at.

    • 106 posts
    September 16, 2018 12:47 AM EDT

    Ok so my plugin is not certified, I bought it on Socialengineaddons.com

     

    I told them that I don't want the plugin to index private pages, I am waiting for their response now.

     

    I have regenerated sitemaps, deleted those on Google Webmaster Tools and sent new sitemaps, and I have now this :

     

    0 Soft 404
    11 Access denied
    413 Not found

     

    This is better but the Not Found Pages are still increasing regularly.

    • 348 posts
    September 17, 2018 5:47 AM EDT
    Yungsun said:

     login/return_url/64-L2xvZ2luL3JldHVybl91cmwvNjQtTDJ4dloybHVMM0psZEhWeWJsOTFjbXd2TmpRdFRESjRkbG95YkhWTU0wcHNaRWhXZVdKc09URmpiWGQyVG1wUmRGUkVUa05sVjBsNVYyNUNhVkl4VmpKV1JtaHpaVmRHV0ZKdVVsSldNMmgxV1d4a1IyRXdOVFpQV0U1cFRXczFiMWxyWkZaUFZuQllUa2N4V21WcVFqUlVWM0JLU2xST1JRPT0%2FYz0xMjI%3D

    There are a lot of urls not found of the same type, at least 950.

      

    Hi Yungsun,

    As per the current functionality of our "Ultimate SEO / Sitemaps Plugin" (a SocialEngine certified plugin now), the Google Web-Crawler indexes only those pages that are public and do not require any authentication to show contents of a website.

    These URL formats come only when a non-logged in user tries to access the Private Pages of your website and thus these pages can't be shown in the Google Search results also as explained by our Support Team to you.

    Please confirm if you are getting Page Not Found message for these URLs, as according to the URL format you have shared it should display Access Denied (403 Forbidden) message.

    If you need any any further explanations on this, please feel free to reach out to our Support Team Yungsun.

    Regards,

    Team SocialEngineAddOns


    This post was edited by SocialEngineAddOns at September 17, 2018 5:57 AM EDT
    • 348 posts
    September 17, 2018 5:52 AM EDT
    Donna said:

    No. You shouldn't need a customization if you set those pages not to be indexed via the robots.txt and most likely via the settings of the plugin.

    Hi Donna,

    We are suggesting customization for above situation as we will need to invest efforts to modify our plugin to make the private pages also searchable as explained by our team to Yungsun.

    • 106 posts
    September 18, 2018 11:14 AM EDT

    Sorry for the long message, I rewrite it clearer so you can delete my previous message.

     

    Here are the urls with 403 errors :

    members
     
    member
    tag/add
    seaocore/activity/share/type/sitereview_listing/id/37/not_parent_refresh/1/format/smoothbox
    seaocore/activity/share/type/sitereview_listing/id/56/not_parent_refresh/1/format/smoothbox
    seaocore/activity/share/type/sitereview_listing/id/57/not_parent_refresh/1/format/smoothbox
    user/friends/suggest/includeSelf/1
    groupitems/dashboard/reset-position-cover-photo/group_id/13
    fanpages/profilepage/tell-a-friend/id/2
    groupitems/dashboard/reset-position-cover-photo/group_id/3
    fanpages/profilepage/print/id/2
    groupitems/create/category/55/categoryname/help-learning/subcategory/57/subcategoryname/computers-and-internet
    editors/editor-mail/user_id/1
    groupitems/create/category/34/categoryname/entertainement/subcategory/98/subcategoryname/movies-tv
    projects/project-owner-faq?c=122
    seaocore/activity/share/type/sitereview_listing/id/3/not_parent_refresh/1/format/smoothbox
    sitecrowdfunding/project/contact-owner/project_id/1/format/smoothbox

     

    And urls with 404 error :

    login/return_url/64-L2xvZ2luL3JldHVybl91cmwvNjQtTDJ4dloybHVMM0psZEhWeWJsOTFjbXd2TmpRdFRESjRkbG95YkhWTU0wcHNaRWhXZVdKc09URmpiWGQyVG1wUmRGUkVTalJrYkc5NVlraFdUVTB3Y0hOYVJXaFhaVmRLYzA5VVJtcGlXR1F5Vkcxd1VtUkdVa1ZVYTA1c1ZqQnNOVll5TlVOaFZrbDRWbXBLVm1GcmNGaGFWbHB6VjBkV1NHUkhhRmRsYkZwNlZteFdhMk15U25SVGJsSlRZV3RLVWxSWGMzZE9WbEpYVld0d1RsSkViSEZWUmxKR1pWVTFRa3BVVGtWS1ZFNUZVREpOT1UxVVNUQT0%2FYz0xMzI%3D

    (there are a lot)

     

    and also :

    format/smoothbox
    tab/
    reviews
    hashtag?locale=en&c=153

    comment/get-likes

    blogs/rss/category/11
    blogs/listing/category/15/sort/recent
    blogs/rss/category/3
    blogs/rss/category/16
    blogs/listing/category/11/sort/recent
    blogs/38/27/what-causes-lung-cancer
    blogs/38/24/benefits-of-walking
    blogs/1/29/gagner-des-cadeaux
    how-does-it-work/15/remuneration-des-articles?c=132
    how-does-it-work/13/gagner-des-cadeaux?c=125
    blog/3/gagner-des-cadeaux?c=30
    blogs/listing/category/8/sort/recent
    blog/10/fonctionnement-global-de-netizz?locale=en&c=122
    blogs/rss/category/6
    blogs/38/21/how-to-take-care-of-your-heart
    credit
    profile/yourname/action_id/102
    blogs/rss/category/15
    blogs/listing/category/17/sort/recent
    blogs/listing/category/10/sort/recent
    blogs/rss/category/17
    stories/review/update/listing_id/43
    stories/review/update/listing_id/42
    blog/10/fonctionnement-global-de-netizz?locale=fr&c=122
    blogs/34/28/5-easy-ways-to-instantly-spice-up-your-style
    dollanz
    blogs/rss/category/13
    how-does-it-work/14/fonctionnement-global-de-netizz?c=125
    blogs/listing/category/7/sort/recent
    requests?locale=en&c=133
    socialads/campaigns
    profile/angela576
    blogs/37/20/stream-one-piece-subbed
    blogs/rss/category/8
    profile/yourname
    blogs/rss/category/10
    blogs/rss/category/5
    blogs/listing/orderby/creation_date
    blogs/listing/category/13/sort/recent
    blogs/listing/category/16/sort/recent
    blogs/listing/category/14/sort/recent
    blogs/listing/category/12/sort/recent
    blogs/listing/category/2/sort/recent
    blogs/listing/category/9/sort/recent
    blogs/listing/category/6/sort/recent
    blogs/1/30/remuneration-des-articles
    blogs/rss/category/1
    blogs/rss/category/12
    blogs/rss/category/2
    blogs/rss/category/7
    profile/writingstudio22
    profile/NintendoSwitchFan
    products/top-rated?locale=en&c=20
    blog/11/atjg
    blog/10/fonctionnement-global-de-netizz?c=121
    wishlist/2/avvv 
    groupitems/map
    blog/10/fonctionnement-global-de-netizz?c=117
    study-languages/index?c=32
    reviews?locale=en_US&c=22
    • Moderator
    • 6923 posts
    September 18, 2018 12:19 PM EDT

    It should never be indexing any of this:

    login/return_url/

    So they either need to add a way for the seo plugin to skip that by default (not as a customization) or you need to add it to the robots.txt which you should do anyway so other bots don't try to access that. 

    I always update my robots.txt to not index any "sort" urls. I realize that I forgot to do it when I switched from another script to SE but here's what my robots.txt file looks like. If you do anything, you'll need to make it specific to SE.

    User-agent: * Disallow: /include/ Disallow: /theme/ Disallow: *sort_* Disallow: *when_* Disallow: *show_* Disallow: *location_* Disallow: /search/ Disallow: /user/ Disallow: /static/ Disallow: /module/ Disallow: /user/password/request/ Disallow: /user/login/ Disallow: /mobile/

     

    For example to disallow the return url you could have:

    User-agent: * Disallow: /login/return_url/

     

    But that sitemap generator will then need to also respect the robots.txt.

    • Moderator
    • 6923 posts
    September 18, 2018 12:24 PM EDT

    Info about robots.txt but you may find good tutorials for it as well. https://en.wikipedia.org/wiki/Robots_exclusion_standard

    • 106 posts
    September 19, 2018 2:45 AM EDT

    Thank you for the informations, so I added

     

    Disallow: login/return_url/

    it includes all the urls beginning by this, right ?

    And I made the same for all the urls with a 404 error. I precise that I modified or deleted some of these urls.

     

    Please let me know if I did something wrong, and if I should do the same or something similar for urls with 403 error.

    • Moderator
    • 6923 posts
    September 19, 2018 5:58 AM EDT

    Yes it will for bots that respect the robots.txt. Ask seao if their plugin does. If not, it should and should not take a customization as this is a very basic thing.

    As for the rest of the URLs, I am not sure. I suggest testing with google once the robots.txt is on your server. You can do a test in the google tools that has it fetch the file after it's on the server.