Google is not crawling to individual topic or post level immediately

[4mpb5]

Google sees my forum site at the top level with description.
But it is not crawling down to individual post or topic level immediately

I see individuall posts made to other phpBB forums show up within minutes on Google.

I do see Google listed as an authorized crawler in my forum.

So what I am doing wrong ?
It seems to take weeks or months before the posts on my forum come up on Google.

BTW are there any other (major) search engines other than Google on internet ?
All other search engines like evn Yahoo seem to crawl off of Google ?

There may be some in China etc but not in English language ?

What free sites do you recommend to list my forum, for maximum visibility on the internet globally ?

[AbuHossam]

Indeed, it may require some patience and valuable content that Google can recognize.

However, let me assure you that Google now considers forums, especially phpBB, to be of lower value.

As you can see from my dashboard, this is the case right now.

image f.jpg

Therefore, don't lose hope and keep working hard.

[Mick]

especially phpBB

Who says so?

[AbuHossam]

Who says so?

Based on my personal experience and what others have said, I initially found it hard to believe that all websites are treated equally by Google.

However, after conducting my own experiments, I realized that phpBB forums indeed require significant effort and work to gain the trust of search engines.

[Mick]

So this is your opinion?

[AbuHossam]

So this is your opinion?

If you see it that way/want to see it that way, then yes, it is.

[dextrous]

Our understanding is that there are lot of factors google considers when it ranks Domain & Pages, bottom line is content has to be valuable & there should not be lot of clutters. Manipulating/Spamming impacts the ranking negatively, it takes lot of time & patience to push the rank. I am not sure about the PHPBB or any other forum not being scanned by google, don't think that is true. Because google has a separate section "Discussions and forums" on the top 2nd or 3rd search results based on the user search words, so if your forum has all the words & intent then google brings up the page on the top.

[Forex Station]

Google sees my forum site at the top level with description.
But it is not crawling down to individual post or topic level immediately...

Our forum's latest posts and files get indexed every hour or two.

How old is your forum? Reason I ask is when my forum was under a year old I had it on the default BOTS permissions and our site barely got shown in search results and I found it a huge struggle to get topics indexed. After adjusting the Google [BOT]'s permissions in the ACP to allow it to ignore flood limit and give it access to search the forum, see profiles etc. that's when I started seeing lots of our pages indexed.

By leaving those settings over the past 7 years we have almost 200,000 pages of our site indexed.

(The recent drop was due to me experimenting with blocking certain pages (like "Thanks For Posts" list) using robots.txt)

GSC.png

[dextrous]

Do you just leave all 200,000 pages in the same forum & sitemap.xml?. Or do you move these into archive?. Thanks.

[Forex Station]

Sorry I'm not sure what you mean by "archive".

I don't use sitemaps anymore. I use to generate sitemaps using Screaming Frog for creating sitemaps but only saw a small boost in SEO and impressions so I stopped using sitemaps altogether.

[dextrous]

Ok, so you still have all these 200K topics sitting in the same forum?.

[JLA]

Be sure that your link rel= tags are correct on your pages (you can see example on our site pages) as well as your structured data markup.

As others mentioned - general site maps too.