Welcome, Guest: Register On Nairaland / LOGIN! / Trending / Recent / New
Stats: 3,153,267 members, 7,818,924 topics. Date: Monday, 06 May 2024 at 08:14 AM

How Long Does It Normally Take Search Engines To Crawl A Site? - Webmasters - Nairaland

Nairaland Forum / Science/Technology / Webmasters / How Long Does It Normally Take Search Engines To Crawl A Site? (2342 Views)

How To Make Google, Yahoo, Bing To Crawl Your Wapka Site / 5 Lies You Need To STOP Telling Yourself As A Site Owner / List Of Top Search Engines In The World (2) (3) (4)

(1) (2) (Reply) (Go Down)

How Long Does It Normally Take Search Engines To Crawl A Site? by 212blaze(m): 7:52pm On Jan 30, 2010
i submitted a site to google days ago and it hasnt been listed yet. and i heard something about a robots.txt file, can someone please explain to me??
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 5:15am On Jan 31, 2010
In seconds nomally and some cases 24 hours, submited mine to google when next i checked it was there as in that same moment,

Try checking your meta tag and also if it ain't good, work on it, that should work it out!!!
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 11:54am On Jan 31, 2010
robots.txt are those places you do not want robots to crawl.
Its a file you can create in your root directory and include the urls you do not want robots to index.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by 212blaze(m): 7:50pm On Feb 01, 2010
its been days now and the site hasnt been indexed. is there any special meta tag that i need to put??
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 11:46pm On Feb 01, 2010
Yes na, the description, and all those complete meta tags and keyword meta tag, with somethig that can easily be indexed as in you will fill it with a well placed tag.

Check www.marknollis.com for an example
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by 212blaze(m): 3:04pm On Feb 02, 2010
wow, mehn your site is fresh. i am definitely going to subscribe.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 11:36pm On Feb 02, 2010
Thanks bro,

Doing more work, to make it great site and far more a great web application, for subscribers to get what they want,

Hope you have gotten the search engine thing done, a look at my source code could help u with the meta tag issues!!!
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by mamagee3(f): 12:17am On Feb 03, 2010
It depends if you're using a highspeed internet.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 12:21am On Feb 03, 2010
@Mama gee. You are wrong, internet speed has absolutely nuffin to do with google indexing ur site
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by mkwayisi: 10:17am On Feb 03, 2010
For me I'd say instantaneously! I developed a site, never sought a link from any site, never submitted it to Google or any other SE but found its way into their index after 48 hours. Amazing, huh! I'd say just do the right stuff and the SEs will be chasing your site. No need to buy links, you can do it on your own.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 11:47pm On Feb 03, 2010
Ya, i know what u are saying is possible, but that is luck,

See, when i got done with mine, hardly can any proffesional say it's homepage meta ain't ok, but it wasn't indexed until i submitted it,

See, its the meta tag that matter and no other right thing xcept u mean prayer and fasting cry
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 12:15am On Feb 04, 2010
@DonPuzo,
You have been making a lot of noise here like you are SEO expert.
If you want to show off your site here, just do it as many times as you wish and then do other things.
[by the way, nice site. good job.]

What did you search on google ?

If you develop a site and publish it , you can see it the next day on google index if you search for the site.

eg.  yoursite.com as search keyword. How about search for the contents of your site and see how many of them google have indexed ?

try    site:yoursite.com  and check how many urls you have there.
Meanwhile You need a site map. Thats the first thing you should create and add to your site. create a file for it in your root directory. Name it sitemap.xml.
generate xml links from http://www.xml-sitemaps.com/  and add the contents in the file you created. Use google webmasters tools to submit your site to google and check how many urls u have indexed in google. I run custom sites too apart from my forum.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 12:34am On Feb 04, 2010
@Sam. i don't see the reason why you are trying to raise a beef,

Ok. is it a matter of connection speed? So i dismissed that point,

Ok, you just spoke on submitting site to google webmaster, is that not what i spoke about,

Whats ur point, the whole sitemap shit sucks, as long as a page meta tags are well written, a search engine can index it, either it is submitted, or automatically crawed, , Search Donpuzo, and you will see my site search Imonitie Nosa Mark and you will see my site, cause they were written in the meta tag, if any of this key word ai'nt there i bet it wont find the site upon search,

Men quite the thinking that i am claiming seo guru, just trying to make people understand simple facts
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 11:56pm On Feb 04, 2010
Think what you are saying doesn't make sense to me, Although i do know many sites have maps, but on the factual part, meta tags are most crucial to seo searches, whether they are automatically inserted or watever,

Lets face fact, never read on sitemap as a way to make search engines index site,

Site like yahoo and facebook paid search engines, that why almost alll shits there are indexed, But for the free one, you simple make the tags well placed, and also submit to google or the search engine,

Think you are trying to have an unneccesary issue with me, so i will rather not respond to your words of insult of misplacement of my personallity,

Diss me and you ain't gonna get a reply for it!!!
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 2:02am On Feb 05, 2010
hmm, i think you totally misunderstood me here. Diss you for what ?
What did you do to me to warrant dissing ?

As long as you get on this ride of seo-know-it-all, You must be prepared for criticisms.
Don't get me wrong, You are no big shot to attract my dissing So Do your thing.
But don't channel seo solutions to only meta tags. Other methods worked for people.

SEO is a myth. How Google index pages has not been revealed yet by google,
We are all at a blind spot on that. Like i said b4, Your search key words can easily be found because you are searching for what you added to your meta tags.

its like putting money under the pillow and then going back in the morning to search for the money under the pillow.
You will definitely fine it. Expansion is the issue with seo.
What i am saying is that if you add sam milla in your meta tags, You will find sam milla on your website by searching for sam milla in google search.
But if you add only sam in your meta tag, your chances of finding it on your site in google search will become lower.
so once more, embrace sitemap but do your meta tags also
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by 212blaze(m): 3:16pm On Feb 05, 2010
mehn you have alot of javascript on your site. your code is very neat and im just tripping for your knowledge
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 3:22pm On Feb 05, 2010
212blaze:

mehn you have alot of javascript on your site. your code is very neat and im just tripping for your knowledge

which site ?
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 11:59pm On Feb 05, 2010
OK i get you, see i think many of you are very rude in your posts and think you guys are so good, i never claimed seo guru, seo is not an issue for me to worry my head on, rather i spend more time getting and learning new conventions and coding,

Ok, come to think of it, many of you dissed segsalerty, that he is a talkative, a critic and a nobody, but the fact is now he is sun java cerified and a great person, very succesful,

So, now making a refrence to me as not a big shot, i think that ain't right. and calling me all sorta unneccesary things and missunderstanding my point is totally unapprciated.

Without boasting, if many of you get to meet the real me off nland you will understand things,


I am no small boy in this web world and also in IT,

I have seen convention, styles, skill, and all there is to it

Please i will appreciate if you respect yourself and avoid unneccsary critism of me and my personallity, cause i aint a seo-know-it-all
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 9:56am On Feb 06, 2010
stop sweating donpuzo, i dont take
when i say u are not big shot, it means you are not big shot.

big shots in this web business are the likes of mark zukerberg, steve jobs, myspace, hi5 etc.
i m not included. so dont take it personal.
keep learning and doing your thing. you will succeed by that and not by attacking any criticism aimed at you.
when people criticize me, i try to correct myself and not attack them.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 12:10am On Feb 07, 2010
Men, you never criticized me or any thing about me.

And please for the last time never raise such issues of big shot or not near me
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 1:01am On Feb 07, 2010
Donpuzo:

Men, you never criticized me or any thing about me.

And please for the last time never raise such issues of big shot or not near me

YOU ARE NOT A BIG SHOT.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by ade9ja(m): 11:30am On Feb 11, 2010
Yes great work has to be done on s.e.o but all to be left to the search engine, your is just to submit to google and do some meta tag work, for example you can type in "naijavoice" as a keyword on google search bar ang you will be impress with the result ti bring out.

www.naijavoice.com Home of freebies
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 4:44pm On Feb 12, 2010
Sir Ade, thanks,

How far shey you don see that thing for the affiliate page.

I need you help on my lappy ohhhh
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by ade9ja(m): 8:25am On Feb 13, 2010
@ Danpuzo, yes i have seen it on the affiliate page, thanks man, what problem are encountering on ur laptop, call me or you can also post it on www.naijavoice.com under the computer board and i asure that in 24hrs it will be resolve.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 12:01pm On Feb 13, 2010
^ Lol Ade.
Dont you think you should wait to know what the problem is before giving a 24hour solution time frame.
It could a problem that needs hardware change or something you cant solve by giving advice or configuration.
Hear him out first to determine what the problem is, then you can talk about how long it will take to solve.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by kamsiyonna: 12:24pm On Feb 13, 2010
Sam Milla, abeg o, Dun Puzo don snatch ur girlfriend before? grin
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 12:41pm On Feb 13, 2010
kamsiyonna:

Sam Milla, abeg o, Dun Puzo don snatch your girlfriend before? grin
This is not about Donpuzo. I have settled with him. I am just responding to ade. Ade wanna help the guy out on his lappie, which is very good. But he gave a time frame when he dont yet know what the problem is.
I do help out a lot online too. ade knows that too. but i like to know the problem first.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by kamsiyonna: 1:14pm On Feb 13, 2010
i was referring to earlier threads
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by SamMilla1(m): 4:05pm On Feb 13, 2010
oh ok. dude seems to get upset over nothing. so i tagged along to see how he self-destruct.
he was wise enough to ignore my last ref though. good job don.
meanwhile you will like me when you get to know me puzo.
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by ogbongzky(m): 10:36pm On Feb 13, 2010
There are lots of factors involved and in 2010, Google has stated that loading time will be a major determinant of how soon a site will be crawled. Just relax and give it like 2 weeksna nd am sure it will be done provided you ahve done your own assignment. Articles on my blogger blog keeps popping up in top 10, all because of series of SEO fixes have nailed into it,
Re: How Long Does It Normally Take Search Engines To Crawl A Site? by Nobody: 4:50pm On Feb 15, 2010
Thanks Ade, i go call you make we see, na smallest thing na, The issue na one stuff brake and e fit dampen by coding for some days if e lost, We go see sha!!!

@Kamsiyona. I hate when people try to be rude, and i apply maturity when handling people who sign-up to forums to insult peeps!!!

@Ogbogesky. Followed your last two post. Know you are good in the web,. Congrats!!!

(1) (2) (Reply)

My WHMCS Addons / Please: What Is The Full Meaning Of OP / How To Build Your Free Website Using Our Free Website Builder

(Go Up)

Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health
religion celebs tv-movies music-radio literature webmasters programming techmarket

Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Nairaland - Copyright © 2005 - 2024 Oluwaseun Osewa. All rights reserved. See How To Advertise. 47
Disclaimer: Every Nairaland member is solely responsible for anything that he/she posts or uploads on Nairaland.