Hi guys,

Here is my situation. google spider(known as googlebot)visited my site a couple of times but according to my wwwlog it appears that the file robot.txt is the only file googlebot interested in because everytime it just tried to read robot.txt but not any other files(like the content of my database which I want it to be indexed).

However the thing is actually I don't even have a robot.txt file in any directories of my forum. Could this be a problem causing google not indexing my forum? My understanding is if I do not use robot.txt then I'm welcoming google to visit/index everything I want in my site. But it looks like google did not find robot.txt then it left!!!

Look, I do have defined 404 error page.

Please any input would be highly appreciated...