I was analyzing the web traffic of some of our most popular member sites to try to understand how to better handle high traffic and I found an interesting statistic. Here are the top four IP addresses in the web logs of one member's web site (over a period of five days).

    IP ADDRESS        NUMBER OF LINES IN ACCESS LOG

    66.249.67.154     61729 
    74.6.8.113        47576 
    74.6.8.107        44039 
    83.231.136.9       6560 

What we're seeing are three IP addresses that are dominating the site with between 7 and 10 times the number of hits than any other IP address. Who are these people?!? Well, the first one identifies itself as Google and the second two as Yahoo. In other words, they are robots feeding search engines. The fourth looks like a regular web browser (running Opera!!).