Family Genealogy - Site Notes
   
   

TNG Traffic Analysis - User Agents

 

The User Agent is the browser or bot that accesses a web site. The agent is identified by a text string in the HTTP request, and includes version and operating system details. This analysis is for agents accessing these TNG web sites by PHP requests for the last 6 months (January - June  2020).

 

There were 1,633 unique agent strings seen (Table 1). When major types are grouped together, there were 488. The Bots were the most frequent individual requesters as expected.

 

The Knowledge AI, MegaIndex, and Seekport Crawler seem the have a monthly schedule. MegaIndex and Seekport have very consistent request quantities. All three do most of their scan in a few days.. They all use the same IP address for each scan burst.

 

BingBot is steady day after day, and month after month, and also Google but much slower. All others are trivial and inconsistent. Bing and Google bots both use a large range of IP addresses.

 

As for regular user access, Windows browsers were highest as expected. And iPad access is higher than I expected. Browser users have much higher overall request counts tan TNG counts, probably because they request all the icons.

 

Table 1 - Agents All TNG: Jan Feb Mar Apr May Jun
Knowledge AI (California) 152,377 150877 0 1,645 35,045 218 71,444 28,007
MegaIndex (RU) 128,307 126,754 0 31,805 31,802 0 31,804 31,343
BingBot (US) 100,164 91,047 11,492 17,258 20,061 13,938 11,867 16,601
 Seekport (DE) 77,010 76,808 24,705 896 24,703 897 24,703 904
AspiegelBot (Huawei, CN) 42,923 40,088 0 0 9,708 14,618 12,234 0
PetalBot  (Huawei, CN) 32,861 32,548 0 0 0 0 8,069 22,230
SerpstatBot (Hetzner, DE) 10,396 10,151 0 200 0 0 9,951 0
Windows Trident 57,905 5,433            
Windows; Safari 47,633 7,583            
Macintosh; Safari 25,348 5,123  
Linux, Android, Safari 19,061 21,505            
Windows; Firefox 17,209 11,121            
iPhone; All 13,613 1,663            
iPad; All 11,658 6,203            
GoogleBot 10,056 11,472 3,079 1,662 724 663 1,472 3,012
Macintosh; Firefox 5,693 1,881            
X11; Safari 5,344 1,244            
Unspecified 4,483 4,033            
SeznamBot 4,230 0            
X11; Firefox 4,062 1,879            
MS IE (Win 7-) 2,793 1,748            
TurnitBot 2,721 2,577 0 0 0 0 2,577 0
GoogleBot-Image 1,976              
AlphaBot 1,817 751 0 0 348 0 0 152
Opera 1,132 1.132            
netEstate 1,041 918 0 918 0 0 0  
CC Bot 882 761 302 145 4 145 55 0
Search Atlas SEO 726 438 0 0 438 0 0 0
Others 25,572 4,131            
Total 858,363 613,260            

 

.

Page modified: 08 Jul 2020 11:59:06 -0700

Back to Top Send comments or questions to Contact Us. Home Page