Benchmark methodology and definitions

  Benchmark methodology and definitions



Vertain's benchmarks rank groups of websites on weekly averages of uptime and speed. By default, data for the most recent week is displayed, sorted by speed. You can select previous weeks, or sort the list by uptime instead of speed.

The benchmark data is collected by Vertain's main MoniQA™ website monitor, which uses a 6000 Kbps T3 connection (the same monitor used for our website speed testing service). This monitor visits each benchmark website about 18 times per day, measures the page download time, and notes any errors, timeouts or other failures.

The benchmark data reflects the relative performance of the respective Web servers, rather than variations in geographical locations or local network conditions.

The benchmark data columns display the following information:

Average Speed: The (weekly) average of the delivery time (in sec) of the complete content of the website's home page to our monitor's web browser. In the case a throughput index, this column shows the average delivery time of a single file common to all the servers in the benchmark.

Baseline Speed: The speed chart of many (if not most) websites exhibit a character similar to the following chart:



This type of chart appears as a "flat baseline" with occasional "spikes". This kind of speed chart can be interpreted as follows: the "flat baseline" corresponds to the speed at which the server can deliver the content under "optimal" conditions, and the "spikes" correspond to occurrunces of "less than optimal" conditions, when content delivery is slowed down (for example by overloading or a hardware issue, at the server itself, or on-route from the server to the monitor). If our statistical analysis indicates the presence of a consistent baseline, the Baseline Speed column lists the speed of that baseline (which is obviously always faster than the average speed); otherwise, this column shows "NA".

Slowdowns: The "speed spikes" can significantly effect the average speed of a website. For example, in the speed chart shown above, the average speed reported for the website is about 6 times slower than the baseline speed (1,3 vs. 0.2 sec). Obviously, a large difference between the average and the baseline speed warns us of a serious degradation in a server's performance. Another parameter that may indicate such performance degredation is the Slowdowns. This parameter is the percentage of our visits to the website, during the week, in which the speed was slower by 50% or more than the average speed. A large value means that a large portions of the visits are significantly slower than the average speed. For example, in the speed chart below, the average speed is 0.8 sec, and there are 7 visits slower than 1.2 sec (50% slower than the average), hence the Slowdowns rate is 7%.


Uptime: The percentage of successful visits during the specific week (i.e., visits in which no error or timeout was detected).

Weeks of 100% Uptime: The number of weeks, during the last 10 weeks (or less if data is not available for 10 weeks), in which the Uptime for the website was 100%. The value may range from zero (not a single week of 100% uptime in the last 10 weeks) to a perfect 10 (not a single error or timeout in the last 10 weeks).

Availability Errors: The percentage of visits in which the web site could not be found or accessed (e.g., the server was down or a DNS Error was encountered).

Timeouts: The percentage of visits in which we our monitor was able to access the website, but the page download was not completed within 25 seconds.

Content Errors: The percentage of visits in which we our monitor was able to access the website, but an HTML error was encountered.

Free Speed Test
Home
Clients
Services
Benchmarks
Technology
Company
Contact Us

©2001-2015 Vertain Software. All rights reserved. Site Map Terms of Use Privacy Policy