|
Answers
Can you recommend a minimal "adequate speed" for our website?
There is no such thing as a "minimal adequate speed". What's adequate
for one website may be completely unacceptable for another.
You may want to look at some of our benchmarks, for example the
eService Index, where you
can see the wide range of speeds for different websites. Obviously, what
works for American Express (4 sec) or Weather.com (7 sec)
doesn't work for Google (0.3 sec). It's up to you to decide what is
an adequate speed for your own website. Our job is to measure it and
let you compare it to the speed of your competitors, partners, or other
site that you wish to compare with.
Why are your speed data different from those of other monitoring services?
Viewing a web page in a web browser is not a "one step" process,
but rather a set of back-and-forth "hand shake" interactions between
browser and server, which involves several steps (called "events").
Some web pages require only 3-4 steps, while some require 10-20
steps. Each of these steps can take anywhere from 0.01 sec to
more than 10-20 sec. The main reason why different monitoring services
report different speed for the same URL, is that they measure different
set of steps.
At Vertain we measure the entire sequence of steps required to
view a web page by a "human" user via a web browser. We believe
that this is the most useful result to report, because (1) it reflects
most accurately the actual experience of the users, and (2) it gives
our clients the most useful information on how to improve the speed
of their websites. If you wish to see these steps in detail in your
alerts, click Alert Management for the desired scenario, than
click Customize, select "Yes" for Include visit log and
click Save.
Why does a second speed test usually show faster result?
In order to duplicate the experience of "real" users, our speed test is
performed by Microsoft's Internet Explorer which (like other web browsers)
"remembers" the content of recently-accessed websites in order to speed up
access. When you request the first speed test, the entire content of the
requested page is loaded. If you request a second speed test shortly after the
first, part of the content may not be loaded again (as it is already
available to our browser), and the speed will appear faster. Since we use
a minimal size for the "temporary Internet files" and we visit a large
number of websites, your content will be remembered for only a few minutes.
If you do a second speed test more than 10 minutes later, the entire page
content will be loaded again.
Why do several speed tests on the same site sometimes show a widely different results?
The most common cause of wide speed variations is overloaded server or
overloaded server environment. For example, if someone else is downloading
a large music or image file from your server while you do one speed test,
and nobody is accessing your server when you do a second speed test, the
first is likely to show a slower speed.
Do you normalize the speed test to the size of the tested page?
The size of the page content (including images and other files) are one of
the main factors effecting the speed of a website. The relationship is not
exactly linear. For example, twenty files with total size of 100,000 bytes
will load slower than one 100,000 bytes file. However, our results are not
"normalized" by the size of the content. We measure the time required to
load the content, regardless of size. So if you test two pages hosted on
your server, one sized 10 bytes and the other 100,000 bytes, the first will
show faster speed. You can try this yourself.
How can I measure the speed of my Internet connection?
Vertain is the leader in website and server speed measurement, but we
do not provide a tool to measure the speed of your Internet connection.
There are many other companies that provide this service. To locate
such services, please go to your favorite Internet search engine, and
search on a phrase such as "measure internet connection speed".
Could timeouts result from problems with your monitor, rather than problem with my server?
Timeouts may be caused by a problem on any of the nodes along the
connection route between our monitor and your server, as well as
the monitor itself or your server. Usually a TraceRoute test positively
identifies the node that causes the timeout. However, this will not
work if some node along the route is set up to block TraceRoute packets.
How do I prevent your monitoring visits from interfering with my Google Analytics reports?
Since our monitor visits your site via standard Web browser, your
server sees our visits just like any other visits by "human" users.
If the URL that we monitor is within your Google Analytics "funnel", our
visits will count as "entrances", and will "distort" your Google Analytics
reports. Here is a simple method to prevent our visits from
influencing your Google Analytics reports:
1. Log into your Vertain account, go to the "Visiting Options"
page for the desired scenario, and add a special query
parameter to the "URL to monitor" field. For example, if we are
monitoring the URL http://www.myDomain.com, change it to:
http://www.myDomain.com?vertain
2. Make sure that your server ignores this parameter, in other words, it
should handle the new URL exactly as it handles http://www.myDomain.com.
3. Tell Google Analytics to ignore visits containing this parameter, by adding
a custom "exclude filter" with the regular expression ^vertain.
For step-by-step instructiona on how to do that, please see
How do I create a custom filter?.
How do I prevent alerts during server maintenance periods?
Set up your server so that during maintenance periods it return a page
with a message such as "We are currently under maintenance". This is a
good practice, better than returning an error such as "Page not found".
Let us know what the message is and we'll tell our monitor to consider
this a valid page and not deliver an alert (or count this as an error).
Another option (not available under the "Economy" account) is to set
up an alert delivery schedule that excludes maintenance periods.
Log into your Vertain account, go to My Scenarios --> Alert Management
--> Schedule, select "Deliver alerts only when on duty", and enter
the daily "shifts" to exclude your maintenance period. This will prevent
alert delivery, but our visit log will show error for these visits. This
method is useful for managers who want to know the total "down time",
including maintenance time, but do not want to be alerted during maintenance.
Should I set up to be alerted only if timeouts are detected by more than one monitor?
If your site is being monitored from more than one location,
you can set up to be alerted ONLY when a problem is reported by
more than one monitor (to prevent "false alerts"). Before you
do this, please be aware of the possibility that a specific geographical
region may experience persistent timeouts or availability issues, caused by
a problem in a specific node (such situations are not uncommon).
So if you supress "single monitor" alerts, make sure you check your
incidents online regularly, so you are informed of such a situation.
What should I do if my "uptime" is less than 100%?
Every server and/or its hosting environment must be interrupted from time
to time for upgrade and maintenance. This may result in an uptime of less
than 100% on our Uptime Report.
In fact, a consistent uptime level of 100% over a long period may suggest
to some people that the server is not upgraded or maintained regularly
(which may result in security risk or less than top performance), or
that the uptime data is not "honest". So an uptime level of a fraction of 1%
below 100% should not be a problem. However, if your server shows an uptime
level below 99% for more than 1-2 weeks in a row, you should invest some
resources in diagnosing and fixing the problem.
Why monitor from the user's perspective?
Even if each component and infrastructure level that make up a
complex transactional Web site operates correctly on its own, it
is possible that the performance of the overall, integrated system
will be less than optimal. The most effective way to measure the
overall performance of your website is to monitor the ability of end
users to perform the specific activities that the site supports.
If the intended user of the system is able to perform such activities,
the site has fulfilled its purpose. On the other hand, if a significant
portion of end users (or their attempts) fail, the system's performance
is inadequate. Vertain monitors your site by exactly duplicating the
activities of end users through a standard Web browser that measures
how well the server responds. As a result, the data we collect
accurately reflect the quality of the experience of the users.
What is a Scenario?
A scenario is a set of specifications that determines how a website
is monitored. At a minimum, a Scenario requires an inital URL to visit.
Optional specs may include a list of items to verify on each page,
a list of data items to be entered, and a link or button to be clicked
on a page in order to proceed to the next page (for multi-page scenarios).
Some scenarios specify the visits very strictly, so that all visits
are exactly the same. Others are more "loosely" defined, resulting in
different "path" through the site on each visit. The simplest scenario
is one where only the Start URL is specified, and only one page
is visited. To validate the content of the page, the scenario may
specify the title, page signature, specific links, buttons, data entry
fields, pictures, etc. If the page is a simple data entry form, the
scenario may specify the data to be entered in the different fields,
clicking the "Submit" button, and checking the web server response to
verify that the data has been submitted correctly. A more complex
scenario, consisting of 5 to 10 pages, may verify that the web site
allows customers to select retail items and successfully check them
out and have them shipped to the customer's address.
Unlike other website monitoring tools, which are script-driven,
Vertain's scenarios are data-driven. Scenario specifications are
stored in a database, and can be entered and modified through heirarchical
data entry pages, including a scenario level, page level and page component
level. At each level one can specify validation rules, data entry logic
and "click" logic using condition and value comparisons, branching,
random selections, and other logic elements designed to allow us to
mimic a real user or customer. Scenarios are easy to modify when the
site changes, and do not require script programmers to maintain.
How often should I monitor?
Should you monitor once a day, every 5 minutes, or anywhere in between?
If your primary monitoring goal is problem notification, ask yourself
how long you can afford to wait before being notified that your users
or customers cannot access your site or cannot conduct business on the
site. You may want to weigh the cost of more frequent monitoring with
the business loss that may result from broken site functionality.
If your primary monitoring goal is measuring uptime or performance,
you'll want to consider the desired accuracy, or the details of the
service level agreement that you are trying to enforce. For example,
to see weekly uptime expressed in percent (as shown on our
uptime report) with accuracy of 0.5%, you'll want to monitor at least
200 time a week, i.e., every 50 minutes.
In either case, changing the monitoring frequency is very easy, and you can
adjust it as often as you need to fit your specific needs. For example,
after launching a new site or deploying a new server hardware, or during
a busy season, you can increase the monitoring frequency. When the site
has stabilized you can reduce the frequency.
Should I monitor from different locations?
If your users or customers are served by a single server hosted in one
location, monitoring from a single location may be sufficient. If you
are targeting users in a specific geographical area, you may want to
select a monitor located in that area. However, please
note that speed and accessibility are governed as much (and even more) by
backbone infrastructure than by geography, so geographic location may
mean less that you may think.
If your users or customers are served by more than one server (whether
your own distrubuted servers or via content distribution verdors such as
Akamai), monitoring from different locations will give you much better
information on the speed and accessibility by users in different locations.
For example, if the bulk of your users are in the East and West Coast of
the US, you may want to select our San Francisco and New York
monitors.
What constitutes a server problem?
We record as an incident any visit in which the transaction
specified by a scenario could not be completed successfully. We
classify incidents into three main categories:
1. Availability errors occur when the server
cannot be located or accessed. This type of error is primarily caused by a
network connectivity problem or a technical problem with the Web
server or the web hosting service.
2. Timeouts occur when the server has been successfully
located and accessed, but the required content is not completely
deleivered within a specified time period. Most timeouts are
caused by a problem in the server or the server's infrastructure, or
by a software problem.
3. Content errors occur when the server returns the
wrong content (such as error messages or a wrong page), or when
some of the content is missing. Content errors are caused mainly
by software problems or server infrastructure problems.
What is a timeout?
The process of displaying a web page on your web browser is not a one step
affair that can either succeed or fail. Rather, it involves a sequence of
steps, all of which must succeed in order for the page to be displayed
correctly. The sequence begins with "finding" an appropriate IP address
via domain name lookup, then establishing an initial "contact" with a
specific web server, optionally via "load balancer" or some other
"redirection" mechanism within the hosting network infrastructure.
Once the specific server has been located, your web browser begins
a dialog with the server, in which the browser "asks" a sequence
of "questions" and waits for the server to provide appropriate "answers".
Finally, optional "client-side" processes (such as JavaScript functions)
may have to be executed in the web browser. The page is considered "complete"
when all the above steps have been successfully completed.
If the server cannot be located, or if the server is "down" (i.e.,
not operational), we report an "availability error". But what if the
server is successfully located, but then doesn't respond within a
reasonable time period to one of the "questions" in the dialog?
In this case we report a timeout. We are all familiar with long
waits, as we sit watching our broswer's icon spinning, until our patience
runs out. The length of time we are willing to wait for
a page to complete obviously depends on the importance of the page and
on how much time we have, but everyone has a limit. When this limit is
reached, we say that the page has timed out. Our
benchmarks indicate that timeouts are
as common as availability problems. Given that they are more frustrating
to users (since they take up more of the users' time) we consider them
in general more serious problem than downtime.
Timeouts are rarely caused by a slow Internet connection. Most timeouts
are caused by specific server or software issues that can (and should)
be diagnosed and fixed.
How am I notified when a problem occurs?
By default, we'll alert you by email on each incident. You can specify
several email addresses, some of which may be redirected to text messaging
and paging services. You may also select to be notified by a regular phone
call, to ensure immediate notification when not monitoring incoming email.
We can also deliver alerts to your customer service ticket system.
(If we don't support it, let us know and we'll integrate with it).
How do I schedule and escalate alerts?
By default, the alert recipient for a new account is the account owner
(yoursef), and an alert will be emailed to you on each incident.
When this default alert handling no longer serves your needs, you
can easily customize the alert delivery rules to fit your particular needs.
We support advanced alert scheduling and escalation rules, including our
exclusive "Hyper Visits" feature.
When is it important to monitor transactions?
If your website supports data entry forms or business transactions
(such as purchasing, reservations or banking activities), it is not
enough to monitor the server uptime or the availability of the home page.
You must also ensure that customers are able to successfully submit
forms or complete supported transactions. In facts, timeouts
or content errors on one of the transaction pages are usually more
damaging to your business than the inability to access the home page,
because the customer has already invested time in entering data or
selecting a product or an itinarary, and her time and efforts are now
wasted. Such errors may hurt business not only because of the abandonment
of the attempted transaction, but because of poor expertience that will
drive customers to a competing site.
Our eTailer and eService
benchmarks show that the levels of timeouts and content errors in the
online retail and service industries are consistently higher than
availability error levels. If you monitor only the server uptime
or the availability of your home page, you wouldn't know about these
important problems.
If the above statements are too "technical" for you, or if you are not
sure how important it is for you to monitor transactions, please let
us know and we'll be happy to explain this issue further.
Which transactions should I monitor?
It is rarely practical to monitor literally all the possible activities
supported by a complex Web site. Therefore, we recommend that you
initially monitor what you think are the most critical activities,
based on your own understanding of the business.
To find out which transactions have the highest priority, you may want
to list the different functionalities offered on your site, then sort
them by their importance in achieving the specific business goals of
the site. Maximizing on-line revenue may not necessarily be the highest
priority, even for an on-line retail site. The primary value of the
site to your organization may be, for example, helping customers find
store locations, browse item availability or get after-sale customer
service.
The goal of the monitoring may also be technical, for example, to
evaluate the performance of a legacy/Web integration link. In such
cases you may want to get our help in setting up a monitoring
scenario that focuses on the relevant technical activity.
How many Scenarios will I need?
This depends primarily on the range of "similar" activities that
you need to monitor. For example, you may need only one scenario
to monitor even a complex retail site that offers many types of
items for sale, since our monitor can select items randomly, thus
select a different item on each visit, and test your entire product
inventory over time. On the other hand, since the cost of monitoring
depends on the number of pages visited, it may be advantageous to set
up several scenarios, each one customized to test issues specific to
individual product categories, or related to specific technical
functionality. For example, monitoring a retail site may be more
cost-effective with two scenarios: (1) a one-page, simple scenario that
visits fairly frequently (e.g., every 15 minutes) and verifies that the
home page is available, and (2) a multi-page scenario that verifies the
checkout functionality.
Our staff will be happy to help you sort out the
different options and determine the best set of scenarios for your
specific needs and budget.
What are the basic features?
The basic features of our Service, included for all account types, consist
of visiting a URL at specified interval, performing the specified test,
recording the results (e.g., speed), and emailing an alert if an error
was detected. Users also have online access to performance reports,
account preferences and billing information.
What are Snapshots of Error-Producing pages?
When we encounter a content error, we can optionally store a "snapshot"
of the error-producing page, and include a link to this snapshot in
the alert that you receive. You can then view the snapshot by clicking
this link. The snapshot may contain information that will help you
diagnose and fix the problem.
What are Full URLs of Error-Producing pages?
When we detect a problem with your site, we can optionally store the
URL that "requested" the error-producing page, and include it in the
alert that you receive. You can then re-submit the request and may be
able to duplicate the problem, and help diagnose and fix the problem.
Even if this is not possible, the full "request" may contain
usefull information.
What are Hyper Visits?
Hyper Visits are additional visits that are automatically scheduled
shortly after an error is detected, in order to determine if an error
is a short-lived "glitch" or a repetitive, reproducible problem.
Each Hyper Visits is scheduled about a minute after an error is detected,
as long as the error persists. To illustrate, suppose you specify two
(2) Hyper Visits. If an error is detected during a regularly scheduled
visit, an additional visit will be scheduled about a minute after the
regular visit. If this additional visit is completed successfuly
(without an error), no more Hyper Visits will be scheduled. If, on the
other hand, an error is detected in the first Hyper Visit, a second
Hyper Visit will be scheduled, about a minute later. If you wish to be
alerted only if an error persists longer than some critical period,
specify the desired number of Hyper Visits, then set up an appropriate
Grace Period (under Alert Management / Scheduling).
For example, to be alerted only if a website error persists for more than
about 5 minutes, select 4 Hyper Visits and set up a Grace Period of 5 minutes.
What is a Ping Report?
When an "availability" problem is detected, a Ping report may help
determine whether the cause is in the server itself, in the domain name
server, or in the Internet connection of the server. A Ping report,
executed from our monitoring engine to the IP address of your server,
indicates whether a connection exists between the two. If the connection
exists, it is likely that the availability problem is caused by a
server issue or by a DNS (Domain name Server) problem.
Otherwise, it is likely that your server's Internet connection is
interrupted. Please note that Ping will indicate "no connection" if
ICMP
packets are blocked. For more information about Ping, click
here.
To activate the Ping Report feature, login to your account, go to the
Visiting Options page, enter the IP address of your server in
the field IP# to Ping, and click Save. When this feature
is activated, we will automatically execute a Ping test to the specified
IP address whenever we detect an availability error or a timeout.
The outcome of this test will be emailed to you in addition to the alert.
What is a TraceRT Report?
When an "availability" or "timeout" problem is detected, a TraceRT
report may help diagnose the nature of the connectivity issue. A TraceRT
report is similar to Ping's, but contains more detailed information.
It shows the path between our monitoring engine and your server, and the
access speed to the intermediate nodes. This information
may be particularly useful in understanding the cause of recurring "timeout"
errors, and in diagnosing connectivity problems revealed by negative
Ping results. It will also show DNS issues. Please note that TraceRT
will not work when
ICMP
packets are blocked. For more information about TraceRT, click
here.
To activate the TraceRT Report feature, login to your account, go to the
Visiting Options page, select "Yes" in the field
Do TraceRT on error, and click Save. When this feature
is activated, we will automatically execute a TraceRT test to the
start URL of the scenario whenever we detect an availability error or a
timeout. The outcome of this test will be emailed to you in addition to
the alert.
What are Custom Performance reports?
In addition to the standard reports (Visit Log, Incident Log and
Performance History), you can create Custom Performance reports
that let you see, at a glance, the "bottom line" of your
site's performance, as you define it. This "bottom line" is a number
from 0 to 100 called Performance Index, that summarizes the
error level, page and transaction speed, and repair time according
to your own priorities. To create such a report, you define your
goals for the items above, and specify the importance each goal
on a 1 to 5 scale. For example:
- Max total error: 0.2% (extremely important)
- Page speed: 0.5 sec (important)
- Transaction speed: 3.0 sec (less important)
- Longest repair time: 30 min (very important)
Once the goals are defined, you can view the report for the desired
period (a day, week, month, or custom period). The report displays
the Performance Index, showing if your goals have been reached.
In case the goals have not been reached, it shows the gaps between the
goals and the actual performance.
|
|
