ATG Helps Customers Learn Their ABCs (Testing)
| Type: |
Competitive Intelligence Report |
| Analyst: |
R. Lerner,
|
| Report Date: |
March 17, 2004 |
|
| Current Perspective: |
Positive |
|
| Market Impact: |
Moderate |
|
| Vendor Importance: |
High |
|
|
 Summary |
March 15, 2004 -- ATG announced ATG Campaign Optimizer, an innovative
A/B testing application. ATG Campaign Optimizer allows business
managers to test multiple promotions, products, and creative
simultaneously, so that they can know which choice best maximizes
conversions and revenue.
| Analytical Summary |
• Current Perspective: Positive on ATG’s
new A/B testing
application, ATG Campaign Optimizer, because the
product offers a strong
range of A/B testing capabilities, and because ATG
continues to increase
the number and range of products for customer
self-service.
• Vendor Importance: High to ATG,
because the ATG Campaign
Optimizer potentially opens up new customers to
the company, and because
the product is one of the core pieces of the
company’s e-commerce self-
service initiative.
• Market Impact: Moderate on the e-
commerce market, because while
there are a number of products, services, and
companies providing some
form of A/B testing capability, ATG comes to market
with a strong
product. Still, the company does not yet have a
substantial customer
base for the product.
|
· Global 2000
· Large Enterprises
· Other
· Small to Medium Enterprises
| Perspective |
| Current Perspective |
: Positive |
We are taking a positive stance on ATG’s new A/B
testing application,
ATG Campaign Optimizer. ATG continues to
redefine itself as a provider
of self-service e-commerce applications, and this
offering is an
important piece of this effort and, to some extent,
underlines the
company’s overall self-service push. While ATG
brings a lot of
sophistication to what is a mature concept, the
product is competing
with similar offerings and in-house A/B testing
efforts.
A/B testing, sometimes called split testing, is an
idea that probably
derives from mail order companies. It refers to the
manner in which
these companies would split their marketing mail
into discrete batches,
after which the companies would send out the
batches with different or
varying market messages. On the basis of the
returns, they could track
the effectiveness of these messages and hone
their marketing
accordingly.
ATG Campaign Optimizer works more or less on the
same principles,
although it had been productized and is far more
sophisticated than
these earlier A/B testing efforts. In essence, ATG
Campaign Optimizer
enables users to test, measure, and analyze two
or more promotions,
products, and content in order to measure
effectiveness, to track user
navigation, and so forth. The product also enables
users to test
multiple site elements (e.g., images and price
points) simultaneously,
while at the same time giving the user control over
how visitors are
split between choices, which visitors are tested,
which are control
groups, and when the tests start and stop. Among
other features, the
product offers Slot Testing (which enables users to
rotate products,
promotions, advertisements, and so forth into a
specific location on a
page [slot] based on weighted percentages),
Promotion Testing (which
enables users to test multiple promotions to
determine the greatest
click-through), Location Testing (which allows users
to test for the
optimal page location for pricing information,
product information,
navigation, and other features), Dynamically
Served Content versus Fixed
Content Testing (enabling users to determine
whether predetermined
content produce better conversion rates than
dynamically-generated
content that is based on a customer's buying
habits), and Functionality
Testing (which enables users to determine which
functionality – e.g.,
search engines, recommendation engines, etc. –
performs best on an
organization’s site). ATG also provides other
features such as Change
Price Location, as well as the ability to run multiple
tests
concurrently, combine multiple tests, and review
content in a set-up.
Overall, this is a strong, sophisticated offering. ATG
codifies, so to
speak, functionality that is not often found in a
single solution,
functionality that is applicable to more than mail
order companies.
Indeed, the product may find a home in virtually
any vertical, and it
should therefore open ATG’s door potentially to
new customers.
Significantly, ATG has designed the solution for the
business user,
which helps differentiate the product and should
make it all the more
attractive, for it is the business user, not the IT
department, that has
the most at stake in terms of an organization’s
marketing initiatives.
In fact, empowering business users, people that
can react quickly and
that understand what is needed for successful
marketing, is tantamount
to increasing the effectiveness of almost any
marketing initiative
(among other things, the product’s UI, which
leverages a set of wizards,
was designed specifically for the business user).
Moreover, ATG also
leverages its traditional strengths, such as
personalization, to empower
users to segment the audiences that see the
tests; ATG also gives users
the ability to run simultaneous comparisons, which
save time and
potentially increase the effectiveness of such tests
(if an organization
is marketing around a special event, it may want
the ability to run more
than a page one day and another the next).
Among the features most
likely to generate interest are the ability to
determine whether
predetermined content is better for conversion
rates than dynamic
content, and Slot Testing and Location Testing,
because they enable
users to determine optimal page location.
Now, while ATG comes to market with a strong,
appealing product, it is
not entering an empty market. A/B testing has
been around for some
time, and it is competing with vendors that have
established products in
the market, such as Unica and E.piphany, and also
in-house solutions
that rely on analytic tools from companies such as
SAS and SPSS. On the
other hand, ATG does not detail either the price or
the requirements for
the solution. The product is not a standalone
solution but has been
integrated into ATG Commerce (a positive) and,
significantly, it
requires version 6.2. Users of earlier versions will
have to upgrade to
gain the benefits of the product. Finally, ATG has
not made a plea to
smaller business for this product. Small business
would benefit from
the product, although pricing will certainly be a
consideration.
|
| Positives and Concerns |
| Competitive Positives |
• ATG Campaign Optimizer enables users
to test, measure, and
analyze two or more promotions, products, and
content in order to
measure effectiveness, to track user navigation,
etc. The product also
enables users to test multiple site elements (e.g.,
images and price
points) simultaneously, while at the same time
giving the user control
over how visitors are split between choices, which
visitors are tested,
which are control groups, and when the tests start
and stop. The
product is pre-integrated with ATG Commerce.
• Among the product’s features that are
likely to elicit strong
interest are Slot Testing and Location Testing. Slot
Testing enables
users to rotate products, promotions,
advertisements, and so forth into
a specific slot or location on a page based on
weighted percentages.
Location Testing allows users to test for the
optimal page location for
pricing information, product information, navigation,
and other
features. Dynamically Served Content versus Fixed
Content Testing
should also generate interest, because it enables
users to determine
whether predetermined content produce better
conversion rates than
dynamically-generated content.
• ATG leverages its traditional strengths,
such as
personalization, to empower users to segment the
audiences that see the
tests; ATG also gives users the ability to run
simultaneous comparisons,
which save time and potentially increase the
effectiveness of such tests
(if an organization is marketing around a special
event, it may want the
ability to run more than a page one day and
another the next).
• ATG has designed the solution for the
business user, which helps
differentiate the product and should make it all the
more attractive,
for it is the business user, not the IT department,
that has the most at
stake in terms of an organization’s marketing
initiatives. In fact,
empowering business users, people that can react
quickly and that
understand what is needed for successful
marketing, is tantamount to
increasing the effectiveness of almost any
marketing initiative (among
other things, the product’s UI, which leverages a
set of wizards, was
designed specifically for the business user).
• ATG made this announcement in tandem
with other, related
announcements, all of which strengthen its ability
to address e-
commerce, self-service needs and, ultimately, make
ATG’s Customer
Experience Platform (formerly the ATG Relationship
Management) more
attractive. For example, ATG announced ATG
Adaptive Customer
Assistance, which gives users direct answers to
their questions by
leveraging natural language search, a structured
unique answer
repository, and ATG's personalization capabilities
(see “ATG Jumps into
Customer Self-Service with a Personalized Touch,”
March 15, 2004). ATG
also released updates to ATG Portal, ATG Content
Administration
(formerly ATG Publishing), and ATG Adaptive
Scenario Engine. ATG
includes support for the JSR 168 for JDK 1.4.
|
| Competitive Concerns |
• While ATG comes to market with a
strong, appealing product, it
is not entering an empty market. A/B testing has
been around for some
time, and it is competing with vendors that have
established products in
the market and also in-house solutions, which are
typical of smaller
organizations.
• This is not a standalone offering. It
requires ATG Commerce
and, significantly, it needs ATG version 6.2. Users
of earlier versions
will have to upgrade to gain the benefits of the
product.
• Pricing was not announced.
• ATG has not made a plea to smaller
business for this product.
Small business would benefit from the product,
although pricing will
certainly be a consideration.
• Among its other new product
enhancements, the company does not
yet support Web Services for Remote Portlets,
which is garnering some
attention.
|
| Recommended Actions |
| Recommended Vendor Actions |
• ATG should do more to attract small
businesses. While larger
organizations will likely be the main market for the
ATG Campaign
Optimizer, small organizations will likely be
interested, and indeed
small organizations perform A/B testing, albeit
probably informally.
Pricing breaks to attract them may be a possibility.
• ATG should also do more to highlight the
services that it can
provide with this solution. Despite the fact that the
product is geared
to the business user, the business user
experienced in A/B testing will
still probably need help in designing tests that are
effective. Indeed,
ATG needs to temper expectations with this
product, for the improvement
of conversion rates, for instance, may be
incremental and not dramatic –
and if users come to the product with
unreasonable expectations, the
product may be blamed for any perceived failures.
• ATG should highlight the fact that A/B
testing is not a one-off
effort, but an effort that takes time (i.e., it takes
time to hone the
proper message) and that testing continues over
the lifetime of product,
Web site, company, etc. ROI therefore becomes
attractive, and the
initial cost of the product seems quite reasonable.
• ATG should also do more with best
practices.
• ATG should offer incentives for customers
to move up to ATG 6.2
in order to implement this and other new solutions
from the company.
|
| Recommended Competitor Actions |
• Small vendors with A/B testing products
should be concerned with
the ATG Campaign Optimizer, since the company is
providing a wide range
of features and functionality and – especially –
directing the product
to the business user.
• Vendors in general should place more
emphasis on the business
user. Not only does such emphasis increase the
speed and effectiveness
of such testing, but it also empowers the people
who have the most at
stake in successful e-commerce.
• Competitors such as Blue Martini, Fort
Point, Optimost,
WebTrends, Vertster, and others should highlight
their capabilities for
enhancing marketing and merchandising
initiatives. They should also
downplay ATG by suggesting that the company
lacks the experience that
they bring to market.
• Portal competitors should downplay the
new product and other ATG
initiatives, suggesting that the company is once
again changing gears.
They should use ATG’s push on self-service and
applications as a sign
that it is leaving the portal market.
|
| Recommended End User / Customer Actions |
• ATG customers should be pleased with
the company’s new products
and self-service push. ATG is providing some
strong products in the
arena of customer self-service, and it intends to
continue developing
and enhancing these capabilities. ATG is serious
about such
initiatives, and it is positioning itself nicely in e-
commerce market
with these.
• Organizations with home-grown
solutions should consider the ATG
Campaign Optimizer. ATG is providing a rich
solution that should be
more effective than in-house solutions, particularly
because of the
tendency of organizations to assume that one test
or set of tests
determines everything. Markets and audiences
shift over time, and a
solution like ATG can enable customers to
accommodate such changes. In
fact, organizations often lack the sophistication
that is inherent in
ATG’s product.
• Prospects should inquire about services
that go along with the
ATG Campaign Optimizer. Users may need some
help in designing effective
tests. Users also need to temper their
expectations with this product –
the improvement of conversion rates, for instance,
may be incremental
and not dramatic. If users come to products of this
sort with
unreasonable expectations, they may be
disappointed.
• Prospects should also be aware that A/B
testing solutions are
not a complete substitute for empirical user
testing. ATG’s product,
however, complements such testing.
|
Event Assessment
All materials Copyright 1997-2010 Current Analysis, Inc. Reproduction prohibited without express written consent.
Current Analysis logos are trademarks of Current Analysis, Inc. The information and
opinions contained herein have been based on information obtained from sources believed to be reliable,
but such accuracy cannot be guaranteed. All views and analysis expressed are the opinions of Current Analysis
and all opinions expressed are subject to change without notice. Current Analysis does not make any financial
or legal recommendations associated with any of its services, information, or analysis
and reserves the right to change its opinions, analysis, and recommendations at any time based on new
information or revised analysis.
Current Analysis, Inc. 21335 Signal Hill Plaza, Second Floor, Sterling,
VA 20164. Tel: +1 (703) 404-9200. Fax: +1 (703) 404-9300. or 2 rue Troyon, 92316 Sevres Cedex, Paris, France, Tel: +33 (1) 41 14 83 17 http://www.currentanalysis.com
|
|
|
