Thursday, December 8, 2011

Marketo Analytics are disappointing

Yesterday, we did our last Marketo Quick Start session with our implementation consultant and the session was focused on Analytics and Subscriptions (Reporting).  Unfortunately, it was the most disappointing session that we've had.  In the sales process, I felt like I was very clear in terms of the types of analysis that I wanted to be able to do in Marketo and I was assured that it was possible to export the data from the Marketo Lead Management module into Excel and I could the analysis there.  I was really disappointed to see that the analytics capabilities are really limited and the user interface was not intuitive at all.

The user interface required you to know to click a specific button in a specific part of the window and there was nothing in the workflow or intuitive about the user interface that would lead you to take that action.  The terminology on the screens is unique to Marketo and has little relevance to terminology.  The more I deal with Marketo, the more I feel that the user interface is almost as poor as the user interface in Eloqua.  I would give Eloqua an F for its user interface and Marketo a D for its user interface.  Now I understand why Marketo wouldn't let us do a free trial of their product because we would have realized that their "ease of use" is not that good.

Back to the Analytics.  If you want to do some basic reporting, Marketo Analytics and Subscriptions are fine.  Subscriptions is simply Marketo's term for a report that runs on a regular basis and is automatically emailed to you.  In general, most of the reporting that is available in Marketo can be replicated and with better detail in  What Marketo is good at is looking at individual lead, contact, or opportunity behavior.  However, if you want to do any analysis of groups of leads, contacts, or opportunities to identify trends and averages, then Marketo doesn't do that very well.

We tried to look at two analyses and Marketo's reporting on both of them were not very useful. 

1) We wanted to run a report on new leads that came in from the website, what date they were created, and what specific content or form on the website generated the lead.  Marketo could easily show us how many website leads were created and what was the source of each lead.  However, when we tried to get the system to show us the create date in addition to that data, Marketo couldn't do it.  In, I would simply run a report with Create Date, Lead Source = Website, and Lead Source Name to get that information.  Marketo was unable to break down the data along those dimensions.  It could only generate a table showing the aggregate leads in that timeframe by the source of each website lead.

2) This is a more complex analysis, but my expectations had been set that Marketo could export this data into a spreadsheet where I could analyze it.  I wanted to do an analysis based on engagement level.  This engagement activity analysis can be used to segment your database to identify leads that are highly engaged and leads that have low engagement so you can potentially drive different marketing programs based on a lead's level of enagement.  Example, for someone that is unengaged, then send really interesting subject lines just to try to get them to open an email and click through or send them a survey to try to understand what they're interested in.

So we tried to run a report to identify all leads based on number of email clicks that they've done.  Example:  Identify all people that have never clicked thru on an email, all people that have clicked once, all people that have clicked twice, etc.  The only filter that was available in Marketo do this analysis was "clicked email a minimum of X times."  So if I want to get a list of everyone that has clicked only once, I would have to run three reports (everyone clicked mininum zero times, everyone clicked minimum 1 time, everyone clicked mininum 2 times).  Then I would have to use Excel VLOOKUP to compare all three spreadsheets to find only the people that have clicked one time.  Our database is so big that doing that analysis with 200,000 records would be painful.

Here are a couple other issues with Marketo reporting/analytics that people should be aware of:
  • Marketo doesn't have a way to filter out data based on IP address.  What that means is that when you're visiting your landing pages and clicking emails to for testing, your visits and clicks are going into the Marketo reporting.  Marketo doesn't have a way to filter visits from your IP addresses from their database.  What that means is that your visit data is going to be inaccurate.  Marketo's position was that your number of visits will be small relative to your total visits so the level of error should be acceptable.  Really disappointing that you can't filter out your testing from the Marketo data.
  • It's not practical to analyze the data in the Marketo Activity Log.  Marketo has an activity log that tracks every activity on a lead record including clicking emails, site visits, downloads, and field value changes.  So if an address is updated or information is entered into a field, that goes into the activity log.  This is great if you want to look at an individual.  The activity log is useless if you want to do any analysis to identify trends.  Example:  Analyze all Leads that converted into Contacts over the last 30 days to identify what activities drove conversion across those people.  Marketo is designed to look at each person individually.  It doesn't have the ability to analyze that group of people to identify any trends.  So I run the Activity Log report and it generates 100 people that have converted from lead to contact over the last 30 days.  It provides a list of each person.  You can click on each person and will display the detailed activity log for that person.  The activity log contains every single change that occurred to that lead.  So you have field value changes in addition to activity.  There's no way to export that list for all 100 people in the report.  What I would have to do to do any trend analysis would be to export the activity log as a spreadsheet for each individual resulting in 100 separate  spreadsheets.  Then I would have to combine all 100 spreadsheets to be able to do the trend analysis that I'm interested in.  You could do it, but it's not something that you would want to do ofter or with large numbers.  I was expecting to be able to export the data for all 100 people into one spreadsheet and to be able filter out field value changes - not possible in Marketo.
If we had done the Analytics and Reporting Quick Start at the beginning of this pilot project, we might have bailed at that point.  However, we're so deep into Marketo now, that we're probably going to just deal with the poor analytics capabilities and find manual workarounds.  If Analytics is important to you and you're considering Marketo, I would encourage you to look at the Analytics and Reporting aspects of Marketo early in any evaluation or pilot project.

Tuesday, December 6, 2011

Marketing automation evaluation criteria

Here is a list of questions that I used to evaluate the capabilities of the different marketing automation platforms.  When we were discussing the platforms with each vendor, they primarily focused on promoting their lead nurturing capabilities.  This list of criteria has strong emphasis on being able to effectively track, segment, and report based on customer activities.  Example, implement effective tracking to be able to analyze and score all customer behavior and then set up marketing programs based on customer engagement (activity) level.  Send one email to people that are highly engaged (actively reading email) and a different email to people that are not engaged (rarely open email).  Use different offers and messaging strategies based on engagement level.

Here's the criteria list that I developed:

Marketing Automation investigation

1.       Registration Forms:
o   Where do the forms sit.  We want the forms to sit on our website so we can track visits to pages, click paths, drop offs, and conversions.  Don’t want the pages to sit on a marketing automation platform micro site because we don’t get traffic and tracking if it’s not on our site.
§  Where do the Thank You pages sit and are there any issues adding Adwords tracking code to the Thank You pages
§  How do you protect content (asset) links.  You must be cookied in to be able to access the document/link.  If you pass the link to someone and that person is not cookied-in, that person will get either a short or long registration form
o   Go through the form behavior and tracking process for two scenarios:
§  Standard web form on our website passing data to the marketing automation system (double posting)
§  A form generated by the marketing automation system that is integrated with our website page.
o   Form integration with Adwords.
o   Tracking links:
§  Ability to parse data from tracking links built with Google URL builder
§  Capture the data and pass it into SFDC fields
§  Generate reports to show site visitors and conversions filtered / sorted by tracking link data
§  Preferred model is one landing page that parses data from the inbound link and then sets SFDC field values based on the parsed data.  Example:  have one landing page.  Inbound tracking link from Partner 1 sets SFDC lead source as Partner 1.  Inbound tracking link from Partner 2 sets SFDC lead source as Partner 2.
·         When is the data from the inbound tracking link detected by the marketing automation system.  Is it detected as soon as the visit happens or is it tracked after the form is submitted.  Can the form serve a different thank you page based on the data parsed from the tracking link?  Can the marketing automation system send a different confirmation email based on the data parsed from the tracking link?  (assume that once the parsed data is passed into SFDC, then that individual can be automatically added to a segment and lead nurturing campaign.
o   Gated forms:
§  Does it take custom programming to set it up
§  If there is a main gated form, how do you track the content that was downloaded and not just track the main registration form submission.
o   How will tracking work with a landing page that has multiple content items.
§  Example: Landing page with single form with 3 separate links for whitepaper, case study, and video.
·         Eloqua requires a tracking link on each piece of content.  For documents and videos, Eloqua requires a redirect page with tracking link on the redirect to get to the asset.
o   How to verify that the content downloaded data in the marketing automation system matches the campaign data in SFDC. 
§  Run a report to identify every email that has downloaded content in the marketing automation system by content piece.
§  How does document download tracking work
§  How does document download tracking work when the document is a not a webpage (i.e. the download is a PDF or exe file).
o   Is there any capability to create Personal URLs
o   Is there any capability for A/B testing that would automatically split traffic to each form based on visit percentages specified by the marketing automation user.  Example:  run an A/B test on a landing page and automatically send 75% of the traffic to the control landing page and 25% of the traffic to the test landing page
2.       Show us the email capabilities.  We’re currently emailing 200K people twice a week.
o   How to create emails
o   How to segment email lists
o   Is there any capability designed to make it easier to send A/B tests or different versions of emails to different sub-segments of lists.
o   How to stage emails to send at certain days/times
§  If we’re sending 200K emails, how long would it take the system to process and send all of those emails
o   Is there a way to stage email to send by time zone or state
o   Email reporting/metrics
o   What to do with our opt out list.  Is there a way to create an email suppression list without having to load all the opt out people into the marketing automation system (don’t want to have to pay for keeping these opt outs in the marketing automation system)
o   Can we track email activity history in the marketing automation platform and report on that data, but not pass the email activity history in SFDC.  Don't want to clutter SFDC activity history with email activity data.
o   Can we track every activity history except email activity in SFDC
o   Can we selectively trigger email alerts on opens/clicks by email campaign and by individual/segment.  (as opposed to an All On or All Off model)
o   How does the email system handle soft and hard bounces
o   What if any capabilities will the system provide to ensure the best possible email deliverability rates
o   How does the system handle duplicate email addresses
o   How does the system handle bad email addresses
o   Email preferences:
§  How to implement email preferences by content type
§  How to implement email preferences by email type: mobile, html, text
o   Email Opt Out:
§  Where are email opt outs tracked
§  Can we provide an email opt out process that offers an Opt Down choice before offering an Opt Out.
3.       Segmentation
o   Ease of building segmentation based on either demographic (field) data, campaign data, or activity data
4.       Reporting and analytics capabilities:  
o   ability to report and segment on activity data over any timeframe.  I’d like to see an analysis that allows me to segment the database by level of engagement:  identify those people that have opened only once, clicked once, opened twice, clicked twice, etc over any given timeframe.
o   Ability to dump all activity data into an excel spreadsheet.  Can I analyze the data in excel instead of using any analytics capabilities in the marketing automation platform.
o   Is there any way to cluster people based on tracking data and measure their activity.  Example, could we analyze everyone that came in on a certain Adwords ad group and identify the most frequently visited pages and the most frequent activities.
o   How does the system handle analysis when the customers being analyzed reside in both Leads and Contacts in SFDC.  Are there any constraints depending upon whether someone is a lead or contact in SFDC?
o   Are there any capabilities that make statistical analysis easier?  (linear regression?)
o   Is there a data warehouse behind the system and can you access more detailed data in the warehouse using SQL queries?
§  If there is a data warehouse, is each company’s data in a separate date cube?
§  Is there documentation on the data warehouse?
5.       Real-time site visit alert capabilities
o   What information can be inferred from people that are unknown (haven’t registered)
o   How does the data compare to the data in Leadlander reports.  Do the visit counts match and to what extent can the system infer company names from the IP address.
o   Can the marketing automation platform provide any click path analysis?  Can it trigger an alert or workflow based on a click-path (sequence of web pages visited)
6.       Lead scoring:
o   ability to use mathematical formulas, to apply aging factors, to score both demographic and activity data
o   ability to track lead scores over lead lifetime instead of pre-set timeframes (30/60/90 days)
o   Can we track and trigger workflow off lead score velocity: rate at which lead score is increasing or decreasing over a given timeframe.
o   Are there any limitations in passing lead score data from the marketing automation system into SFDC
7.       Lead Nurturing: setting up lead nurturing campaigns
o   Setting up lead nurturing campaigns with multiple paths depending upon how the lead/contact responds.
o   What kinds of analytics are available to measure the effectiveness of a multi-stage, multi-channel nurturing campaign.
8.       Webex and Adobe Connect integration for the registration process
o   How to implement the registration process in the marketing automation platform: registration pages, confirmation email with webinar link, confirmation email with calendar appt file, reminder emails
o   Can we set up multiple registration pages for one webinar.  Example:  if we have several channel partners, each channel partner would have their own custom branded registration page for the webinar.
9.       Other:
o   How to deliver actionable information to sales reps.  Don’t want to have to buy extra licenses just to send alerts or to share activity data.  Prefer to pass lead scoring and activity data into SFDC and triggering alerts from there.  Prefer to get real-time site visit alerts without having to buy an extra module/license.
o   Are there any opportunities to integrate social media with the marketing automation platform: linkedin, twitter, facebook, blog
o   Is there any way for the website to read the marketing automation system cookie and serve a different banner ad or web page based on the information in the cookie.  Example:  if a person is cookied in and visits the home page, serve a different page than if the person is not cookied in.
o   How frequently does the system synch with SFDC?
§  Is it possible to keep purchased email address records in the marketing automation system and not synch them to SFDC.
§  If there are 100K records in SFDC, does the marketing automation system have the intelligence to only synch incremental changes since the last synch.
§  How long does a synch take
o   How does the system behave  if an existing lead or contact fills out a registration form.
§  Does it generate a new lead or does it add just add the campaign activity history.
o   Issue:  Sales reps don’t associate all Leads/Contacts with Opportunities in SFDC.  Is there any way to use the tracking and analytics capabilities of the marketing automation system to infer who is associated with an opportunity based on Account name and their behavior?
§  Is there an easy way to create sub-segments based on Account Name and then track and score the field data and activity data in associated with an account that is associated with an opportunity.
10.   Implementation:
o   What code has to be implemented on the website
§  Are there any issues with our site being a word-press site
o   How does the data synch process work between and the marketing automation platform.
§  How does it process a new lead registration when there is an existing lead or contact that matches the email address.
o   When we transition from Eloqua to the new marketing automation system, how do we maintain the data that associates Records to purchased Lists.  That is not currently in a field in
o   How will the behavior of the resource center change
o   Is there any way to transfer any of the historical site visit and email open/click data in Eloqua into the new marketing automation platform.
o   What does the contract include.  Are there any other modules that we would need?
o   What goes into a typical implementation
o   Support:
§  What methods are available for support
§  Test the effectiveness of calling into the support lines
§  Is there a community forum where you can ask questions

Monday, December 5, 2011

Why we left Eloqua

We started this marketing automation evaluation several months ago.  We have been running Eloqua for the past 2 years, but mainly using it for email marketing.  My general opinion of Eloqua is that it is a very powerful platform, but almost everything you want to do in Eloqua requires custom programming.  As a result, it's very difficult for a marketing user to turn on functionality like tracking links or tracking document downloads.  

Some of the other issues that I experienced with Eloqua were: 
  • The Eloqua user interace is not intuitive at all.  The layout and controls are not consistent or intuitive.  To do anything, it's a matter of memorizing how to interact with the interface.
  • The reporting is very limited.  Eloqua provides standard reports and it's very difficult or impossible to do custom reporting.  For example, I wanted to run a report to segment our database by activity level: no email opens, 1 email open, 2 email opens over all time.  Turns out that the only report that Eloqua can produce is a report that says these email addresses have never opened an email in the past 12 months.  It can't do any other possible reports based on different types and levels of activity.
 We looked at Eloqua 10 which is their attempt to improve the usability of the product.  What we saw were some improvements in reporting and the UI for creating email campaigns.  However, all things that we wanted to do like implementing tracking links, passing document download data into campaigns, and lead scoring seemed to require custom programming.

We're a small company.  We don't have the resources to hire a full-time Eloqua admin with programming expertise or the funding to be hiring consultants to manage and customize Eloqua for us.  As a result, I pushed to look for another marketing automation platform that would be easier to use where marketing users could exploit the platform without having to be programming experts.

We decided to look at Marketo because it claims to be easy to use and Act-On because it has such strong integration with webinar platforms like Web-Ex.

Editing HTML for Marketo Landing Pages is complicated

Instead of taking the time now to provide all the background information and catching up to where we are today, this post covers where we are today and as I find the time, I'll go back and catch up on other details that people might be interested in.  We decided to go with Marketo and in we're in the process of an implementation pilot that started at the beginning of November.

For the past few days, I've been trying to figure out how to change the "Submit" button on a landing page.  I want to change the color to a custom color and I want to change the words to "Download Now."  It's turning out to be very complicated and I still haven't figured out how to do it.

Marketo Landing pages consist of three parts.  The Form: which are all the fields that you want to capture.  The Landing Page Template: which consists of background formatting,  And the Landing Page itself.  So you have to create and edit those three components separately to build a Landing Page.

Well, you can edit the HTML in a LP template, but it's not easy to edit the HTML in a Landing Page.  And the Submit button is behavior is built using Javascript.  For someone like me with no HTML skills, simply trying to go through the HTML and trying to make some minor edits is not working.  I still haven't figured out how to do it.

So the bottom line, Marketo forms can be easy to use if you stay within their pre-set templates, but when you want to do something even a little more customized with a landing page, it becomes a lot more complicated and starts requiring someone who understands javascript.

Update 12/6:

After talking with our Marketo implementation consultant, I understand how to change the submit button.  It takes these specific steps listed below.  What the takeaway is that you can customize the submit button, but it's not straightforward.  All of the instructions below are very specific to Marketo and you can see that they only make sense once you familiarize yourself with the Marketo user interface.  I see this as an example, where the ease of use of Marketo is only slightly better than Eloqua.
  • To change the label on the Submit button on a landing page, you have to do that in the Marketo Form.  Edit the Submit button label to change it to something else like Download Now.
  • To change the color of the button you have to create a new background image for the button and change the link in either the Landing Page Template or the Landing Page itself.  The difference being that a pre-defined Landing Page Template will have button defined in the template.  If you start with a blank landing page template, the submit button has to be created in the Landing Page as a custom HTML box.
  • The background image for the submit button has a very specific design layout.  It's not easy to change the color.  You need to open the background image in Photoshop and create a layer matching the background image with your desired color.
  • Here's a link to a Marketo Knowledgebase Article that describes this: