Search Engine Marketing Automation with You Behind the Wheel

This blog post was written in collaboration with Ville Sillanpää.

Introduction

Search engine marketing (SEM) is a great tool for growing your business through well contextualised advertising. Getting your product in front of potential customers when they are looking for it can result in high conversion rates, and profitable marketing. It’s a lot of work, which is why you should look into automating your SEM processes.

For online stores the power of search engine marketing can also be a curse. How do you manage your campaigns, when thousands of online products all need to be present for dozens of contextual search terms? Do you build up your campaigns manually one piece at a time? Should you hire an agency to do this for you? Or should you automate the entire process?

There are many ways to automate search engine marketing, and the various strategies depend on your overall marketing strategy. One strategy can be to set up a “broad brush” account plan, and refine the most effective methods based on the data. This strategy can be very effectively automated to a high degree, but it requires a large initial budget. It also effectively multiplies the risks of automation, as poorly set up automation strategies can pour money into ineffective campaigns, and you might not alway notice this or be able to correct for this in time.

Another commonly used growth strategy is to build campaigns incrementally, starting from the presumably most profitable core of your products, and expanding towards the peripheries based on gradual improvements to customer insights, and experience. This strategy can be quite cost-effective as learning can focus on a limited section at a time, and learnings can more easily be scaled once the marketing campaigns is well optimised. This strategy can, however be quite labour-intensive, and thus costly. It can also limit the speed of the initial growth, as only a certain amount of campaigns can be managed effectively at the same time.

There are various effective tools that marketers can use to limit the amount of work needed to run effective campaigns and grow your business. The aspects of search engine marketing that can be automated, vary substantially between these strategies, and thus require different tools to be implemented. While there are many options in the market, there are no silver bullets, so be sure to choose wisely.

What can I automate?

Automation in search engine marketing can be grouped into three basic groups by function: 

• Automating keyword research, structures, and ad copy
• Automating performance and optimisation
• Automating anomaly detection and reporting

Automating structures can be done holistically, or piece by piece. There are various tools currently available for most of the elements for automation, as well as some solutions to automate the account structure setup from start to finish. We will first discuss the piecemeal solutions, and then move to the more holistic options.

Automating keyword research

The first part of almost any search engine marketing process is conducting keyword research. This is typically time consuming work. Rough outlines of relevant keywords can be extracted through Google’s Keyword Planner or various other tools (e.g. Ahrefs, MOZ Keyword Tool, keywordplanner.io, Ubersuggest, Answerthepublic etc.). While these tools are useful in gathering ideas for keywords based on your website, or product names, they are far from complete to plan the keywords for a new webshop, as keywords should be categorized to be effectively split into Campaigns and AdGroups with strong contextual links.

Campaigns and AdGroup structures

Campaigns and ad groups are the backbone of SEM accounts. The amount varies widely from a few campaigns and some ad groups within them to thousands of campaigns multiplied with ad groups. If you are working with, for example, an ecommerce or airline account where you use a product catalog with thousands of products, automation of the campaign and ad group creation is fairly easy and saves you a lot of time. However, this is not directly possible by automating with scripts. Fortunately here the automation platforms are especially handy. After you have set up your feed into the platform you can select which fields you use as the basis of the campaign and ad group creation. Besides the creation of the structure from zero platforms will keep the changing product catalogs up to date in the ad platform.

Let’s say you are an ecommerce vendor selling furniture. You could arrange and create your campaigns by product category (sofas, armchairs, tables) and either divide the product category into sub-categories or directly into product based ad groups. You can create the whole account based on this classification and as your product selection is updated the platform adds new products and removes discontinued ones from advertising.

Campaigns and ad groups can be created through categorization of keywords. This can be approached with various degrees of automation. There are some services currently available to categorize keywords into categories. These tools (Adcore, Optimzr, Acquisio, Opteo, Adalysis) tend to be somewhat expensive, but do save lots of time compared to manual categorization. These solutions are useful for both strategies of setting up search engine marketing campaigns, but their benefits tend to correlate with the size of the campaigns.

For smaller webshops, full automation will likely be too expensive compared to manual work. Manual work should, however, not be thought of as going through keywords one-at-a-time, and grouping them based on the know-how of the practitioners. Rule-based sorting algorithms can also be used on various platforms, as can creative use of spreadsheets.

Ad copy

Ad copies are perhaps the most frequently tested and time-consuming part of the account management after the initial setup. The basic mission for copy in SEM is to provide a contextual match for the user’s search intention. The first task is to add the initial version for each ad group plus a second version to be tested against. This means that there is going to be thousands of ads to be created for best matching context in each ad group.

In addition to creating the basic structure like this, companies, such as airlines or transit companies have to account for constantly changing prices, as well as the need to pause campaigns when products are out of stock. This part of account creation is not wise or even possible to do without some help from automation. The good thing here is that you can start from really basic automation and add complexity once you learn what works. Areas where you can utilize automation include product names, prices, discounts, timing, and location.

Performance & bid optimisation

Optimizing campaign performance is long term work of setting bids for specific keywords, and campaigns. It’s modifying bids based on customer insights that are likely to reduce costs for the results you are looking for, whether it is clicks, purchases or signing up to your awesome newsletter! With complex campaign structures, it takes a lot of time to go through all the data to be confident in your course of action.

Some of the automation this side of SEM has already been built into the system. Search advertising platforms have various targets that you can optimise your bids towards. Using these will not add to the time needed for optimising your campaigns towards your desired outcome.

In addition to the built-in bid automation systems, it is also possible to work around—build upon—the black boxes within search engine marketing platforms. Using custom scripts allows you to control the bids for your campaigns based on the logic you set up, while still making is possible to automate the process of doing changes manually.

Setting up iterative automated cleaning cycles will allow you to weed out the poorer performing activities while still allowing for the effect of seasonal variations on the performance levels. This method will let you set the criteria for when to keep, when to kill, and when to adjust current activities—in advance. Once the logic has been set up, you can simply observe and monitor the results, while only needing to intervene if something seems to be going wrong.

Some of this automated optimisation can be done within the ad platforms themselves through scheduled bulk edits, and rules. Forming the logic for these edits can, however, require more advanced data analysis through machine learning models. Creating machine learning models on your own can be achieved through the quite extensive reporting and editing APIs for the marketing platforms, but in most cases it will be easier to simply go with one of the existing products for performance optimisation. Such platforms have seen improvements in performance in the 20-100% range for desired KPIs, so creating your own tools to circumvent these tools should only be done if you are highly confident on your own machine learning skills, or are unwilling to work with anything even somewhat resembling a black box.

Automating anomaly detection and alerts

Anomaly detection is the way to achieve peace-of-mind while running your campaigns. Having a quick detection system set up to notice when the account is not performing in the way you expected allows you to lessen the amount of reporting to be done for the account. A longer cycle of review can be achieved when well-defined goals and definitions have been set up with clear signals for things going wrong. For this, as most other forms of automation, there are pre-build third party tools—and abundant ways to set up the specific automation you want.

A simple way to automate anomaly detection is built into Google Analytics, which is the industry standard for analysing website traffic. Google Analytics runs a machine learning algorithm which detects anomalous behaviour on the site, and reports it as a custom insight. Using the mobile app for this allows you to get notifications for any anomalies on the site, and to an extent, from your SEM campaigns. This is, however, quite general in scope, and won’t put you in command of the nuts and bolts of things.

The second option would be to implement a machine learning system to detect anomalies and alert you to them. This can be done through various services such as Anodot, Loom Systems, Numenta, or Splunk Enterprises. These are advance tools and can be immensely useful, but will likely be unavailable for smaller companies, as they do incur some costs. Custom tools are the ideal option to make sure the solution is suited for you, but requires lots of work from machine algorithm experts, and will thus in all likelihood be even more costly.

For relatively simple accounts, the simple solution for anomaly detection, available to all users, will be simple rule based email alerts you can set up directly from the SEM platform. These alerts can be quite granular, especially when built using the script functionalities. Even higher levels of detail can be achieved with relative ease using automated data export tools, some creative thinking, and a knack for online spreadsheets.

In summary

Automating your search advertising can be a very effective way of optimising the visibility of your ecommerce platform. It does, however, also have it’s pitfalls. Unless you plan the implementation properly to suit your needs, you may end up with an expensive and rigid platform that you can’t adjust properly, or an uncontrollable behemoth that takes up more time than running your campaigns manually. In successful use, automation enhances the quality of routine work and saves time for developing advertising on a strategic level.

If you are thinking about automating SEM, be sure to consider the following

• Which parts of account management do I need/want to automate?
• How much would the results need to improve to be worth the initial investment? Is this reasonable to expect?
• Are the available solutions suitable for me?
• How much would a custom solution cost (in money or in performance)?

Once you are satisfied that these questions are answered, feel free to get started, and enjoy the view from behind the wheel!