Predictive Agent Guidelines

This short post will hopefully help you get started with using Conductrics for predictive targeting. In it we will cover, at a high level, a few things to consider as you move forward with your first project.

1 - Start Simple


This is really the single most important thing, especially true if this is also your first time using Conductrics. While predictive targeting can be a powerful tool to help you get the most out of your marketing applications, it does require thought and care to implement.

We suggest that if you haven't used Conductrics before, that your first Agent should be a simple AB or even A/A test, rather than a predictive targeting agent. Why? So that you can first make sure that you have a good feeling for how to set up agent, and to make sure that there aren't any basic issues with your integration.

By having a simple set up to start, It will be much easier to be able to check to see if the traffic and conversion rates are consistent with your expectations and historical records. This will also let you get comfortable with Conductrics reporting and see how visitor trait data can get broken out by reviewing the default targeting data (Browser, Weekend/Weekday, etc.)

2 - Add Complexity Incrementally


After you have run a few basic tests, and feel confident that data is being collected correctly and all of the decision options are being represented properly in your applications, it is time to try your first predictive targeting agent.

Your first impulse might be to add as many targeting traits as you can think of. While there is a natural appeal to the idea of just 'letting the machine figure everything out', there are a few reasons to reign in this impulse.

Going back to Point #1, the preferred strategy is to "start simple" and add complexity incrementally. This means adding in only the targeting traits that you have some hunch, or past data to suggest they might be useful, especially if they are in some way related to the options being testing in your agent.

For example, perhaps you know that visitors tend to buy different products on the weekends than on the weekdays. Then adding the 'Weekday/Weekend' trait would make sense.

3 - Consider your Traffic


You can think of Conductrics machine learning algorithms as having two interrelated tasks:

  • To select only the visitor traits that are useful in discriminating which of the different possible decision options are the best one for a given visitor; and

  • To estimate the values of each decision option given those selected visitor traits.

This means that the difficulty, or length of time learning will take, for a given predictive agent will depend, in part, on both the number of decision options, and the number of traits the agent needs to consider.

There is no hard and fast rule on how many traits to use, but one rough guide for the max number of traits to try is to take the agent's daily traffic, divide that by the number of decision options, and then take the square root of that.

So if you expect about 5,000 visitors per day, are testing 4 possible options, then you might want to keep the number of visitor traits below 35. In general, even if you have a great deal of daily traffic it often makes sense to keep total number of possible traits below 250.

4 - Consider the traits


You can think about the visitor traits as a type of meta data, or tag that each visitor either has or does not have.

For example, when using the built in data traits that Conductrics provides, a visitor may be assigned to 'Weekend', 'Morning', and 'Chrome', and therefore will NOT be assigned to 'Weekday', 'Afternoon', and 'Edge' .

You are able to extend the trait set using Custom Visitor Traits. Try to give some consideration to what you plan to pass to your agent during this set up.

For example, traits that are closer to splitting visitors 50/50 in either having or not having the trait are in some sense the most potentially value (for more on entropy and information see: https://en.wikipedia.org/wiki/Binary_entropy_function). Therefore, data with very high cardinalities, tend not to make very good traits, because each value that the data can take will be true for a tiny percentage of your visitors.

For example, let's say you think that the time of day may help determine which option will be best for your visitors.

  • A good way to incorporate time of day would be to use something like Conductrics daypart auto-trait, which assigns visitors into traits like 'Morning'; 'Afternoon'; 'Evening' etc. Using this auto-trait option, everyone who arrives in the morning, gets the 'Morning' trait, everyone in the afternoon gets the 'Afternoon' trait etc. When your visitors tend to hit the agent will affect what percentage of visitors gets assigned to each trait, but since we have high level groupings, we should expect to have a reasonable amount of traffic in each (say at least 5% of total traffic).

  • Another way to include time of day information, would be to pass in a timestamp accurate to the second, such '16:47:08'. You might expect that this is even better way to encode daypart, because it gives a more accurate reading of when the visitor entered the agent. However, this is actually a very poor way to code the traits because it's maximum cardinality is 86,400 (that's how many seconds are in a standard day). That means any given time stamp trait will be active for a very low percentage of your visitors. This makes the value of any given time stamp worth almost nothing, since there won't be enough instances of it for Conductrics to learn how it affects the final option scores.

A guiding principle is to try to keep the cardinality of your traits low, by using more aggregate groupings. A good rough guide is to try to keep the number of traits for a given concept them below ten (20 at the maximum) so you are less likely to have very rare instances that won't be useful for targeting and will just clog up reporting.

5 - Interpreting the Audience Report


Congratulations! You followed the above suggestions and are in market with your first predictive agent. However, you are wondering why the Audience Report isn't showing any Audiences other than the default 'Everyone' audience. There are a few reasons why this could be:

  • By default, the Audience report gets updated after every 24 hour period. Unlike other reports you may be familiar with, the audience report is also a description of the active targeting program that Conductrics uses to make selections in real time (for adaptive agents). You can think of rerunning the report as also rebuilding the targeting logic that Conductrics uses. While you may view the report whenever you chose, in order to avoid altering that day's targeting logic, Conductrics will only update the report every 24 hours by default (the 24 hour default setting can be changed by contacting Conductrics). If you are not seeing any change in the Audience report, it may be that the report has not rerun since you last checked.

  • Not enough data. By default, Conductrics will only consider splitting the 'Everyone' audience into targeted audiences if there have been at least 1,000 visitors that have passed through the agent.

  • There is enough data, but Conductrics has determined that there isn't enough signal yet to include any of the traits to create new audiences. Remember that one of the main tasks for Conductrics is to build audiences only with visitor traits that have strong evidence that they will be useful in discriminating between which options are best. If it appears to Conductrics that most of the predictive results are probably just due to noise, or if the inclusion of the trait won't meaningfully change what options to select (say option 'A' is the best option for everyone both before and after including the trait, then the trait really isn't useful in helping us target, and won't be included).

📘

If it still looks like something isn't quite right please feel free to reach out to us and we will be more than happy to help out!