Reporting API
The reporting data shown in the Conductrics admin is also available via a relatively simple Reporting API, described here. You can use this Reporting API to retrieve the same data for whatever custom purposes you require, for instance to display the data "natively" in your own admin pages.
There is also a corresponding API to retrieve data about your Conductrics Surveys. See the Survey Reporting API page for details.
API AuthorizationTo keep this page relatively clear, we're omitting certain details from the example snippets:
- These examples assume that your account is available at
https://api-v3.conductrics.com. Your account may use a different endpoint (URL), especially if you're using a private Conductrics server or if your Conductrics account is located outside of the United States.- These examples show a pretend account code (
acc-xxxxxx). You will need to replace that with your actual account code.- You also need to provide an
adminAPI key for your account, either as a query parameter called apikey (so,?apikey=admin-xxxxxxxx) or as a HTTP header calledx-conductrics-apikey.
All of the above details are available in the Conductrics Admin, under Developers > API Info & Keys. If you get an "Unauthorized" message when trying to use these APIs, it's likely that one of these items wasn't provided correctly.
Activity Report Data
To retrieve the data shown in the "Activity" report in our admin, issue a GET request like the following (substituting your own account and agent code):
https://api-v3.conductrics.com/admin/acc-xxxxxx/reports/decision-stats/a-12345678?dateFrom=first&dateThru=last
(Make sure to provide your API Key as discussed in the API Authorization callout at the top of this page.)
Specifying The Date Range
You can specify the date range that you want the report to cover using these optional query-string parameters:
dateFrom- The start of the date period to report on (inclusive).dateThru- The end of the date period to report on (inclusive).
The report's numbers will include data from dateFrom through dateThru (inclusive). You can provide the date in yyyy-mm-dd format, or as a negative number such as -5 for 5 days back in time. You can also provide the special strings today or first or last. The default for both is is today if not provided.
If you need to know the available date range, you can first request the report data using
dateFrom=firstanddateThru=lastas shown above, then look at the returneddatessection in the response data (see example response below) to obtain the date range for which data is actually available. This is the same date range that you'll see in the calendar-style date-range picker on the reporting pages in the Conductrics Admin.
Response Data
The main part of the data is the data array, which will contain an item for each combination of choice (aka variation or option), visitor feature, and selection policy.
{
"status": 200,
"data": [
// an item like this for each choice/feature/policy combo
{
// these first few fields describe the item
"choice": "A",
"label": "A",
"policy": "explore",
"feature": "*",
"featureLabelText": "Everyone",
// these fields are the actual numbers of interest
"count": 0,
"goals": 0,
"val": 0,
"goalsPerDecision": 0,
"valPerDecision": 0,
"valPerGoal": 0,
"selsPer": 0
},
// ...other items for other choices/features/policies...
],
"dates": {
// the date range used to retrieve the above counts
"dateFrom": "2024-02-08", // first day of the date range
"dateThru": "2024-03-18" // last day of the date range
}
}The first few fields within each item simply describe the choice/feature/policy combination being reported on by the item:
choice- the API code for the choice/variation.label- the display name/label for the variation.feature- the API code for the visitor feature. Some items will use the special feature code*which indicates data for all visitors.featureLabelText- the display name/label for the visitor feature (or the string "Everyone" for the*feature).goal- the API code for the defined goal being reported on. If the report is for all goals together, then this will be set to*.policy- the Selection Policy that was used by the agent:explore- for random selections. If the agent in question is a "testing" agent that only makes random selections, all of the data items will showexplore. If the agent in question is a "predictive" agent, the agent will still make someexploreselections by default to help refine its internal predictions.exploit- for predictive selections, when the agent used its predictive model to select the given choice/variation (not used by "testing" agents).control- for control-group selections (not used by "testing" agents by default).fixed- for selections made because of targeting rules you have set up at the agent level in our admin (for example, telling the agent that it should always pick variation "B" for UK visitors, etc).
The next few fields provide the main "numbers" of interest:
count- the number of selections that were made.goals- the number of goals (rewards) that were received as goal events.val- the amount of reward value that was received with goal events. Relevant mainly for e-commerce and other similar scenarios where each goal ("conversion") has been given a monetary or other explicit numeric value. If your goals don't specify an explicit numeric value, then the system uses an internal default value of1, which means you can expect thevalto be the same as thegoalsvalue.
The remaining fields are simple "this over that" calculations:
goalsPerDecision- This is simplygoalsdivided bycount.valPerDecision- This is simplyvaldivided bycount.Expect this to be the same asgoalsPerDecisionunless your goals have explicit numeric values (seevalabove).valPerGoal- This is simplyvaldivided bygoals. Expect this to be1unless your goals have explicit numeric values (seevalabove).
Testing Report Data
To retrieve the data shown in the "Testing" report in our admin, issue a GET request like the following (substituting your own account and agent code):
https://api-v3.conductrics.com/admin/acc-xxxxxx/reports/confidence/a-12345678?dateFrom=first&dateThru=last
(Make sure to provide your API Key as discussed in the API Authorization callout at the top of this page.)
The server should respond with a JSON structure like the following:
{
"status": 200,
"data": {
"items": [
// an item like this for each choice/feature/policy combo
{
// these first few fields describe the item
"choice": "B",
"label": "B",
"feature": "*",
"featureLabelText": "Everyone",
"goal": "*",
"policy": "explore",
// main stats of interest
"count": 490,
"goals": 128,
"val": 128,
"bLo": 0.22219158445939569,
"bHi": 0.3002573951324411,
"vMean": 0.2612244897959184,
"confidence": "0.0",
// extra-credit stats
"vVari": 0.19338091064646717,
"stdDev": 0.43975096434967276,
"tScore": -3.6234244820743826,
"dgFree": 956,
"pValue": 0.9998469796917684,
"lift": {
"default": "-29.2",
"random": "-7.2"
}
}
// ... other items, for other choices/features/policies...
],
"dates": {
"dateFrom": "2016-09-03",
"dateThru": "2016-09-08"
}
}
}Returned Data
The main part of the data is the items array, which will contain an item for each combination of choice (aka variation or option), visitor feature, and selection policy.
The first few fields within each item simply describe the choice/feature/policy combination being reported on by the item:
choice- the API code for the choice/variation.label- the display name/label for the variation.feature- the API code for the visitor feature. Some items will use the special feature code*which indicates data for all visitors.featureLabelText- the display name/label for the visitor feature (or the string "Everyone" for the*feature).goal- the API code for the defined goal being reported on. If the report is for all goals together, then this will be set to*.policy- the selection policy that was used by the agent:explore- for random selections. If the agent in question is a "testing" agent that only makes random selections, all of the data items will showexplore. If the agent in question is a "predictive" agent, the agent will still make someexploreselections by default to help refine its internal predictions.exploit- for predictive selections, when the agent used its predictive model to select the given choice/variation (not used by "testing" agents).control- for control-group selections (not used by "testing" agents by default).fixed- for selections made because of targeting rules you have set up at the agent level in our admin (for example, telling the agent that it should always pick variation "B" for UK visitors, etc).
The next few fields are the main data points of interest for each item:
count- the number of selections that were made.goals- the number of goals (rewards) that were received as goal events.val- the amount of reward value that was received with goal events. Relevant mainly for e-commerce and other similar scenarios where each goal ("conversion") has been given a monetary or other explicit numeric value. If your goals don't specify an explicit numeric value, then the system uses an internal default value of1, which means you can expect thevalto be the same as thegoalsvalue.bLo- The lower bound for the estimate of this choice's value. Typically the lower and upper bounds are used to draw a horizontal bar that makes it easier to interpret the report data visually. The bar for each item is drawn from bLo to bHi, on a scale based on the overall bLo and bHi values (at the 'data' level).bHi- The upper bound for the estimate of this choice's value (see bLo).vMean- The estimated value of this choice, based on goal events attributed to the choice in question. Typically this value is displayed in the center of the bar for the lower/upper bounds. This value is the conceptual focus of the report for most users.confidence- A confidence score that indicates the amount of evidence there is to support the value (val) prediction. The value will range from 0 (no evidence) to 1 (certainty, though in practice 1 is seldom achieved). The confidence score is currently computed as the inverse of the p-value, assuming that the options being compared have each been selected a reasonable number of times (if not, the confidence score remains at zero until they have).
Some additional "extra-credit" stats are also provided for each item:
pValue- the statistical P-Value, used internally to compute some of the other calculations. Conceptually important to end users, but more typically displayed by simply indicating whether the P-Value exceeds the basis confidence measure (see the 'signif' flag, below). Generally speaking the 'confidence' and 'confidenceLevel' values are preferred over the raw p-value.ue.dgFree- Degrees of freedom, used internally to compute some of the other calculations (typically not important to end users).vVari- The statistical variance associated with the value of the goal events attributed to the choice in question.stdDev- Standard deviation calculation, used internally to compute some of the other calculations (typically not important to end users).tScore- Student's T calculation, used internally to compute some of the other calculations (typically not important to end users).
Optional Parameters
You can provide these optional query-string parameters:
dateFrom- The start of the date period to report on. The report's numbers will include data from the date period fromdateFromthroughdateThru(inclusive). You can provide the date inyyyy-mm-ddformat, or as a negative number such as -5 for 5 days back in time. You can also provide the special stringstodayorfirstorlast. The default istodayif not provided.dateThru- The end of the date period to report on. The default istodayif not provided.policyComparisonMode- If you providetruefor this parameter, the conceptual "choices" being reported on are actually the selection policies (explore, exploit, control, or fixed). Our admin uses these numbers to provide a sort of "meta-report" that, rather than providing information about which of your agent's choices are more or less effective, instead gives you similar information about which of the selection policies are more or less effective.use-bonferroni- set this to 'false' to indicate that you do not want the Bonferroni Correction applied to the calculations. The default is 'true'.confidence-measure- An optional confidence measure to be used in the report calculations, from 0 to 1. The default value if not provided is 0.9
Additional Data
When you call one of the Reporting API endpoints above, the returned response also contains some additional information about the available date range and so on.
Here's an example:
{
"status": 200,
"data": {
"dailies": [
{
"d": 1709078400, // date, as a
"sels": 2,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709164800,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709251200,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709337600,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709424000,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709510400,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709596800,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709683200,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709769600,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709856000,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1709942400,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1710028800,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1710115200,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
},
{
"d": 1710201600,
"sels": 0,
"goals": 0,
"gvals": 0,
"gps": 0
}
],
"dates": {
"created": 1663718400,
"today": 1710201600,
"started": 1663718400,
"elapsed": 538,
"day_num": 539,
"num_days": 14,
"dataFrom": 1663718400,
"dataThru": 1709078400,
"timestamp": 1709226632,
"offset": -240
},
"totals": {
"sels": 2,
"goals": 0,
"gps": 0
},
"info": {
"created": {
"user": "[email protected]",
"ts": 1663797659
}
},
"reset_ts": 1700681111,
"_lift_by_goal_ok": true
}
}Updated about 1 year ago