Concept learning update migration guide

Documentation

March 13, 2020

Aito is rolling the concept learning update starting March 23rd. This update includes major improvements in Aito's statistical reasoning, bug fixes, performance improvements and API changes. You can find more information about the update here.

The update also introduces two API incompatibilities. This guide describes the changes and the migration.

Changes in Explanation Objects ($why)

Consider the following query:

{
  "from" : "invoices",
  "where" : {
    "Item_Description" : "Store Management Rent"
  },
  "predict" : "Product_Category",
  "select" : ["$p", "feature", "$why"],
  "limit" : 1
}

Previously, Aito would return the following result:

{
  "offset": 0,
  "total": 36,
  "hits": [
    {
      "$p": 0.9995717068809318,
      "feature": "CLASS-1274",
      "$why": {
        "type": "product",
        "factors": [
          {
            "type": "baseP",
            "value": 0.17761033369214208
          },
          {
            "type": "normalizer",
            "name": "exclusiveness",
            "value": 0.999573491370787
          },
          {
            "type": "relatedVariableLift",
            "variable": "Item_Description:rent",
            "value": 5.641011031293095
          },
          {
            "type": "relatedVariableLift",
            "variable": "Item_Description:manag",
            "value": 5.410543091141314
          },
          {
            "type": "relatedVariableLift",
            "variable": "Item_Description:store",
            "value": 3.0082144959094568
          }
        ]
      }
    }
  ]
}

Now Aito returns the following result:

{
  "offset": 0,
  "total": 36,
  "hits": [
    {
      "$p": 0.9753500064252928,
      "feature": "CLASS-1274",
      "$why": {
        "type": "product",
        "factors": [
          {
            "type": "baseP",
            "value": 0.1761870760442699,
            "proposition": {
              "Product_Category": {
                "$has": "CLASS-1274"
              }
            }
          },
          {
            "type": "normalizer",
            "name": "exclusiveness",
            "value": 0.9763895461514779
          },
          {
            "type": "relatedPropositionLift",
            "proposition": {
              "$and": [
                {
                  "Item_Description": {
                    "$has": "rent"
                  }
                },
                {
                  "Item_Description": {
                    "$has": "store"
                  }
                },
                {
                  "Item_Description": {
                    "$has": "manag"
                  }
                }
              ]
            },
            "value": 5.6353753851278805
          }
        ]
      }
    }
  ]
}

The new format has following main changes:

BaseP-component now contains the 'proposition'-field, which contains the predicted value's proposition. In recommend queries the BaseP-component's proposition field will contain the goal proposition.
The explanation component type names have been renamed according to the following list
1. "relatedVariableLift" -> "relatedPropositionLift"
2. "hitVariableLift" -> "hitPropositionLift". "hitPropositionLift" occurs in $why of Match Query
3. New "hitLinkPropositionLift" proposition is added to explain the effect of the predicted link variable in Match Query
The explanation components no more have 'variable' field containing a string, but a 'proposition' field containing a proposition object.

So in essence, the '$why' component no more contains components of form:

{
  "type": "relatedVariableLift",
  "variable": "Item_Description:rent",
  "value": 5.641011031293095
}

Instead it contains components of form:

{
  "type": "relatedPropositionLift",
  "proposition": {
    "Item_Description": {
      "$has": "rent"
    }
  },
  "value": 5.635622992714686
}

So in order to do the migration, you need to look for 'relatedPropositionLift' and 'hitPropositionLift' component type identifiers instead of 'hitVariableLift' and 'hitPropositionLift' identifiers. Also, you need to parse proposition objects, instread of proposition strings. You can find more information about the proposition objects at the end of this guide.

Changes in Relate Query Result

Consider the following relate query:

{
  "from" : "invoices",
  "where" : {
    "Item_Description" : "Rent"
  },
  "relate" : { "Product_Category" : "CLASS-1274" },
  "select" : ["related", "lift", "condition"]
}

Previously, Aito provided the following response:

{
  "offset": 0,
  "total": 1,
  "hits": [
    {
      "related": "Product_Category:CLASS-1274",
      "lift": 5.641011031293095,
      "condition": "Item_Description:rent"
    }
  ]
}

Now Aito provides this response:

{
  "offset": 0,
  "total": 1,
  "hits": [
    {
      "related": {
        "Product_Category": {
          "$has": "CLASS-1274"
        }
      },
      "lift": 5.635622992714686,
      "condition": {
        "Item_Description": {
          "$has": "rent"
        }
      }
    }
  ]
}

Previously 'related' and 'condition' fields would contain strings identifying the propositions. Now the 'related' and 'condition' fields contain proposition objects identifying the proposition.

To do the migration, you need to parse the 'related' and 'condition' fields as proposition objects, instead of parsing them as proposition strings. There is more information about the proposition objects at the end of this chapter.

The proposition object format:

So previously the propositions were identified with strings like:

"Item_Description:rent"

Such strings been replaced with proposition objects like:

{
  "Item_Description": {
    "$has": "rent"
  }
}

A proposition object can be complex:

{
  "$and": [
    {
      "Item_Description": {
        "$has": "rent"
      }
    },
    {
      "Item_Description": {
        "$has": "store"
      }
    },
    {
      "Item_Description": {
        "$has": "manag"
      }
    }
  ]
}

Both "$why" explanation objects and "relate" results may contain complex propositions in cases where complex propositions were presented in the "where" clause. "$why" field may also contain complex propositions in cases where the concept learning has formed complex expressions to do higher-level statistical reasoning.

It is guaranteed that the proposition objects can be reused in the where clause, and in place of propositions in a query.

Following propositions can occur in the proposition object:

$has
$and
$or
$on
$not
$startsWith
$gt
$gte
$lt
$lte
$defined
$numeric
$knn
Field proposition of form { "field" : PROPOSITION }
The normal 'is proposition' containing a value primitive like "string", true, 4 or 4.3

As an additional note: the $numeric proposition no longer shows the numeric range.

You can consult the existing documentation to learn more about the proposition structure: https://aito.ai/docs/api/#language-text-operators

Back to developer docs

Address

Episto Oy

Putouskuja 6 a 2

01600 Vantaa

Finland

VAT ID FI34337429

Contact