Alternative for a rules engine in RPA

Let's say you’re working on automatic handling of purchase invoices with Robot Framework, basically making a new pair of hands to take care of the repetitive manual tasks involved. Each invoice is saved as a text or JSON file in a specific folder, but many of them are missing the information for the right product category. What do you do? Sounds like a job for machine learning.

Instead of hacking together a rule engine or relying on constant human intervention, you can insert a brain to guide the RPA hands with the decisions. This template walks you through how to use Aito for predicting the missing category in each file without any rule engines or model training.

Before we get into the code, let’s quickly go over the starting situation. You have already uploaded into Aito instance some historical data (sample data from Kaggle) which you have collected previously when the product category was added manually. Here’s how the data looks like:

The last column, Product_Category, is what your robot needs to determine for each new entry.

Now, when a new purchase invoice comes in and no product category is defined, you can use Aito to predict the most likely category based on the previous data. If the product category already exists, however, then no prediction is needed and the data can be uploaded into Aito. This will further improve the accuracy of future predictions.

Setting up

To keep this tutorial from bloating up, we assume you already have an Aito instance (create one here) and some data uploaded (sample data here). Here are the next steps:

1. Install Aito and Robot Framework: pip install aitoai==0.4.0 and pip install robotframework==3.2.2
2. Download and extract the template .zip file
3. Set AITO_INSTANCE_URL and AITO_API_KEY as your environment variables

In the zip file, you’ll find:

1. invoice.robot: The main file with task automation
2. aitohelper.robot: Helper keywords for Aito
3. invoice_data.csv: The dataset which has been uploaded to Aito beforehand
4. files/ directory: Mock files to test with

The code

You don’t really need to worry much about aitohelper.robot since the main purpose of the keywords are just to make Aito API more convenient for Robot Framework. Let’s go through the invoice.robot file line by line.

Imports

First, we import OperatingSystem to allow file operations and aitohelper.robot for the keywords.

*** Settings ***
Library     OperatingSystem
Resource    aitohelper.robot

Variables

Next, we set the variables. We define the name of the table schema where your data is found in Aito instance and the name of the target feature which we want to predict. We also define the paths to our test files.

*** Variables ***
${table}=       invoice_data
${target}=      Product_Category
${path}=        ./files
${pathCate}=    ./files/categorized

Task

Now we build up the task. Since we want to go through all the files in the files folder, we set up a loop through the list of files in the directory.

*** Test Cases ***
Categorize Invoices
    # Loop through all files in directory
    ${files}=                   List Files In Directory     ${path}
    FOR     ${fileName}         IN              @{files}
        …
    END

The first thing we do inside the loop is opening a file and reading its contents.

        # Read file
        ${filePath}=            Join Path       ${path}     ${fileName}
        ${data}=                Get File        ${filePath}

If we printed out the file content to console, this is what it would look like:

{
 'Inv_Id': 15014,
 'Vendor_Code': 'VENDOR-2513',
 'GL_Code': 'GL-6050310',
 'Inv_Amt': 69.85,
 'Item_Description': 'Ground Transportation'
}

Since at this point the content is still in string format, we need to turn it into a dictionary by using the Evaluate keyword. We also use Evaluate to check whether our target exists in the input data dictionary. This allows us to decide whether or not we should predict the value or upload the data.

        # Turn file content into a dictionary and check if target exists in dictionary
        ${data}=                BuiltIn.Evaluate        ${data}
        ${exists}=              BuiltIn.Evaluate        '${target}' in ${data}

Now is the time for Aito to turn up. If the target did not exist in the input data, we run the keyword Aito Predict and provide the table name, input data and target name. Aito returns us the predicted value and its probability.

        # If target does not exist in dictionary, predict the most probable value
        ${pred}   ${prob} =     Run Keyword Unless      ${exists}     Aito Predict      ${table}    ${data}      ${target}

If the target DOES exist in the input data, then there’s no need for a prediction and we can upload the clean data entry to Aito. The reason we only upload data when a prediction was not made is to keep the training data clean. Using predicted data for future predictions may gradually reduce the accuracy.

        # If target exists in dictionary and no need to predict, upload dictionary into Aito
        Run Keyword If          ${exists}     Aito Upload       ${table}    ${data}

And finally, if a prediction was made, we add the new value to the data dictionary and save it into a file in the /categorized/ directory.

        # If prediction was made, add predicted value to dictionary
        Run Keyword Unless      ${exists}     Set To Dictionary    ${data}    ${target}   ${pred}

        # Convert dictionary to string and write into a file
        ${data}=                Convert To String   ${data}
        ${filePath}=            Join Path       ${pathCate}    ${fileName}
        Create File             ${filePath}     ${data}

And that’s it. Run it with robot invoice.robot in command line, and each file in /files/ directory gets looped through and saved in /files/categorized/ directory with the predicted Product_Category (if predicting was necessary).

Here’s the full invoice.robot file:

*** Settings ***
Documentation     Invoice categorization with Aito.
...
Library     OperatingSystem
Resource    aitohelper.robot

*** Variables ***
${table}=       invoice_data        # Name of table in Aito insance
${target}=      Product_Category    # Name of the feature we want to predict
${path}=        ./files
${pathCate}=    ./files/categorized

*** Test Cases ***

Categorize Invoices
    # Loop through all files in directory
    ${files}=                   List Files In Directory     ${path}
    FOR     ${fileName}         IN              @{files}

        # Read file
        ${filePath}=            Join Path       ${path}     ${fileName}
        ${data}=                Get File        ${filePath}

        # Turn file content into a dictionary and check if target exists in dictionary
        ${data}=                Evaluate        ${data}
        ${exists}=              Evaluate        '${target}' in ${data}

        # If target does not exist in dictionary, predict the most probable value
        ${pred}   ${prob} =     Run Keyword Unless              ${exists}     Aito Predict      ${table}    ${data}      ${target}

        # If target exists in dictionary and no need to predict, upload dictionary into Aito
        Run Keyword If          ${exists}     Aito Upload       ${table}    ${data}

        # If prediction was made, add predicted value to dictionary
        Run Keyword Unless      ${exists}     Set To Dictionary    ${data}    ${target}   ${pred}

        # Convert dictionary to string and write into a file
        ${data}=                Convert To String   ${data}
        ${filePath}=            Join Path       ${pathCate}    ${fileName}
        Create File             ${filePath}     ${data}
    END

Check out our other use cases and ping us on Slack community to give me feedback or to ask any questions, and talk with the team!