Javascript Transform

DataSiv's Javascript Transform enables you to write custom javascript to preprocess, postprocess your data as well as trigger queries/analytics.

The Working with Json Dictionaries section shows you an example of how to use the javascript component to post process objects such as JSON dictionaries.

The Triggering Transforms section shows you how to trigger javascript code from buttons.

This section will show you how to use the javascript transform to run chained queries together (e.g. for paginating through a Rest API to get all of the pages, and using the cursor property to link responses together).

Triggering Chained Queries

If you need to trigger chained queries together, you're able to do so by using async and await in conjunction with the DataSiv.ExecuteQueryAsync(...) method call, followed by the DataSiv.StoreResults(...) command, which will store the results of the async calls in the current javascript component.

For example, suppose you have a query sql0 with parameter customerId that you'd like to trigger with multiple customer Ids.

The way to do so would be as follows in the Javascript component.

async function f() {
let response = await DataSiv.ExecuteQueryAsync("queries.sql0", {"customerId": 5}, {"wait": 100});
let response2 = await DataSiv.ExecuteQueryAsync("queries.sql0", {"customerId": 6}, {"wait": 300});
let result = response.data.concat(response2.data)
result = DataSiv.StoreResults(result);
};

f().catch(ex=>DataSiv.StoreResults({error: ex.toString()}));
     
let data = []; 
data

The await DataSiv.ExecuteQueryAsync("queries.sql0", {"customerId": 5}, {"wait": 100}) would run the query with customerId set to 5 after waiting for 100ms, and the await DataSiv.ExecuteQueryAsync("queries.sql0", {"customerId": 5}, {"wait": 300}) would run the query with customerId set to 6 after waiting for 300ms.

If you have an error, it'll be saved as an error field through the .catch(ex=>...) statement, for easier debugging.

📘

The DataSiv.StoreResults function should only be called once in async methods

DataSiv.StoreResults will only be triggered the first time it's run, so make sure to only use it once in the input method.

Once you concatenate results, and run the DataSiv.StoreResults(results) function call, you'll see the results in the output property of your javascript component, which you can reference in downstream computations.

Paginating through API calls

Suppose you want to load all the results from a Rest API query, that is paginated with the offset field. The Rest API query returns items when there's more items to return, and the items field is null otherwise.

Here's how you'd be able to paginate through this API in DataSiv.

async function f() {

let offset = 0;

let results = [];

let initialResults = -1;

let result = [];

while (initialResults == -1 || (initialResults.data && initialResults.data.items && initialResults.data.items.length > 0)) {
    initialResults = await DataSiv.ExecuteQueryAsync("queries.rest0", {"offset": offset}, {"wait": 120});
    if (initialResults.data == undefined) {
        break;
    }
    results = results.concat(initialResults.data.items)
    offset = results.length;

}
result = DataSiv.StoreResults(results);
};

f().catch(ex=>DataSiv.StoreResults({error: ex.toString()}));
     
let data = []; 
data