Tables

The tables module provides functionality to read and write information stored on Kwanza and accessible from the configuration interface (Cuesta). It is meant to provide an easy way to add mapping or other types of tabular data to a flow. The UI for managing tables are shown below.

Note that only UTF8 formatted csv files are supported.

Navigation

Navigating the indvidual cells in the table can be done via the keyboard in a spreadsheet like manner. alt+<arrow-key> will move the focus depending on the arrow-key pressed. The video below shows an example of this (the keys pressed are shown in the bottom left corner of the video).

Inserting and removing rows

Inserting and deleting rows can also be done via the keyboard. Press ctrl+n to insert a row directly below the currently focused row.

Deleting a row is done via the ctrl+backspace key. It will remove the currently focused row.

Inserting and removing columns

This is done similarly to adding and removing rows but the cursor must be placed in the column header. ctrl+n adds a new column, while ctrl+backspace removes the current.

Shortcuts

Key	Action
alt+<down-arrow>	Focus cell below
alt+<up-arrow>	Focus cell above
alt+<right-arrow>	Focus cell right
alt+<left-arrow>	Focus cell left
ctrl+shift+a	Insert new row/column
ctrl+shift+backspace	Remove row/column

Read table as a map

The .map function will parse a table as a map, meaning that it will use a given column as an index. This is mainly useful if there is a column with unique values to use for the index. The returned structure will be a map with the column headers as keys.

Example

Given the table named foo:

A	B
idx1	val1
idx2	val2

And the code:

var m = Table.map("foo", "A");

You’ll get the following object back:

{
  'idx1': { 'A': 'idx1', 'B': 'val1' },
  'idx2': { 'A': 'idx2', 'B': 'val2' }
}

Which can then be used in the following manner:

var idx2val = m["idx2"]["B"];
// or if the column names are valid javascript identifiers
var idx1val = m.idx1.B;

Parameters

• name - [string] the name of the table to create a map from
• index - [string] the name of the column to use as an index
• options - [object] an optional options object which supports
  • useCache to set whether to allow use of a disk-based cache when fetching the table (default is true)

Read table as list of rows

The .rows function will return the raw table as a javascript array of arrays.

Example

Given the table named foo identical to the table from .map and the code:

var m = Table.rows("foo");

You’ll get the following object back:

{
  rows: [
    ["idx1", "val1"],
    ["idx2", "val2"],
  ];
}

Which can then be used in the following manner:

var idx2val = m.rows[1][1];

Parameters

• name - [string] the name of the table to create from
• options - [object] an optional options object which supports
  • useCache to set whether to allow use of a disk-based cache when fetching the table (default is true)

Update the contents of a table

The object returned from both .map and .rows contains a .save function which can be used to write data back to a table.

Examples

Update existing entries

Given the table from the previous examples and the code:

var m = Table.rows("foo");
m.rows[0][1] = "newval1";
m.save();

Will change the value of the specified cell and update the table.

If you add data to a table that contains double-quotes (") you should either [escape them yourself] or save with the escape: true option:

// ...
m.rows[0][1] = '{ "foo": "bar" }';
m.save({ escape: true });

Content generated by JSON.stringify

If you save JSON.stringify content in a table you should use the escape: true option. This will escape the content so that it can be read back correctly.

.save() also works if .map is used:

var m = Table.map("foo", "A");
m.idx1.A = "newval1";
m.save();

Add new entries

Adding to a table read by the rows approach:

var m = Table.rows("foo");
m.rows.push(["idx3", "val3"]);
m.save();

This will add a new row with idx3 and val3. When using rows the order of the input elements matter and should match the order of the columns.

The same information can be added when the table is read via the map approach as follows:

var m = Table.map("foo", "A");
m["idx3"] = { A: "idx3", B: "val3" };
m.save();

Remove entries

Removing a row from a table read by the rows approach is done by removing the corresponding array entry:

var rowToDelete = 0;
var foo = Table.rows("foo");
foo.rows.splice(rowToDelete, 1); // Delete the row w index 0
foo.save();

and the equivalent delete of a entry from a map table:

foo = Table.map("foo", "A");
delete foo["idx1"]; // Delete the entry with key 'idx1'
foo.save();

Use the contents of a Table as options for a typeahead

This is achieved by calling the selectFrom method on the structure created by the .map function. The selectFrom function takes either a format string or an object with options to generate the content for a typeahead.

var m = Table.map(...);
m.selectFrom('{{someColumn}} some text {{someOtherColumn}}');

Using a format string (above) and an object with options (below).

var m = Table.map(...);
m.selectFrom({
  format: '{{someColumn}} some text {{someOtherColumn}}',
  minInputLength: 3,
  filterMode: 'contains'
});

Tables as queues

It is possible to use a table as a sort of message queue. To do this use the Array.push, Array.pop as well as Array.unshift and Array.shift methods on a table. These will modify the table which can then be .save()d to the backend. The methods are safe to use concurrently from multiple machines.

var foo = Table.rows("foo");
// Enqueue two items to the table/queue
Array.push(foo, ["idx10", "val10"], ["idx11", "val11"]);
foo.save();
// Now we'll remove them again
var item1 = Array.pop(foo);
var item2 = Array.pop(foo);
foo.save();
// The save is needed to ensure the table has not been modified elsewhere

Tables as a means to synchronize flows

You can also use Tables as a global synchronization mechanism:

Update

The update function does check-and-write in one operation. Given the table:

Name	Age
Bill	42
George	21

You can increment the age of “Bill” like so:

Table.update("ages", function (row, index) {
  if (row[0] == "Bill") return [row[0], row[1] + 1];
});

The arguments for the function are the row as an array and the index of the row in the table.

I.e. you return the updated rows as individual arrays. You can stop the iteration by throwing any kind of error. In the example below we rename the first person we encounter who is older than 20:

var firstUpdate = true;
var updates = Table.update("ages", function (row, index) {
  if (!firstUpdate) throw "stop"; // kind of errors do not matter
  if (row[1] > 20) {
    firstUpdate = false;
    return ["Toby", row[1]];
  }
});

Deleting rows

You can also delete one or more rows by returning null or false, e.g. to delete all rows with an age less than 30:

Table.update("ages", function (row, index) {
  if (row[1] < 30) return false;
  return row; // otherwise keep the row
});

Inserting rows

You can also insert new rows by returning an array of the new rows, e.g. to insert a new person before and after “Bill”:

Table.update("ages", function (row, index) {
  if (row[0] == "Bill") return [["Alice", 25], row, ["Bob", 30]];
  return row;
});

This will insert two new rows before and after “Bill” making the table look like this:

Name	Age
Alice	25
Bill	42
Bob	30
George	21

The return value is a list of the rows that have been saved.

An optional 3rd options argument may contain:

• attempts how many attempts to update before giving up (default is 10)

Delete

If you just need to delete rows then you can use this function. Delete any rows for which the function given returns true.

var deletes = Table.delete("foo", function (row) {
  return row[6] === "123";
});

This will atomically delete all rows where the value in column 6 is '123' and return the deleted rows. Throwing an error also stops the iteration here. Like for update the 3rd optional argument may contain an attempts property.

Wait for a global lock

The Wait.forLock func has been modified to be able to grab a global lock backed by a Table. The first argument is the name of the table and then the global: true option must be set while optionally giving values for:

• wait how many seconds to wait to acquire the lock before giving up (default is 10s),
• hold how many seconds the lock is maximum held, after this duration any other flow will be able to grab the lock, even if you are still within the critical region (default is 60s)
• delay seconds to wait between trying to acquire the lock (default is 5s)
• reason a human readable reason for acquiring the log (will get written to the table so you can see it there)

An example would be;

if (
  Wait.forLock(
    "foo",
    function () {
      Wait.forSeconds(10);
    },
    { global: true },
  )
) {
  Notification.show("done", "I had the lock");
} else {
  Notification.show("done", "I did not get the lock");
}

Caveats

• If you have a great number of machines competing for e.g. a global lock then you’ll rather quickly run into some of the rate-limiting and throttling mechanisms in Kwanza and you’ll have a hard time getting access to a lock then
• The larger the clock-drift between Manatees the greater the risk of a Manatee grabbing a lock that it had no business grabbing. This is due to the fact that Manatee record a timestamp after which the lock is “automatically” released, and if another Manatee thinks the time is ripe for grabbing a lock it will do so.