Fields

Fields are used for interacting with the UI of an application. A field represents an element in the UI, a button, a text-field, a panel etc. Each type of field can be interacted with similarly to how a user can interact with the field. A button can be clicked, a text-field can have text written into it and text extracted from it and so on.

Fields are identified either by a path or a screenshot of the field. The screenshot is simply used to identify the field visually and should only be used if the field cannot be identified by a path. Paths are more flexible and robust with regards to changes in the UI of the application.

A path is a string which identifies the given field by containing instructions on how to traverse the tree that is the structural model of the UI. UIs are normally hierarchical in nature and structurally composed like a tree (e.g. the DOM of a web-page). The window is normally the root of the tree and it will contain a few panels as its children which again will contain other panels or UI elements directly. A path is thus a way to navigate from the root of the application (the outermost window) to the field we are interested in. An example:

/OuterPanel1/OuterPanel2/OkButton

This path specifies that we are ultimately looking for an OkButton element which is contained in an OuterPanel2 which itself is contained in an OuterPanel1. The elements along the path are matched on multiple criteria e.g. OkButton might be the actual text shown on the button or the class/type of button used.

The path editor (the text field in which the path to your field can be seen and edited) applies some formatting to make it easier to parse what is going on in the individual path elements.

Hovering over an element of the path will display some information about how that particular element is matched as well. You can read more about fields and how the individual elements match the nodes in the UI tree in the Manatee manual.

You can edit the path and its elements directly but if you right-click on an element you will get a menu in which you can edit an element using predefined controls. There are three distinct editors. The first one here is the editor for the window-match element (the first optional element in a field path).

Secondly there is a menu for all the intermediate elements where you can choose how the match should proceed:

And lastly there is a menu for the “tail” path element. The last element may contain a relation between a reference and a target element so it is more complex:

Creating fields

It is not normally necessary to manually construct these paths as Cuesta will connect to a locally running Manatee instance and try to figure out the path to a given field when you use the field-finder functionality in Cuesta.

Then you will simply point at the element in question and Cuesta will input the path for you. If the element is in a dialog that needs to be activated after the field finder is activated, you can hold down ctrl, which will pause the field finder while you activate the dialog.

Instead of clicking on the app element, you can also press spacebar to select the currently highlighted element.

Tree view

You can also use the tree view to explore the UI structure if the application and select an appropriate field to get a path for it.

Using a field in a flow

Once defined the field may be used in one or more flows for the application. For example the following code snippet clicks the button designated by the field name “OkButton”.

Fields['OkButton'].click();

A more complex example could be a field which represents a table:

// Inspect (i.e. get a JSON serialized version of) the table
var table = Fields['MyTable'].inspect();
// The `table` variable is an object serialized according to 
// the JSON serialization document
var rows = table.rows;
var someCell = table.rows[10]['Header1'];
// There is also functionality for clicking in a certain cell
table.clickCell('row1', 'Header1');