Manatee v2.1

Appearance

Html ​

The Html module can be used to parse and query html formatted files and remote pages. It also contains encoding/decoding helper methods for html.

Requires Manatee v2.1 or greater

This version of the Html module cannot be used with Manatee v2.0 or earlier.

Loading data ​

The methods load and loadFrom can be used to load and parse a html document. They both return a HtmlDoc object which can be used for querying/extracting information.

javascript
// Load html from a string
var doc = Html.load("<html><body>Hello, world!</body></html>");
// Load html from an url
doc = Html.loadFrom("http://sirenia.eu");

Html.encode ​

Use this method to encode a string to replace unicode characters etc with their html encoded counterparts.

js
var encoded = Html.encode("1 < 2");
// encoded is now "1 &lt; 2"

Html.decode ​

Decode an already html encoded string also includes html5 named entities in the decoding.

js
var decoded = Html.decode("1 &lt; 2 = &angst;");
// decoded is now "1 < 2 = Å"

Html.btoa ​

Encode a string to base64.

js
var encoded = Html.btoa("Hello, world!");
// encoded is now "SGVsbG8sIHdvcmxkIQ=="

Html.atob ​

Decode a base64 encoded string.

js
var decoded = Html.atob("SGVsbG8sIHdvcmxkIQ==");
// decoded is now "Hello, world!"

HtmlDoc ​

The HtmlDoc object return from Html.load and .loadFrom has two primary methods for querying and extracting information from the html document it represents - the first is via an XPath query and the second is to convert the html to json.

XPath ​

The xpath method can be used to query the HtmlDoc with a given XPath query. All innerTexts are html decoded strings.

xpath always returns an array, even when the expression matches a single node, so index into it:

javascript
var d = Html.load("<html><body>Hello</body>");
var body = d.xpath("//body")[0];
Debug.showDialog(body.innerText); // shows "Hello"

Use querySelector instead if you want the first match directly. If the expression cannot be evaluated, xpath returns null rather than an empty array.

Converting to json ​

Converting the html to json is done with the .json() method. Each node in the resulting tree of objects has the following properties:

  • attrs an object containing the attributes of the html node
  • children is an array of child json nodes
  • innerText is a textual representation of the contents of the node (html decoded)
  • tagName is the name of the original html node

It also has xpath, querySelector and querySelectorAll methods which can be used to query the subtree of the json node as is possible for the HtmlDoc object.

javascript
var d = Html.load("<html><body>Hello</body>");
var json = d.json();
Debug.showDialog(json.tagName);

The json() function can also include #text nodes.

javascript
var d = Html.load("<html><body>He<br>llo</body>");
var json = d.json({ includeTextNodes: true });

This allows for better reconstruction of the original html using the html() function (perhaps after modifying).

javascript
var d = Html.load("<html><body>Hello</body>");
var json = d.json();
// Now we get back the original html (if possible)
var html = json.html();

html() serialises the node’s contents, not the node

Pass includeRoot to have the node included:

javascript
var body = d.querySelector("body");
body.html(); // the markup inside <body>
body.html({ includeRoot: true }); // <body>…</body>

QuerySelectorAll ​

Use the querySelectorAll method to query the HtmlDoc using CSS selectors.

js
// We'll assume we have a `HtmlDoc` object in `d`
var myClassDivs = d.querySelectorAll("div.myClass");

QuerySelector ​

The querySelector works similarly to the querySelectorAll except it returns the first hit only.

Table ​

The table(...) function can be used to extract js objects from html tables.

Given the table:

html
<table id="myTable">
  <thead>
    <tr><th>A</th></tr>
  </thead>
  <tbody>
    <tr><td>100</td></tr>
    <tr><td>200</td></tr>
  </tbody>
</table>

We can use the table function as follows:

js
// Assume we have the html already loaded in `d`
var t = d.table("#myTable");
// and now we can query the contents of the table as follows
var firstRowFirstColumn = t.rows[0]["A"];

The result also carries a header array, holding the column names in order.

If the table has no header information, rows holds a plain array per row instead of an object keyed by column name. So t.rows[0][0] rather than t.rows[0]["A"].

We can also use an object to pinpoint the header and/or the body of the table. This is useful if we have on our hands a table where the header is one location while the data is somewhere else. This is often the case for scrollable tables.

html
<table id="myTableHeader">
  <thead>
    <tr><th>A</th></tr>
  </thead>
</table>
<table id="myTableBody">
  <tbody>
    <tr><td>100</td></tr>
    <tr><td>200</td></tr>
  </tbody>
</table>

Now do this:

js
// Assume we have the html already loaded in `d`
var t = d.table(
  {
    tableAt: "#myTableBody",
    headerAt: "#myTableHeader thead tr th",
    rowAt: "#myTableBody tbody tr",
    cellAt: "td"
  }
);
// and now we can (again) query the contents of the table as follows
var firstRowFirstColumn = t.rows[0]["A"];

headerAt needs to point out the individual header elements, typically th elements, while rowAt must point out the tr elements in the table. cellAt selects the cells within each row.

The defaults ​

Every one of the four is optional. Passing a plain string, as in d.table("#myTable"), sets tableAt and leaves the rest at their defaults, which are derived from it:

optiondefault
tableAttable, [role='table']
headerAt<tableAt> thead tr:first-child th, or <tableAt> [role='columnheader']
rowAt<tableAt> tbody tr, or <tableAt> [role='row']. If neither matches, <tableAt> tr is tried
cellAttd, [role='cell'], resolved within each row

The ARIA alternates mean table(...) also reads grids built from role attributes rather than real table elements.

Always say which table

tableAt defaults to table, [role='table'], meaning any table or ARIA grid in the document. On a page with more than one, pass a selector that identifies the table you mean.