Analyzed tokens

Concepts

Reading a digital edition produces a sequence of text units with specified semantics. In this documentation, we will refer to those units as tokens.

Implementation

The top-level concept of the hmt-textmodel is the TokenAnalysis. A TokenAnalysis has two members: a CTS URN identifying the text, and a complex HmtToken with analytical data. The HmtToken captures everything expressed about a single token by the XML documents of the HMT project, but the TokenAnalysis has high-level functions (illustrated here) for working with collections of those tokens. In general, these functions either:

• filter a Vector of analyses to create a new Vector of analyses, or
• transform a Vector of analyses into a Vector of citable text nodes