4.3 Parsing and Interpretation of Visual Phrases in the User Interface

The text-graphic objects representing parses for four of the visual communication objects are shown in Figure 2 (the `spiderwebs' are a concise way of diagramming tree structure recovered by parsing; the more traditional notation will be shown later). The four parses were all performed in the same image space by the same function: parse-appropriately tries spatial-parse with each grammar from a list until the parse succeeds (if no parse succeeds, then ``unrecognized visual expression'' is signaled). The claim is that parse-appropriately represents progress is building user interfaces for formal visual languages: we now have one general purpose function with a list of `known' formal visual languages for which it can handle visual expressions. The elements for the expressions were created in a general purpose graphics editor, and then the spatial syntactic context of the elements was utilized in parsing them.

After parsing a visual expression, the structure thus recovered is then used directly by a specialized semantic processor in taking appropriate communicative actions. Interpretations based on the parses in Figure 2 are shown in Figure 3. Again, the four interpretations were all performed in the same image space by the same interpretation function, interpret-appropriately. interpret-appropriately calls parse-appropriately, and selects the proper semantic processor based on the result of the parse.