4. COMPARISON OF VISUAL LINGUISTICS AND VERBAL LINGUISTICS

Spatial parsing can be used as a context to investigate the similarities and differences between visual linguistics and verbal linguistics. A preliminary itinerary for the explorations:

Similarities to verbal linguistics:

• First list item
• Symbolic input is a permissible finesse (rather than aural or raster images).
• Morphemes can be identified (in the sense of smallest meaningful unit, where `morpheme' relates to morphology and form, not phoneme).
• Visual languages have lexicons of visual terms.
• Visual languages have syntax: grammars can be written and used to recover syntactic structures.
• Visual languages have semantics: meaning under a system of interpretation.

Differences from verbal linguistics:

• Morphemes and lexicons are visual ¹², not aural or textual.
• Phrases are not linear strings, but rather 2-D text-graphic images.
• Phrases are not fading and transient but linger in space ¹³.
• The user has chronological freedom during phrase construction: the pieces can be put up in any temporal order as long as the spatial order is correct when the phrase is finally completed (and perhaps the phrase iscompleted when the spatial order is finally correct).
• The syntax and semantics of a textual phrase may be influenced by its graphic/spatial context (when a text-string is encountered while parsing a text-graphic image, how will visual linguistics handle it differently that verbal linguistics?).
• Many parsing strategies in textual linguistics depend on "used up the input string without any errors" as a termination condition. Often in spatial parsing a more appropriate termination condition is "searched in a direction for a distance and used everything found without any errors".
• Recursion in visual expressions almost always involves a decrease in scale for more deeply nested elements      the size differences must be accommodated, if not utilized, in spatial parsing algorithms.

Further visual linguistic research will try to uncover the common internal structures related to meaning which go across the different visual languages described in this paper. Such an "internal structure which is organized for meaning" is opposed to the "external structure organized for expression; the concrete form" ¹⁴ the visual communication objects. This paper has dealt with the varieties of external structure in visual languages, but we feel the tug of the linguistic quest: the intuition that although diagramming systems may vary, there are some universals which go across visual languages. We observe that the same meaning can be expressed by diagrams in two different notations using two different syntaxes. Given the right representation for semantics, then perhaps that same representation could be `run through' the two different syntaxes to produce the two different diagrams ¹⁵

¹² Alex Pentland coined the term `visual morpheme' for use in his work with the perceptual organization of natural form [Pentland86].

¹³ Richard Steele has pointed this out for the VIC language [Steele85].

¹⁴ Using the definitions of functional-structure and constituent-structure, respectively, from lexical-functional grammar [Bresnan82].

¹⁵ Relates to Mackinlay's work on automatic generation of diagrams [Mackinlay83].