A Structure from Manipulation for Text-Graphic Objects

The general purpose graphics systems of the future will need a simple logic for visual objects one structure underlying both text and graphics. As an experiment, perhaps the immediate handling of visual objects by the user can provide the starting point for developing that structure. This paper describes the PAM graphics system, in which the structure of text-graphic objects arises directly out of manual manipulation. The needs of manual manipulation determine the text-graphic pattern as the simplest organizing structure for images; PAM stands for PAttern Manipulating. The PAM system is designed for the agile manipulation of text-graphic patterns first manually, and then, later, programmatically.

Starting from this strict front-in viewpoint where immediate manipulation (hand powered animation) was to be the primary application a "manipulative grammar" was evolved to give the user a simple yet powerful handle on text-graphic images. This grammar turned out to be a generalization of LISP syntax from textual symbolic expressions to text-graphic forms, structuring such forms as trees and then offering:

Spatial grabbing of objects into attention;
Tree guided attention shifters like first, rest, next, and up;
Spatial & tree manipulations on any object in attention.

A Manipulative Grammar * for Text-Graphic Objects

A text-graphic object is either a line or a pattern

A line is a drawline or a character or a textline

A drawline is a vector chain drawn through one or more locations
A character is one or more drawlines
A textline is one or more characters

A pattern is a group of none or more lines and/or patterns

* (or a taxonomy, or even an "ontology" if you like, since an ontology is just a taxonomy with an attitude)

The resulting structures also offer surprising computational power (in a manner directly analogous to the way the basic list structures and functions of LISP give rise to the flexibility and power of a full blown LISP system, McCarthy and Talcott [1]), leading finally to computing with text-graphic forms.

Evaluation supports facilities like naming (and thus saving) visual objects, programming, and creation of menus (patterns of evaluatable function objects).

An experimental version of the PAM system has been implemented in MACLISP at the Stanford Artificial Intelligence Lab.

Key words and phrases: text-graphic objects, front-in design, hand powered animation, manipulative grammar, LISP, computing with text-graphic forms, phenomenology, interactive computer graphics, man-machine interface, graphics command language, graphics programming language, visual linguistics.

CR Categories: 3.36, 3.41, 4.22, 8.2.

Sections

Model of User on the Electric Blackboard (handPAM)
Selective Manipulation Leads to Structure
A Structure from Manipulation for Text-Graphic Objects
The Separation of Space & Tree
Handling Space & Tree Structure
The 4 Levels of handPAM
A Typical Blackboard Image
Ambiguities Resolved by Manipulative Grammar
Relation of handPAM to writtenPAM
Manipulative Grammar is Pre-semantic
Applications
Phenomenology and Graphic Interface Design
Notes on the Illustrations
Acknowledgements
References

The 4 Levels of handPAM

... the possibility of evaluating text-graphic objects is important to this paper. It means that writtenPAM possesses the processing power for doing linguistic analysis of text-graphic discourse (ie. the user interaction) in at least the same way that LISP facilitates such computation on textual interactions (Winograd's SHRDLU [15], Martin's DIP [9], etc.). In PAM as in LISP, processing power means providing atomic objects, ways of structuring them into complex objects, and equality tests for both atomic and complex objects.

A visual linguistics made possible by such processing could be useful in several ways. First, knowledgeable manipulation assistants could be built to increase the speed and dexterity of manual image handling; creation/modification of text-graphics at blackboard speeds is a very challenging application and help will be needed. And then, once good agility has been achieved in the manual handPAM medium, more complex machine understandings could be attempted of discourse taking place in that medium.

Phenomenolgy and Graphic Interface Design

Careful description of something without concern for its reality yields a phenomenology an exposure and delineation of the logic of the phenomenon. A phenomenon is literally a `that which appears', an appearing to a consciousness; phenomenology provides a way to give a precise account of such an appearing (Natanson [16]). Taking manipulation of text-graphic images as a phenomenon, this viewpoint allows us to describe it while for a time disregarding any hardware or software `behind the scenes' which `really' causes the existence and behavior of the images. What we have left is the human, the text-graphics, and his manipulation of them . . . which is the starting point for this paper.

Designing a graphics system in the method used for handPAM starting with data structures and operations which arise from text-graphic manipulation itself is thus a phenomenological approach. The design grows out of the logic of the phenomenon of manipulation as experienced from the human's point of view. Another way to think of this approach is as front-in design starting with the human and the graphics, and then (slowly!) moving toward the computer. It cannot be emphasized too strongly how important this phenomenological principle was in creating the PAM system: the project was born out of it and returned to it many times for renewal and guidance.

Key words and phrases: text-graphic objects, front-in design, hand powered animation, manipulative grammar, LISP, computing with text-graphic forms, phenomenology, interactive computer graphics, man-machine interface, graphics command language, graphics programming language, visual linguistics.

This paper was published in the proceedings of SIGGRAPH '80, Seattle, Washington, July 1980.

Return to papers list