LiveGraph, a Robotic Web

Introduction

I wish to unify various ideas that have been current in the robotics and generally in science and engineering. The end effect should be a system in which any aspect of a robot lab can be realised as an object. Any such object can be presented topologically as a labelled node. If it is a physical object such as a camera or a table, it can also be presented geometrically.

The ideas come from:

(a) and (b) are characterised by having a graph or web in which various physical entities are represented by nodes and their relationships are represented by arcs. It is characteristic of simulators that activity representing the behaviour of the system is propagated through the web, passing into a node via its inputs and transmitted to successive nodes via its outputs.

The pose-determination problem is not often viewed in terms of a graph, but in fact there is a 2-node graph implicit in any formulation, e.g. Kumar's: the world is one node and the viewed body is another. Arcs in this case represent the body-feature to image-feature correspondence.

This unification is to be accomplished by supporting two capabilities over the web which it is useful to distinguish:

(1) Behaviour: Each node is contains code for a function and data. When a node is activated, the function is applied to the data treated as arguments. I use "function" in the C sense here, as a piece of code that may, and normally will, cause changes in its arguments. A node is said to be migratory if its behaviour can be made independent of the host environment for the web, that is the X-Window System and Poplog.

(2) Analysis: A node may contain information which supports analysis of portions of the web. An analysis procedure will examine a part of the web. As a result it may modify the web, or produce some structure which characterises an aspect of the web.

(3) Simplification: Some portion of the web may be replaced by a simpler, equivalent sub-web. For example multiple spatial relationships between two bodies in an assembly may be replaced by a single equivalent spatial relationship.

(4) Migration: The initial behaviour of a web will be rather like that of a novice piano-player. The necessary movements will be made, but not within the correct real-time schedule. However critical portions of the web can migrate to lower level controllers. An analysis procedure will be used to generate the programs that run in these controllers.

(5) Uncertainty Analysis: In creating the web, a primary guide will be the nominal behaviour of each node. However this is inadequate for predicting the correctness of a given configuration.

(6) Code Derivation: The code of many nodes depends functionally on certain mathematical (e.g. kinematic) properties. Thus it should be possible to derive the code (Yin did this for RAPT in the early 80's)

Logical and Geographical Presentations of the Web

The logical presentation of the web represents the nodes by graphical tokens whose nature is indicated by an alpha-numeric identifier (e.g. EdgeFinder), and in some cases by a stylised shape. Those nodes which represent physical objects, and so have a location (or pose), can be presented in a geographical format on a map.

Inspecting a Node to see what is inside.

Nodes can be inspected as a way of revealing their components as a data-structure. An inspection is a live-object just like any other entity appearing in the graphics widgets. It can be moved, deleted, grouped etc. Mousing (left button) on a field in the inspection can be used to inspect a sub-structure. Most compound sub-structures will be presented in a sub-inspection object. This facility is enabled only after hitting the "Inspect Object" button. Inspecting an component which is a named POP-11 procedure will result in the code of that procedure being displayed. This facility is not currently available for external (C) functions and for anonymous procedures.

All Displays can be Annotated, idraw Style, and we draw the web.

The mouse and keyboard handlers of all graphics widgets used by the web are set up to allow the user to annotate the presentation. The annotations, which are made in the style of the graphics program idraw are administered graphically in the same way as the web nodes (which are indeed a sub-class in the O-O hierarchy of entities which appear in a widget).

Thus the web itself is composed within the same set of conventions. One selects the kind of device one wants from a device menu. The left mouse button will then place an instance of the device on the screen, located to the grid if desired. Certain graphical objects (currently only straight lines) can be used to form connections between nodes. They are drawn in the usual way and then a connect button is activated on the main -Device menu. This will build a connection between an output of one device and inputs of another. The connection is made to a terminal which is within range of the ends of the connecting-graphical-object. This is then incorporated into the device whose output it is attached. Repositioning of devices to which connection is made will result in modification of the connection (...currently only on redraw).

Saving and Restoring Web Layouts

The state of a web can be saved to a disc file and restored from it. (Note this has only been tested for purely graphical objects devices will present some problems, since there are some entities (external code and anonymous procedures) which cannot be saved by the generic structure-saver.

It is intended to provide a capability to dump a widget in PostScript.

Some web-nodes can have a widget to themselves

Where the information that is normally presented to the user by a node is too extensive to be conveniently presented on the main "drawing-board", it is possible to encapsulate in a separate widget, leaving just a small "stub" in the board. Digitised images, for example, can be treated thus, although they can also appear on the main board. This latter is suitable only for small images.

In most cases it will be convenient to use a new widget for an inspection, so a special button is provided for this purpose.

Any selected group of graphic objects can be moved to a widget. Currently there is no logical way of doing this with devices.

Code can Migrate out of the Web into the Controllers (unimplemented)

A sub-web can be migrated. To do this, a mapping is constructed which takes each web-node to the processor to which it is to be migrated. For each such processor, we determine the object file for the function of each node which migrates to that processor. The migration-supervisor then builds a C-program for that processor. In this program, the arguments to which each node-function is applied are determined by inspecting the web. Code is generated to construct these arguments, with an equivalent structure-sharing to that found in the web. Data which is only accessed by nodes which are migrating to the same processor will reside in that processor's local memory. That which is accessed by nodes in the same cage will reside in the cage-local memory. That which is accessed by nodes not in the same cage will reside in the scram-net memory. Once the structure-building code has been generated, activation code is then generated. For this purpose the migrating sub-graph must be acyclic, with a single least "clock" node. Connection back to a cyclic structure characteristic of closed-loop behaviour is effected through the physical actuator-sensor link implicit in the real world, and/or through a relaxation step. The partial ordering inherent in this acyclic graph must be reflected in the calling-order for the code of the nodes. For a single processor there is an order-homomorphism from the portion of the acyclic graph mapped to that processor and the totally ordered call-sequence.

Specific Kinds of Web Node

Bundling and Unbundling Nodes

A bundling node has a number of inputs and a single output in which the inputs are formed into a vector (or array). An unbundling node converts a vector input into separate outputs.

Acis body models can be web-nodes.

The models produced by ACIS can be read into the system. An instance of such a model can then become a web-node. The input-terminal of a model-node is its pose (or location). Each feature is an output-terminal. The function of the node relocates the features from body-centered axes to axes determined by the input (normally world axes). All bodies have their location stored internally. Most physical objects are kinds of model-nodes.

A Lens is a web-node

A lens has focal-length, focal-distance and aperture as attributes.

A Camera is a web-node.

A camera is a body, but it has an additional attributes length. It has an additional input terminal which takes in 3-D features expressed in world coordinates. These appear in its output as projected 2-D features. There is no code associated with the actual operation of the camera, since those we have are not programmable in any way.

An Image is a web-node

Image data can be presented in a human-accessable form by an Image-node. An Image web-node has an input and a output. No transformation is performed, but the picture is presented to the user. There is a separate input for LinePic data which is superimposed on the intensity image. (There is also code around for other superimpositions, e.g. an edge-image).

A Digitiser is a web-node

A (single) image digitiser is a node which has as inputs A multiple digitiser has these inputs in multiple. (Note I thought at first to allow multiple instances of a given physical digitiser. However this is a shared resource, and having ONE actual node would seem to be the best way of supporting the control of contention, which, given the possibility of multiple threads of control, is a real problem.)

A LineFinder is a web-node

The LineFinder takes as inputs an environment and image, The environment serves to specify certain parameters such as the minimum length of line that is not to be weeded out. The outputs are an edge-strength image, a edge-direction image and a line-picture.

A Matcher is a web-node

The matcher takes as inputs two line-pictures. One is derived from the image and the other contains n-lines derived from from the projection of a model. The matcher finds a "good" match, mapping from model-lines to image-lines. As output it produces a vector (dr_1,da_1,dr_2,da_2....dr_n,da_n) of errors in (r,a) derived from the match.