
From Aaron Sloman Mon Jun 15 10:58:53 BST 1998
To: nigel.birch@epsrc.ac.uk
Subject: EPSRC Human Factors Strategy
Dear Nigel,

Thanks for sending me the strategy document. I am sorry I was not able
to comment by the deadline of 12th June: I've been unable to keep up
with a host of deadlines in the past month.

I hope my hastily written comments a few days late will be of some use.
Let me know if you would like to have a nicely formatted printed
version. Apologies for length. I hope the tone gives no offence, but
fear that's inevitable.

I am making these comments available at my web site:

    http://www.cs.bham.ac.uk/~axs/misc/epsrc_human_factors.text

Aaron
=======================================================================


EPSRC: IT & COMPUTER SCIENCE PROGRAMME

DEVISING A STRATEGY FOR THE SUPPORT OF HUMAN FACTORS: A DISCUSSION
PAPER
URL http://www.epsrc.ac.uk/progs/technology/it-cs/hfstrcon.htm

Comments by Aaron Sloman:

CONTENTS
    (1) Defining the problem space
    (2) How do people change?
    (3) Cognitive architectures
    (4) The relevance of industrial research
    (5) Human factors and machine intelligence: increasing symmetry
    ADDITIONAL NOTES:
        A. A bizarre claim in the document
        B. The need for responsive mode funding
        C. This whole field is far too large to be managed on a national
           scale.
        D. Support for research infrastructure

My main comment is that I felt the document was not based on a proper
design analysis but simply reflected a collection of views of groups of
researchers who happened to see themselves as working in the field.

(I have not read the "People and Computers" booklet -- it may overcome
some of the limitations of this document.)

What I mean by a design analysis would start from an attempt to
characterise in a general and principled way what is currently known
about the total situation of a human being interacting with one or more
machines (and often simultaneously with other humans) in a social or
organisational context.

The list of headings at the beginning "People", "Domains of application"
and "Technologies led me to expect such a survey or the results of
such a survey, but I didn't find it --- perhaps I missed something.

Such a principled survey would include the following sorts of analyses:

(1) Defining the problem space
There are several dimensions of variation in human-machine interaction
including the following:

    (i) There are enormous differences between different kinds of people
    (some innate, some a result of individual learning, some a result of
    the culture, some a result of defects such as blindness or other
    disabilities, some of them motivational, some cognitive, ...).
    A good theory, and good designs, both need to take account of these
    differences in a principled way.

    (ii) There are enormous differences between the purposes for which
    people interact with machines. This includes both domains of
    application and what the user is doing in relation to the domain.
    E.g. controlling a system, learning to control the system, and
    finding out whether the control facilities are adequate to the
    domain are quite different purposes even when the domain is the
    same.

    (iii) Related to (ii) but partly orthogonal to it, there are
    enormous differences between the types of machines and modes of
    interaction they permit. Does anyone have a good theory of the
    dimensions in which these can vary? To be relevant it needs to
    be based on a theory of how minds and bodies work, so that only
    relevant dimensions are included. (This includes dimensions
    relevant to producing or preventing RSI and dimensions relevant
    to producing or preventing boredom, incomprehension, etc.)

    (iv) There are enormous differences in social and/or organisational
    contexts (home, school, office, emergency service, plane cockpit,
    station information service, etc.) Many of these impact on
    motivational and emotional processes. It makes a difference whether
    how you perform is compared with how others do.

Systems designed without a deep understanding of all these dimensions of
variation may end up being suitable only for tiny sub-spaces, with huge
loss of opportunity as a result.

Any general overview of the field should be based at least in part on a
survey of such dimensions of variation, and address the great difficulty
in covering all combinations in a national research strategy, or even an
international strategy.

Of course, it should be obvious that what I've called "dimensions" are
not dimensions in vector spaces with numerical coordinates. Each
"dimension" is itself a complex space. For instance the space of types
of minds and the space of types of purposes are not linear spaces and
can't even be characterised by vectors: they do not have a homogeneous
topology. (Unfortunately people who examine such spaces often restrict
themselves to what can be depicted on paper, e.g. 2-D tables and
graphs.)

Can the total space, combining all these dimensions of variations be
divided up in some useful way? A taxonomy of sub-regions in the total
space of combinations might provide a good way to think about how to
evaluate research (and research proposals) in this field.

Some points to note about these dimensions of variation:

Regarding (i) the same person can be a different kind of person in
different contexts, e.g. an advanced expert in some contexts and a total
novice in others. (I am useless at doing anything with a PC.) Are there
ways in which transfer of knowledge and skill can be facilitated? See
also point (2) below.

Regarding points (ii) and (iii) the situation is not static: both tasks
and technology are changing rapidly and any general research strategy
must include an attempt to get to grips with the changes and the
dynamics which drive them. Research in this sort of field can be out of
date soon after it starts.

(2) How do people change?
Related to point (i) above is the fact that people change in various
ways for all sorts of different reasons, including natural development,
increasing skill with practice, forgetting through lack of practice,
conceptual learning, illness, brain damage, senility, cultural changes,
changing interests and preferences, etc.

In particular, dramatic changes can be brought about by the
human+machine interaction itself e.g. through development of new skills,
learning of new concepts and ways of thinking, and exposure to new
possibilities which trigger motivational changes.

Thus one of the major research problems is to understand the variety of
processes of change in people (perhaps designers and managers as well as
users?) and the laws (if any) defining the space of possible
trajectories. This will, of course, be related to general issues in the
psychology of learning, development, motivation, perception, etc.
Computers are just a subset of machines, along with levers, pencil and
paper, packs of playing cards, window catches, etc.

(3) Cognitive architectures.
One of the most important facts about the total human+machine system is
that human beings have motivations and emotions, which make a huge
difference to what happens, as every teacher knows (though not all
know how to use this knowledge). This fact is not mentioned anywhere.

More generally, I get the impression that the contributors to section
(e) on cognitive science have a very piecemeal and probably excessively
narrow view of what is needed in an adequate theory of what's going on
when people interact with machines.

This is partly because of the difficulty and breadth of the field,
which increasingly has to take account of rapidly growing knowledge in
neuroscience as well as developments in many areas of psychology,
linguistics, AI, etc.

It could be important to understand the information processing
architecture of a human being as made up of layers which evolved at
different times and are only partly integrated, though there are rich
mutual interactions.

I also think that currently our knowledge of mechanisms capable of
explaining how humans and animals cope with spatial structure and motion
is pathetically limited. (Section (b) seems to suggest otherwise.)

Even though there's a great deal of factual information available about
WHAT such mechanisms can achieve (thousands and thousands of research
publications, for instance), I think we know very little about HOW they
achieve it, though there are many models of special, limited, cases
(e.g. edge detection - contrast the recognition processes of an expert
pianist sight reading a Beethoven sonata).

Understanding vision is crucial for any theory of how people can
interact with machines which increasingly use high bandwidth dynamic
visual forms of interaction, both by producing richer and faster moving
displays (2-D and 3-D) but also by using TV cameras watching the user
(including facial expression, posture, gestures, etc. as well as perhaps
the performance of tasks). This is an example of the growing symmetry
mentioned below.

As virtual reality systems become more sophisticated some of the
differences between processes within the machine and external processes
will diminish: e.g. machines will need sophisticated 3-D visual
capabilities to control simulated agents in simulated environments
possibly cohabiting with other agents controlled by, or representing,
users.


(4) The relevance of industrial research.
The document mentions industrial progress on speech as a reason for
doubting that academics can compete. If valid, I suspect this argument
could be generalised in relation to ALL research areas in human factors.

I don't work in speech but I would expect industrial/commercial
solutions to be partial and often ad hoc. The latest speech to text
systems working on continuous speech are very impressive. What about
speech *understanding*?

More generally, there is a huge amount of commercial/industrial/military
effort going into the design of new sorts of games, control interfaces,
decision aids, etc. This covers many areas of human factors. Is there
any hope that academic researchers can compete?

As with speech it may turn out that the industrial solutions are
specialised and partly ad hoc, leaving a need for more academic
research to address the deeper, long term, unsolved problems.

However, I suspect that far too many of the academics working in this
field are trained mainly as psychologists and not as system designers.

Consequently they know how to take measurements and how to run
statistical packages, and maybe even how to evaluate some systems, but
they are often not the right people to think about how to extend the
state of the art: that's because psychology degree courses (in this
country) do not train people to understand, produce, or debug
information processing engines or to understand their modes of
interaction. Graduates often don't know the difference between shallow
verbal descriptions of surface features of behaviours and deep
explanatory theories of the underlying mechanisms. (Of course there are
a few exceptions.)

So it is very likely that far more progress will come in the near future
from the extremely well-funded commercial and industrial laboratories:
e.g. those designing computer games or flight simulators, especially if
they employ gifted and creative designers rather than psychologists and
"human factors" experts. (Many of them have learnt from experience whom
to employ.)

Unfortunately the research in these commercially motivated labs (or
military labs) will not necessarily be published and made generally
available (except via products). It may also be too limited, and results
may not generalise beyond the specific applications.

There may be scope for some academics with a longer view and a broader
vision to come up with important new ideas that will not emerge from
industrial labs which have to feed into short term profits.
Unfortunately I believe the current dynamics of peer review and
political influences on selection criteria mean that it is unlikely that
such work will be funded.

EPSRC should not fund research of a type which is very likely to be
done in any case in a commercial laboratories unless it addresses some
deep issue or long term problem that is likely to benefit more from an
academic context than a commercially driven project.

In general, when funds are scarce, I would recommend trying to get
industry to fund the academic researchers interested in solving short
term practical problems (better interfaces for doing X) and use EPSRC
funds only for deeper, broader, more theoretically interesting research.

I know this goes against some recent political views. Academics (and
scientific administrators) should stand up more strongly against short
sighted politicians.


(5) Human factors and machine intelligence: increasing symmetry

I have often found that people who work on HCI or Human factors believe
that the systems they study are inherently asymmetric: with humans on
one side and machines on the other, and not much in common.

They may acknowledge that humans have complex cognitive mechanisms but
they don't think they need to understand the details of those mechanisms
(e.g. how perception, learning, problem solving, motivation, attention,
etc. work in humans) as long as they measure their effects.

I hope it is obvious that this is likely to lead to shallow research.

Moreover, it ignores the fact that increasingly machines are being
given, and will increasingly NEED to be given, human-like competences if
they are to be able to react naturally and fruitfully with human users.

The ability to understand, or at least transcribe, speech is an obvious
example where machines are making progress, and will need to make a lot
more progress if they are to understand the speech.

Another, which is not often noticed, is the requirement as graphical
interfaces become more sophisticated for machines to have the same kind
of grasp of visual structures as the users do. This is especially the
case when the structures are created by the user, not generated by the
machine from some formula. However, even machine generated images and
diagrams may be seen by the user in a way that does not correspond to
the structure which generated it. If the machine cannot look at what it
has drawn and find the new way of analysing and interpreting it, it may
be unable to comprehend a deep misunderstanding and take remedial
action.

More generally, machines with which we interact will become increasingly
autonomous and flexible in ways that require them to have rich cognitive
architectures, including capabilities concerned with motivation,
learning, planning, finding explanations, creating new designs and new
ideas, perceiving things, communicating in various ways, etc. In short
the asymmetry of the human+machine interface will diminish.

Achieving all this is unlikely to be possible without taking account of
how human minds and brains work even if some of the detailed mechanisms
are different. Some of the papers in a workshop to be held at AAAI98 on
toolkits for designing intelligent agents address this issue. See
    http://www.cs.bham.ac.uk/~bsl/aaai98/
Alas the workshop attracted only a narrow range of contributors.

It might be interesting to find out whether the UK Human factors
community can communicate with people thinking about such topics.

Unfortunately people are often too concerned with keeping up with their
own subfield, to learn about different approaches and perhaps too
concerned about dilution or diversion of scarce funds!


ADDITIONAL NOTES:

A. A bizarre claim in the document

Section (b) includes the statement:
"There is now a realisation that all sense organs are under the control
of the motor system".

This does not inspire confidence in the document or the process which
produced it.

That just happens to be ONE theory. Unless "the motor system" is given
an entirely new interpretation, or "control" means "partial influence",
I think this is just false. Sometimes it's the other way round: e.g. the
visual saccades triggered by detection of a change is an example of the
motor system being under the control of a sense organ. (I could enlarge
on the underpinnings of many forms of behaviour from startles to
sight-reading piano music, etc.)

More generally, motivational mechanisms control both sense organs and
motor systems. But not totally: perception is partly driven by incoming
data and some motor control processes can be highly skilled and
autonomous. Control is rarely one-way.

One way of reading section (b) suggests that sensory motor systems are
not deeply connected with the rest of the architecture (including for
instance motivational mechanisms, conceptual knowledge, etc.). If that
was intentional then it is surely deeply mistaken.

However I agree with the claim that understanding how biological systems
(including perhaps toad brains!) work is crucial for this field.
Compare my previous points about growing symmetry of the human machine
interface. However, it is not enough just to collect data on what the
biological systems can and can't do. We need deep explanatory theories.
Are there any?

B. The need for responsive mode funding

My feeling is that it is so difficult to do good research in this
field, and so hard to predict what sort of work will turn out to be of
long term value that the funding should be entirely responsive, provided
that the reviewers and selection committees are encouraged to be
both broad minded about new ideas, highly critical of ``more of the
same'' and willing to take chances with new researchers who have not
yet had a chance to prove themselves.

If responsive mode funding produces an imbalance that may not matter if
the research gaps are being filled in other countries. (See (C) below.)

But we do need mechanisms for reviewing research proposals which are not
loaded in favour of what the current community does, as often happens in
peer review. It may be better to assess the creativity, intelligence,
breadth of knowledge and productivity of grant proposers than the
precise content of what they propose to do!

I suspect some of the current imbalance is a result of the (real or
imagined) political pressure to produce "industrially relevant"
research with likely short term commercial benefits. This comes in part
from the following attitude expressed in the document:

     The task is thus to devise a strategy that will: (a) address the
     imbalance in the research portfolio; and, (b) provide a mechanism
     for involving users in the process of guiding the strategy, and
     taking up the results.

Commercial organisations that want "results" in this field will do
better to produce them themselves rather than depend on over-worked
under-resourced academics who in any case are, or should be, thinking
about the problems current industry has not yet dreamed of.


C. This whole field is far too large to be managed on a national scale.

Increasingly work has to be shared or divided across national
boundaries. If there are subfields that are better funded and making
more progress in other countries there may be no point in trying to
compete in the UK: except to ensure that there's a minimal competence
here which can feed into educational requirements.

This is true of all disciplines, but especially true in a field that is
very broad and inherently multi-disciplinary: in fact not a discipline
at all.

I am not advocating funding of multi-national collaborative projects: I
regard these as very wasteful.


D. Support for research infrastructure

As a result of drastically reduced funding over the last 20 years, and
redirection of funds through research councils, it is now very hard for
UK university labs to be adequately funded from university funds.

The falling cost and increasing power of computers makes it possible now
to have access to far more power than ever before. But some of the
research needs more than can come from departmental budgets, especially
where commercial software packages are needed. It is also hard to get
adequate support with installation and integration of software.

So there may be a case for encouraging relatively small grant
applications to "top-up" departmental equipment provision. Perhaps there
should also be a few sites which employ programmers who are available to
help and advise researchers in different institutions. It is no longer
necessary to be in the same room or even the same building to help
someone with software problems, especially if they don't use PCs! (For
instance I can often help people in other places using the toolkit I've
developed.)

This need for funds for infrastructure is not restricted to Human
Factors research of course.

It may also be useful to provide funds for multi-disciplinary workshops
designed to bring together people working on related topics who would
normally attend different conferences and workshops or contribute to
different journals. However it may be difficult to identify potential
contributors with the ability to communicate across boundaries and the
time to spare -- since academics now working under such intense
pressure. (I feel very sorry for younger colleagues now entering
academic careers.)


=======================================================================
I hope these hastily written comments are of some use. If I had more
time I'd try to find a way to make them more tactful!

Name
       Aaron Sloman,


Address
   School of Computer Science,
   The University of Birmingham
   Birmingham
   B15 2TT


Email: A.Sloman@cs.bham.ac.uk
URL:   http://www.cs.bham.ac.uk/~axs/
