DISCUSSION NOTE
    TEMPORARY REDIRECTION OF INPUT DURING POPLOG STARTUP IN V15 VS V16

Although Poplog V16 (64 bit linux poplog for Intel/AMD CPUs) passes most
tests superbly well, there is an obscure problem concerned with handling
terminal input on startup, which was first noticed when the Popracer
package (product of a 2nd year undergraduate group project completed in
2006) did not work as expected when starting up: it did not deal properly
with terminal input. Trying to understand why, and where this should be
fixed, led me to explore a somewhat obscure network of issues to do with
terminal handling when poplog starts up. The problem is not yet clearly
identified or solved.

This is a partial progress report. I hope it will give someone else ideas I
have not thought of. I'll return to investigating this later after dealing
with more urgent tasks, unless someone else has solved the problem(s) by
then.

Aaron Sloman
14 Feb 2020

Apologies for length -- perhaps due to my missing something.

CONTENTS

 -- 1. INTRODUCTION
 -- 2. TRYING TO START POPRACER IN V15 AND V16 LINUX POPLOG
 -- 3. STARTING POPRACER INSIDE VED OR XVED
 -- 4. TRYING TO NARROW DOWN THE PROBLEM: FOCUS ON CHARACTER INPUT
 -- 5. SWITCH TO TESTING TEXT ITEM INPUT
 -- -- 5.1 Background information about Poplog's input streams
 -- -- 5.2 Creating input streams
 -- A POSSIBLE CLUE

========================================

-- 1. INTRODUCTION

Something has changed between Poplog V15.65 and Poplog V16 (henceforth V15
and V16) which I am finding hard to pin down. V15 uses 32 bit linux wheres
V16 uses 64-bit linux (which also supports 32 bit linux programs). Before
V16 became available, during 2019, I used V15 (actually V15.65) on various
Redhat based versions of linux. I have now also tested V16 on the same
versions of linux, primarily Fedora 29 and Fedora 31, though I also use
CentOS and Scientific Linux occasionally. Others have have been Poplog V16
successfully on Ubuntu, Debian and Arch.

Poplog V16, 64-bit Poplog for Intel/Amd-based linux systems ported by
Waldek Hebisch, is described in

    http://www.cs.bham.ac.uk/research/projects/poplog/V16/AREADME.html

(including prerequisites to be installed on various versions of linux,
and provision of a script to download and install V16).

It has been extremely successful so far, but there is an obscure problem,
related to redirection of text input, that first came to light when the
popracer package did not behave as expected.

Program and documentation available here:
    http://www.cs.bham.ac.uk/research/projects/poplog/popracer/
Download:
    http://www.cs.bham.ac.uk/research/projects/poplog/popracer.zip

Popracer is shown working in a video in this overview:
    https://www.tamias.co.uk/popracer

It runs impressively if started up in the right way in Poplog V15. In
V16 there are two problems:

(a) Getting it to start in the recommended way fails, apparently because of
a problem with text input mechanisms during startup -- which seem to have
changed between Poplog V15 and V16, though pinning down the exact change
has proved difficult (further details below),

(b) If Popracer is started up in the Ved editor in V16 poplog, it runs but
there is something wrong with either the cycle counter or the loop counter
test: it should run for many cycles, as it does in V15 poplog, but runs for
only one cycle, requiring repeated (tedious) clicking of the "start" button
to continue.

Comments:
This must be a loop exit test  not working properly. It is not clear how
any loop exit test could behave differently in V15 and V16. I have not yet
worked out exactly which bits of code are responsible for terminating the
loop, as I have mainly been focusing on (a), as the program is somewhat
obscurely written.

Popracer was produced as a group project by second year undergraduate AI
students at the University of Birmingham (UK) in 2005, using an earlier
version of poplog on a machine running CentOS, no longer available for
testing. A description of popracer, and a video showing the program working
(a long time ago) can be found here, made available by Mark Rowan:

    https://www.tamias.co.uk/popracer

The project report is available at

    http://www.cs.bham.ac.uk/research/projects/poplog/popracer/popracerreport.pdf

That directory also contains all the program files, and instructions in
    AREADME.txt

It can all be downloaded here (1.5MB):
    http://www.cs.bham.ac.uk/research/projects/poplog/popracer.zip

Note: The code is not easy to understand, especially the flow of control.
I shall later try changing the control structure, to see if that helps with
any of the problems.

While trying to locate the source of the startup problem in V16 I
encountered a number of ways in which low-level Poplog input handling
seemed to be working differently in versions 15 and  16. I have not yet
identified the exact differences.

I'll briefly describe the problems running popracer in V16 that don't exist
in V15, and then move onto some related problems concerning terminal input
in V16, but not V15. Feel free to skip the popracer problem descriptions
and jump to section 4. on character input.

-- 2. TRYING TO START POPRACER IN V15 AND V16 LINUX POPLOG

In V16 attempting to run popracer using the recommended command

    pop11 main.p

causes an infinite loop with requests for input being printed out
repeatedly, indicating that the call of 'readline' in file main.p is not
working properly in that context.

The command compiles the file main.p, and several additional files and then
enters a loop using readline to get information about setting up the
package before it is run via menu panels. This works in V15, but in V16 it
generates and infinite loop requesting input.

The same thing happens (in V16) if I run pop11 then

    : load main.p

The main.p file ends with

compilehere
       nnga.p
       physicsEngine.p
       bezierDraw.p
       simEngine.p;

The last file, simEngine.p, ends with the definition and invocation of a
procedure run, which uses a loop to collect information from the user, and
then display a demo window and a collection of boxes and buttons for
monitoring and controlling the simulation. Details don't matter here as
will become clear.

In poplog V15 and V16 running the program as

    pop11 main.p

produces the following printout:

     PopRacer v1.08 (c) Team Popcorn 2005 - Loading Modules...Final Release
         by Michael Brook
     * Neural Network and Genetic Algorithm Engine OK
     * Physics Engine OK.
       - Using Gravitional Constant:9.81
       - Using Friction Coefficient:0.002
     * Bezier Engine OK.
       - Track Editor OK.

     Welcome to PopRacer by Team Popcorn 2005
     *** Type help to list commands ***
     Pop Racer Command Line:

In V15 the program then pauses after printing the prompt 'Command', and
waits for a user command, e.g. 'help', or 'run', among others. Typing
'help' gives a list of commands, followed by a repeat of the command
prompt. Various setup options are available, and typing 'run' starts a run
of the program.

In V16 the program does not pause after that printout, and goes into an
unending loop requesting input, without waiting for the user to
type anything:

     CommandCommandCommandCommandCommandCommandCommandCommandCommandCommandCommandC
         ommandCommandCommandCommandCommandCommandCommandCommandCommandCommandC
         ommandCommandCommandCommandCommandCommandCommandCommandCommandCommandC
         ommandCommandCommandCommandCommandCommandCommandCommandCommandCommandC
         ....etc.

(I had to redirect output to a file to see the initial printout and Welcome
message, because otherwise I could see only an unending sequence of
'Command' prompts on the screen.)

That non-terminating process has to be interrupted (Ctrl C).

In V15 the program can be started either by using the linux command

    pop11 main.p

or by running pop11 and typing

    load main.p
or
    compile('main.p');

The program works as expected in all these contexts in Poplog V15, but not
V16.

In V16 the attempt to read text from the terminal fails repeatedly. I.e.
each call of readline immediately returns an empty list without waiting for
input. So there is a problem with the mechanisms invoked by readline to
read from the terminal in this context.

However, if the program is run in V16 in an editor buffer readline works,
as illustrated below.

Poplog provides a complex variety of machinery to enable programs to read
characters, text items, or lines of items from a terminal, from a file
being compiled or from a character stream that come from a terminal, a
network interface or a running program.

For an overview see REF PROGLIST (in Ved 'ENTER ref proglist'). The program
source files provide more detail, e.g. in

    $usepop/pop/src/*.p
    $usepop/pop/ved/src/*.p

and some other src directories, not relevant to this problem.

The Poplog code file for the procedure readitem, used by readline,
    $usepop/pop/src/readitem.p

has an interesting comment by John Gibson, that may be relevant.

    ;;; The next rather strange test is because calling -null- on -proglist-
    ;;; can actually change it (due to recursive calls of itemread,
    ;;; readitem etc inside xved inside the dynamic list procedure)
    if null(pl) or (proglist /== pl and null(proglist)) then

and
    --- John Gibson, Sep  4 1991
        Fixed horrible problem with null(proglist) -- see comment in code.

I so far cannot tell which portion of that (horrible) complexity accounts
for the difference between V15 and V16.

The behaviour of popracer suggests that something in V16 prevents the
procedure readline waiting for input from the terminal if it is involved in
a file given as argument to pop11, e.g.

    pop11 main.p

However the call of readline works if main.p is first read into the Ved
editor buffer, and the file is compiled using the VED command: ENTER l1

That works in V16, insofar as it allows the user to provide the required
input, but something else goes wrong when the program is run.

-- 3. STARTING POPRACER INSIDE VED OR XVED

The following method works in Poplog V15, and partly works in V16.

Read the file main.p into Ved, the poplog editor, with the command 'ved
main.p' or 'xved main.p', followed by

    ENTER l1

to compile the program. This does not produce an infinite loop -- i.e. the
call of readline works in this context in both V15 and V16, as follows:

A new Ved buffer or Xved window ('output.p') opens, displaying the same
text as was displayed in Poplog V15 running the program by typing 'pop11
main.p', starting:

    PopRacer v1.08 (c) Team Popcorn 2005 - Loading Modules...

ending with

    *** Type help to list commands ***
    Pop Racer Command Line:

It pauses there in both V15 and V16, waiting for input to be typed in via
the editor buffer.

Typing 'run'+RETURN then starts the graphical display of the figure 8 race
track and produces three movable panels used to display intermediate
results, plus a control menu window with Start, Stop and Quit buttons and
three buttons select graphs to be displayed.

In V15 clicking on 'Start' starts an extended run of the program showing
the racing cars evolving and improving. When the run stops it can be
extended by clicking on 'Start' again (i.e. it functions as 'continue').

In V16 the Start button has to be pressed repeatedly. It would need to be
pressed hundreds of times to produce the same result as in V15.

So something is preventing the main loop (defined in the file simEngine.p)
from working. I suspect that's a minor problem to be fixed after finding
out why the normal start mechanism doesn't work. I have not yet looked at
the button definitions for the Popracer control panel.

I'll ignore the details of Popracer from now now on. The rest of this
document is mainly about apparently related startup problems that can be
demonstrated without running Popracer.

-- 4. TRYING TO NARROW DOWN THE PROBLEM: FOCUS ON CHARACTER INPUT

I started looking for differences in the behaviour between V15 and V16
poplog when started up under various conditions, if a program needs to get
information from the user.

This turned up some surprises different from, but possibly related to the
Popracer startup problem.

Pop11 allows a variety of ways of creating, manipulating, and using input
streams, including compiling a text stream from a user or file to machine
code, running editor commands, reading characters from a data structure in
the current process, getting characters from an external file, or using a
network connection to get characters from a process on a named machine.

The standard input procedures, including charin, nextitem, itemread,
readline are defined so as to behave differently in different contexts,
giving enormous flexibility, especially for incremental compilation of
code at run time, but also for providing input for a running program.

When characters are read in, they can be used to build a new datastructure,
e.g. a word, a string, or a list, or to control decision making or to
extend an existing datastructure (or file), or fed into the pop11 compiler
to produce instructions to be obeyed, or stored for later use.

Many such reading processes can also, before they have finished, open a new
input stream of one of the above types and use it for one of the above
purposes, before continuing the original process. So input streams can be
nested in time. They can also be suspended and resumed.

For example, while compiling a file, a portion of the text read in can
cause input to be fetched from another source and processed in some way,
before the original compilation process finishes. Two examples are the
'include' and 'loadinclude' commands described in REF * PROGLIST.

The above uses, including switch of input, can occur if a file of pop11
code, or some other pop11 command stream includes a call of any of these:

    readline
    itemread
    incharitem  (create an item repeater from a character repeater)
        followed by a call of the new item repeater, as in
            incharitem(charin)() -> ....
    charin
    another previously created character-repeater, or item repeater

I suspect that in most programming languages such run-time consultation
will be restricted to reading in a character or string of characters to be
compared with a fixed collection of alternatives, whereas pop11 allows any
stream of characters to be interpreted as a stream of pop11, in which case
it needs to be itemised (chunked), macros run, procedure calls obeyed,
data-structures constructed, e.g. strings, procedures, etc.

Moreover that can happen temporarily in the middle of processing some other
input stream, if interpreting that stream triggers a new pop11 sub-process.

In pop11 this makes use of the fact that the compiler is part of the run
time system and can be used to compile some new text stream or string at
any time (even if it is already in the middle of doing such a thing).

Almost all of this also works as expected in V16. But there are a few
anomalies that may or may not be connected with the problems running
Popracer.

In Poplog V16 most of the above apparatus works perfectly, e.g.

shell command to read in characters to make up an item, e.g. a word

    pop11 ":incharitem(charin)()=>"
    : cat
    ** cat

or string (indicated by use of '....' in the input):

pop11 ":incharitem(charin)()=>"
: 'hello there'
** hello there

(By default pop11 doesn't print string quotes, following Pop2, to simplify
production of readable output, but probably a design flaw: the default
value of pop_pr_quotes should have been true, not false. See
REF PRINT.)

Anyhow in V16 even a loop in the command line works as expected:

    pop11 ":repeat 3 times incharitem(charin)() endrepeat =>"
    : cat
    : mouse
    : dog
    ** cat mouse dog

and a list can be built, in the following case a list with a word, a number
and a string:

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%] =>"
    : buy 6 'mice'
    ** [buy 6 mice]

and the items don't have to be on one line, e.g.:

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%]=>"
    : cat
    : mouse
    : 3.7986

** [cat mouse 3.7986]

and

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%] =>"
    : cat
    : horse dog
    ** [cat horse dog]

including reading in a string:

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%] =>"
: [
: (
: 'a string of 25 characters'
** [[ ( a string of 25 characters]

This time with string quotes printed:

     pop11 ":true -> pop_pr_quotes; [%repeat 3 times incharitem(charin)() endrepeat%] =>"
: a_word
: 99.99
: 'a string of 25 characters'
** [a_word 99.99 'a string of 25 characters']

So almost everything works in that context.

However readline cannot be used in that context in V16, even though it does
roughly the same as the above:

In V16

    pop11 ":readline()=>"

    ** <termin>

But it can be used in the older version, V15

    pop11 ":readline()=>"
    ? Will it read in a string of text and produce a list of words?
    ** [Will it read in a string of text and produce a list of words ?]

As can be seen in $usepop/pop/src/readline.p, the procedure readline depends on
readitem, so let's compare

    pop11 ":readitem() =>"
    ** <termin>

So, in both V15 and V16 readitem can't be used in the command string.
Likewise itemread

    pop11 ":itemread() =>"

Readline uses readitem, which also cannot be used in that context.

So it is hard to see how readline can work in a command string -- but it
does work in V15, but not V16, although the definition is the same in both:

vars
    pop_readline_prompt = '? ';

define vars readline();
    lvars item, list;
    dlocal popprompt = pop_readline_prompt, popnewline;
    if (isincharitem(readitem) ->> item) and fast_back(proglist) == item then
        fast_frozval(1, item) -> item;
        fast_cont(item) -> list;
        if ispair(list) and fast_front(list) == `\n` then
            fast_back(list) -> fast_cont(item)
        endif
    endif;
    true -> popnewline;
    [% until (readitem() -> item; item == newline or item == termin) do
            item
       enduntil
    %] -> list;
    if list == [] and item == termin then termin else list endif
enddefine;

All of which implies that in V16 at startup most of the input handling is
already set up as it should be, but for some reason not readline.

This test gives the same result in V15 and V16

    pop11 ":isincharitem(readitem)=>"
    ** <procedure (Sys$-Incharitem)>

So why does

    pop11 ":readline()=>"

work in V15, but not V16, where it produces only <termin>, without waiting
for input:

    pop11 ":readline()=>"
    ** <termin>



If I create a file testprog.p containing only

     [%repeat 3 times itemread() endrepeat%] =>

then

    pop11 testprog.p

doesn't give any input prompt, it merely prints out

** [<termin> <termin> <termin>]

     [%repeat 3 times itemread() endrepeat%] =>

However
    % pop11 ":compile(charin);"

works as expected. E.g.

    % pop11 ":compile(charin);"
: 99 *9 =>
** 891
: readline() =>
? the cat sat on the mat
** [the cat sat on the mat]

etc.


=======================

I think I have found the bit of documentation giving the specification
that is no longer satisfied.

REF SYSTEM specifies a variety of alternative ways of starting up poplog:

Quote:

    section:

    ----------------------------------------
    2  Starting up Standard Language Systems
    ----------------------------------------

    ......

     o  Otherwise, if the first character of  the argument is `:`, mode  (1)
        is chosen. The rest of the argument is treated as a string of source
        text, and  is  compiled. The  system  then exits.  (Note:  arguments
        containing `:`  usually have  to be  quoted, on  both Unix  and  VMS
        systems). E.g,

            pop11 ":1+2=>"

Endquote:

I.e. the above, apart from NOT printing out standard pop11 startup
string should behave the same as

    pop11

followed by

    1+2=>
    sysexit();

Likewise

    pop11 ":readline()=>"

should do the same as

    pop11

followed by

    readline()=>
    sysexit();

I.e.
- give a prompt,
- read in a line of characters,
- itemize it, build a list, print out '**' then print the list
- then exit.

It does that in old poplog (V15.65) but not in new poplog (V16).

In V16 (unlike previous versions of poplog)

    pop11 ":readline()=>"

just prints out

    ** <termin>

then exits.

I.e. readline isn't working in that context in V16 although it works in V15.65.

That suggests that something is being set up later in V16 than in V15.65



I'll try to narrow this down.

    ENTER sourcefile readline

shows the code for readline.

It includes

    if (isincharitem(readitem) ->> item) and fast_back(proglist) == item then
    ....

So I tried this:

     pop11 ":readitem() =>"

it fails in both Poplog V16 and the 32 bit Poplog v15.65. It simply prints

    ** <termin>

I.e. it acts as if the connection to the terminal has not yet been set up.

The following does work as expected in both old and new poplog:

    pop11 ":charin()=>"

It waits for input, reads a character and prints out the character code, as
expected.

Also this works in old and newpoplog:

    pop11 ":incharitem(charin)() =>"

incharitem takes a character repeater and returns an item repeater: i.e. it
produces a new procedure that can use charin() repeatedly
until the end of a Pop11 text item is reached. (i.e. what readitem does.)

But the following works only in the old poplog v15.65

    pop11 ":readline() =>"
    ? the cat sat on the elephant
    ** [the cat sat on the elephant]

In V16
     pop11 ":readline() =>"
    ** <termin>

The procedure readline is defined in
    $usepop/pop/src/readline.p

The code is exactly the same in old and new poplog. So the difference must
be in something invoked in procedure readline.

Readline makes use of readitem, after doing some setting up (below)XXXXX

In both old and new poplog readitem() doesn't work on the command line:

    pop11 ":readitem() =>"
    ** <termin>

-- 5. SWITCH TO TESTING TEXT ITEM INPUT

-- -- 5.1 Background information about Poplog's input streams

Poplog can read in and deal with streams of characters from different
sources, and the input stream can change dynamically (e.g. temporarily) to
get input from a user or device or datastructure, or a network interface,
while in the middle of processing another input stream.

E.g. while a file is being compiled, using an input stream of characters
from the file, a 'top-level' command may be encountered, requiring input to
be acquired from a user or another file while the file processing is
temporarily suspended. (Similar things can happen if the user types an
interrupt character, requiring input handling to be temporarily switched to
the terminal.)

For example if file1.p includes at some point, not inside a procedure
definition or comment, the instruction

    compile('file2.p');

then reading/compiling of file1.p will be suspended (ready to resume after
the next step is complete) and file2.p will be compiled. It too may be
suspended temporarily if it contains instructions to compile other files.

During such processes instructions may be encountered that request input
from a user. What the user types in response can then influence subsequent
processing (e.g. obeying a new command, or finding the answer to a new
question, or even redirecting input temporarily again). This can require
management of multiple input streams most of which have been temporarily
suspended.

Use of the poplog process mechanisms, allowing processes to be suspended
and resumed in an arbitrary order makes the requirements even more complex.

-- -- 5.2 Creating input streams

Pop11 allows a variety of ways of creating, manipulating, and using input
streams, including compiling a text stream from a user or file to machine
code, reading characters from a data structure in the current process,
getting characters from an external file, or using a network connection to
get characters from a process on another machine.

When characters are read in, they can be used to build a new datastructure,
or to control decision making or to extend an existing datastructure (or
file), or fed into the pop11 compiler to produce instructions to be obeyed,
or stored for later use.

Most such processes can also, before they have finished, open a new input
stream of one of the above times and use it for one of the above purposes,
before continuing the original process, including reading characters or
commands from the original source. See REF CHARIO

For example, the above uses can occur if a file of pop11 code, or some
other pop11 command stream includes a call of any of these:

    readline
    itemread
    incharitem  (create an item repeater from a character repeater)
        followed by a call of the new item repeater, as in
            incharitem(charin)() -> ....
    charin
    another previously created character-repeater, or item repeater

I suspect that in most programming languages such run-time consultation
will be restricted to reading in a character or string of characters to be
compared with a fixed collection of alternatives, whereas pop11 allows any
stream of characters to be interpreted as a stream of pop11, in which case
it needs to be itemised (chunked), macros run, procedure calls obeyed,
data-structures constructed, e.g. strings, procedures, etc.

Moreover that can happen temporarily in the middle of processing some other
input stream, if interpreting that stream triggers a new pop11 sub-process.

In pop11 this makes use of the fact that the compiler is part of the run
time system and can be used to compile some new text stream or string at
any time (even if it is already in the middle of doing such a thing).

Almost all of this also works as expected in V16. But there are a few
anomalies that need fixing.

=================

In Poplog V16 most of the above apparatus works perfectly, e.g.

shell command to read in characters to make up an item, e.g. a word
    pop11 ":incharitem(charin)()=>"
    : cat
    ** cat

or string (indicated by use of '....' in the input):

pop11 ":incharitem(charin)()=>"
: 'hello there'
** hello there

even a loop works as expected:

    pop11 ":repeat 3 times incharitem(charin)() endrepeat =>"
    : cat
    : mouse
    : dog
    ** cat mouse dog

and a list can be built, in this case a list with a word, a number and a
string:

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%] =>"
    : buy 6 'mice'
    ** [buy 6 mice]

and the items don't even have to be on one line, e.g.:

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%]=>"
    : cat
    : mouse
    : 3.7986

** [cat mouse 3.7986]

and

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%] =>"
    : cat
    : horse dog
    ** [cat horse dog]

including reading in a string:

    pop11 ":[%repeat 3 times incharitem(charin)() endrepeat%] =>"
: [
: (
: 'a string of 25 characters'
** [[ ( a string of 25 characters]

Repeat that, but with string quotes printed:

     pop11 ":true -> pop_pr_quotes; [%repeat 3 times incharitem(charin)() endrepeat%] =>"
: a_word
: 99.99
: 'a string of 25 characters'
** [a_word 99.99 'a string of 25 characters']




So almost everything works in that context.

Despite all that this doesn't work:

    pop11 ":readline()=>"

    ** <termin>


All of which implies that at startup most of the input handling is already
set up as it should be, but for some reason not readline.

If I create a file testprog.p containing only

     [%repeat 3 times itemread() endrepeat%] =>

then

    pop11 testprog.p

doesn't give any input prompt, it merely prints out

** [<termin> <termin> <termin>]

     [%repeat 3 times itemread() endrepeat%] =>


However
    % pop11 ":compile(charin);"

works as expected. E.g.

    % pop11 ":compile(charin);"
: 99 *9 =>
** 891
: readline() =>
? the cat sat on the mat
** [the cat sat on the mat]

etc.

=======================

-- A POSSIBLE CLUE

I think I have found the bit of documentation giving the specification
that is no longer satisfied.

REF SYSTEM specifies a variety of alternative ways of starting up poplog:

Quote:

    section:

    ----------------------------------------
    2  Starting up Standard Language Systems
    ----------------------------------------

    ......

     o  Otherwise, if the first character of  the argument is `:`, mode  (1)
        is chosen. The rest of the argument is treated as a string of source
        text, and  is  compiled. The  system  then exits.  (Note:  arguments
        containing `:`  usually have  to be  quoted, on  both Unix  and  VMS
        systems). E.g,

            pop11 ":1+2=>"

Endquote

I.e. the above, apart from NOT printing out standard pop11 startup
string should behave the same as

    pop11

followed by

    1+2=>
    sysexit();

Likewise

    pop11 ":readline()=>"

should do the same as

    pop11

followed by

    readline()=>
    sysexit();

I.e.
- give a prompt,
- read in a line of characters,
- itemize it, build a list, print out '**' then print the list
- then exit.

It does that in old poplog (V15) but not in new poplog (V16).

In V16:

    pop11 ":readline()=>"

just prints out

    ** <termin>

then exits.

I.e. readline isn't working in that context.

I.e. something is being setup later in V16 than in V15.65).

I'll try to narrow this down.

    ENTER sourcefile readline

shows the code for readline.

It includes

    if (isincharitem(readitem) ->> item) and fast_back(proglist) == item then
    ....

So I tried this:

     pop11 ":readitem() =>"

In both Poplog V16 and V15 it simply prints:

    ** <termin>

I.e. it acts as if the connection to the terminal has not yet been set up,
or has been closed.

The following does work as expected in both old and new poplog:

    pop11 ":charin()=>"

It waits for input, reads a character and prints out the character code, as
expected.

Also this works in old and newpoplog (incharitem takes a character repeater
procedure and returns an item repeater, which is then run, here:

    pop11 ":incharitem(charin)() =>"

incharitem takes a character repeater and returns an item repeater: i.e. it
produces a new procedure that can use charin() repeatedly until the end of
a Pop11 text item is reached (i.e. what readitem and itemread do: the
difference is that itemread expands macros.)

% pop11 ":incharitem(charin)() =>"
: Tomorrow is tuesday
** Tomorrow

That works as expected in both V15 and V16, i.e. reading in only one text
item, and discarding the rest of the input line.

In contrast, the procedure readline reads a text stream up to the end of
line,  breaks the stream into text items (e.g. words, numbers, strings,
etc.)

But the following works only in the old poplog V15:

    pop11 ":readline() =>"
    ? the cat sat on the elephant
    ** [the cat sat on the elephant]

In V16
     pop11 ":readline() =>"
    ** <termin>

It does not wait for any input, and does not print the prompt '?'.

The procedure readline is defined in
    $usepop/pop/src/readline.p

The code is exactly the same in old and new poplog. So the difference must
be in something invoked in procedure readline.

Readline makes use of readitem, after doing some setting up.

In both old and new poplog readitem() doesn't work on the command line:

    pop11 ":readitem() =>"
    ** <termin>

REF CHARIO explains what <termin> is:

 termin -> <termin>                                            [constant]
 termin_key -> key                                             [constant]
         The value of  termin is  the unique  item <termin>,  used as  an
         end-of-file marker for character/item  stream I/O (and also  for
         dynamic list streams, see REF * LISTS). termin_key is a constant
         holding the  key  structure of  the  unique item  <termin>  (see
         REF * KEYS).

So far I have found nothing that explains why readline works as expected on
the command line in V15

 pop11 ":readline()=>"
? the cat sat on the mat
** [the cat sat on the mat]

But not in V16 -- readline() just reads in end of stream.

 pop11 ":readline()=>"
** <termin>

If anyone understands this difference, please post an explanation to
pop-forum or to me.

Aaron Sloman
a.sloman [at] cs.bham.ac.uk
http://www.cs.bham.ac.uk/~axs
14 Feb 2020