Table of Contents

• Introduction
• XPL/I Versus Standard XPL
• XCOM-I
• Installation of XCOM-I
  
• Linux or Mac OS
  
• Windows Specific
• Testing the Installation
• Emulating the IBM System/360 CPU
• Insane Bootstrapping!
• How to Build HAL/S-FC
• The Peculiarities of Standard XPL and Intermetrics XPL/I
  
• Program Options
• Built-In Runtime-Library Functions
• The MONITOR Built-In Runtime-Library Function
  
• Debugging XPL Programs
  
• Patches for Insertion of Inline Code
• More on IBM System/360 Inline Code
• Appendix:  Suitable C Compilers
  

Introduction

• The book remains the principal source of information about the language, and it's mainly about relatively-abstract topics.  There is no "user's manual" for XPL.  No "programmer's guide".  No tutorials as to how to get started with it.  Indeed, there is not even a "Hello, world!" program.  The closest thing to a sample program in XPL is the source code for a full-blown language compiler!  In short, there's no convenient way for a programmer to get started with it, except for immediately jumping into the deep end of the pool.
  
• The language was not designed for general-purpose applications.  It was tailored for the specific purpose of writing compilers for other computer languages, and writing compilers is not an everyday activity of large numbers of people.  For example, while there is an integer datatype, there's no floating-point datatype; so how could you easily write mathematically-intensive programs in it?
  
• The definition of the language is highly dependent on the characteristics of IBM System/360 mainframe computers, to the extent that even writing an XPL compiler that could run on non-compatible computers is quite a chore.  Indeed, the preface to the book explicitly states that they were creating a language specifically for the IBM 360.

XPL/I Versus Standard XPL

Aside:  For example, the previous attempt best-known to me of a modern XPL compiler is Daniel Weaver's XPL-to-C translator.  Alas, it is incapable of compiling Intermetrics's extended form of XPL.

Aside:  In case you're wondering, there is no known surviving copy of Intermetrics's own XPL/I compiler.  In fact, there is reason to believe that it was intentionally destroyed.  (Prove me wrong, please!)  If it had survived, however, I presume it would have been written in standard XPL and itself compiled by a standard XPL compiler on an IBM 360 computer.

XCOM-I

Aside:  XCOM-I itself is written in the Python 3 language, and getting XCOM-I to run on your computer is dead simple:  Just install Python 3, copy XCOM-I onto your computer, add it to your PATH, and you're ready to go!  Setting up your computer to compile the C code is something else, though.  Once again it's dead simple, if you have the common C compiler known as gcc.  But it's not so simple on (say) Microsoft Windows where most people use the far-less-compatible Visual Studio system.  And that's a lot of people.  So let's put the installation problem aside for a few minutes and come back to it a little later.

OUTPUT = 'Hello, world!';
EOF

XCOM-I.py --xpl HELLO.xpl
make -C HELLO.build

./HELLO

PAGE 1


Hello, world!

-rw-rw-r-- 1 rburkey rburkey  11207 Aug 12 07:23 COMPACTIFY.c
-rw-rw-r-- 1 rburkey rburkey   1730 Aug 12 07:23 configuration.h
-rw-rw-r-- 1 rburkey rburkey  16921 Jul  7 10:50 debuggingAid.c
-rw-rw-r-- 1 rburkey rburkey   4269 Aug  5 09:01 inline360.c
-rw-rw-r-- 1 rburkey rburkey   1195 Aug  5 09:01 inline360.h
-rw-rw-r-- 1 rburkey rburkey   1835 Aug 12 07:23 main.c
-rw-rw-r-- 1 rburkey rburkey   4545 Aug 10 17:07 Makefile
-rw-rw-r-- 1 rburkey rburkey  24759 Aug 12 07:23 memory.c
-rw-rw-r-- 1 rburkey rburkey    883 Aug 12 07:23 procedures.h
-rw-rw-r-- 1 rburkey rburkey    161 Aug 12 07:23 resetAllReentryGuards.c
-rw-rw-r-- 1 rburkey rburkey 125864 Aug  9 13:52 runtimeC.c
-rw-rw-r-- 1 rburkey rburkey  18037 Jul 18 10:52 runtimeC.h

XCOM-I.py --help

No code is generated for the following PROCEDURE(s):
	Not called:  DESCRIPTORuMONITOR
	Not called:  uRECORDuCONSTANT
	Not called:  uNEEDMOREuSPACE
	Not called:  uRECORDuSEAL
	Not called:  uRECORDuUNSEAL
	Not called:  uRECORDuGROUPHEAD
	Not called:  uRECORDuCOORDINATED
No code is generated for the following PROCEDURE(s):
	Not called:  uINCREASEuRECORD
	Not called:  uALLOCATEuSPACE
No code is generated for the following PROCEDURE(s):
	Not called:  uATTACHuRECORD
	Not called:  uGETuSPACE
	Not called:  uHOWuMUCH
No code is generated for the following PROCEDURE(s):
	Not called:  uFINDuFREE
No code is generated for the following PROCEDURE(s):
	Not called:  uSTEAL
	Not called:  uMOVEuRECS

-rw-rw-r-- 1 rburkey rburkey   2890 Aug 12 07:52 uACTIVEuDESCRIPTORS.c
-rw-rw-r-- 1 rburkey rburkey   2040 Aug 12 07:52 uATTACHuBLOCK.c
-rw-rw-r-- 1 rburkey rburkey   1485 Aug 12 07:52 uATTACHuBLOCKxJOIN.c
-rw-rw-r-- 1 rburkey rburkey   1086 Aug 12 07:52 uCHECKuFORuTHEFT.c
-rw-rw-r-- 1 rburkey rburkey  21641 Aug 12 07:52 COMPACTIFY.c
-rw-rw-r-- 1 rburkey rburkey   1747 Aug 12 07:52 COMPACTIFYxADDuDESC.c
-rw-rw-r-- 1 rburkey rburkey   1711 Aug 12 07:52 configuration.h
-rw-rw-r-- 1 rburkey rburkey  16921 Jul  7 10:50 debuggingAid.c
-rw-rw-r-- 1 rburkey rburkey   1330 Aug 12 07:52 uDETACHuRECORD.c
-rw-rw-r-- 1 rburkey rburkey  10136 Aug 12 07:52 uFREEBLOCKuCHECK.c
-rw-rw-r-- 1 rburkey rburkey   1157 Aug 12 07:52 uFREEBLOCKuCHECKxADDRESSuCHECK.c
-rw-rw-r-- 1 rburkey rburkey   4322 Aug 12 07:52 uFREEBLOCKuCHECKxBLKPROC.c
-rw-rw-r-- 1 rburkey rburkey   4269 Aug  5 09:01 inline360.c
-rw-rw-r-- 1 rburkey rburkey   1195 Aug  5 09:01 inline360.h
-rw-rw-r-- 1 rburkey rburkey   7047 Aug 12 07:52 main.c
-rw-rw-r-- 1 rburkey rburkey   4545 Aug 10 17:07 Makefile
-rw-rw-r-- 1 rburkey rburkey  31342 Aug 12 07:52 memory.c
-rw-rw-r-- 1 rburkey rburkey   1692 Aug 12 07:52 uMOVEuWORDS.c
-rw-rw-r-- 1 rburkey rburkey   1176 Aug 12 07:52 uPREVuFREEBLOCK.c
-rw-rw-r-- 1 rburkey rburkey   1142 Aug 12 07:52 uPREVuRECORD.c
-rw-rw-r-- 1 rburkey rburkey   5236 Aug 12 07:52 procedures.h
-rw-rw-r-- 1 rburkey rburkey   2419 Aug 12 07:52 uRECORDuFREE.c
-rw-rw-r-- 1 rburkey rburkey   1597 Aug 12 07:52 RECORDuLINK.c
-rw-rw-r-- 1 rburkey rburkey   3990 Aug 12 07:52 uREDUCEuBLOCK.c
-rw-rw-r-- 1 rburkey rburkey    652 Aug 12 07:52 resetAllReentryGuards.c
-rw-rw-r-- 1 rburkey rburkey   2362 Aug 12 07:52 uRETURNuTOuFREESTRING.c
-rw-rw-r-- 1 rburkey rburkey   6157 Aug 12 07:52 uRETURNuUNUSED.c
-rw-rw-r-- 1 rburkey rburkey 125864 Aug  9 13:52 runtimeC.c
-rw-rw-r-- 1 rburkey rburkey  18037 Jul 18 10:52 runtimeC.h
-rw-rw-r-- 1 rburkey rburkey    475 Aug 12 07:52 uSPACEuROUND.c
-rw-rw-r-- 1 rburkey rburkey   2023 Aug 12 07:52 uSPMANERR.c
-rw-rw-r-- 1 rburkey rburkey   7067 Aug 12 07:52 uSQUASHuRECORDS.c
-rw-rw-r-- 1 rburkey rburkey   4714 Aug 12 07:52 uTAKEuBACK.c
-rw-rw-r-- 1 rburkey rburkey   1466 Aug 12 07:52 uUNUSEDuBYTES.c

/*
  File main.c generated by XCOM-I, 2024-08-12 08:01:10.
  XPL/I source-code files used: XPL.LIBRARY.xpl HELLO.xpl.
  To build the program from the command line, using defaults:
          cd HELLO/
          make
  View the Makefile to see different options for the `make`
  command above.  To run the program:
          HELLO [OPTIONS]
  Use `HELLO --help` to see the available OPTIONS.
*/

#include "runtimeC.h"

// clang-format off
/*
            *** Memory Map ***
  Address (Hex)   Data Type   Variable
  -------------   ---------   --------
  1318 (000526)   BASED       userMemory
  1346 (000542)   BASED       privateMemory
  1376 (000560)   FIXED       COMPACTIFYxI
  1380 (000564)   FIXED       COMPACTIFYxJ
  1384 (000568)   FIXED       COMPACTIFYxK
  1388 (00056C)   FIXED       COMPACTIFYxL
  1392 (000570)   FIXED       COMPACTIFYxND
  1396 (000574)   FIXED       COMPACTIFYxTC
  1400 (000578)   FIXED       COMPACTIFYxBC
  1404 (00057C)   FIXED       COMPACTIFYxDELTA
  1408 (000580)   BIT(16)     COMPACTIFYxDX(500)
  2412 (00096C)   FIXED       COMPACTIFYxMASK
  2416 (000970)   FIXED       COMPACTIFYxLOWER_BOUND
  2420 (000974)   BIT(1)      COMPACTIFYxTRIED
*/
// clang-format on

int
main(int argc, char *argv[])
{

  // Setup for MONITOR(6), MONITOR(7), MONITOR(21).  Initially,
  // entire physical memory is a single pre-allocated block.
  MONITOR6a(USER_MEMORY, PHYSICAL_MEMORY_LIMIT, 0);
  MONITOR6a(PRIVATE_MEMORY, PRIVATE_MEMORY_SIZE, 0);

  if (parseCommandLine(argc, argv)) exit(0);

  static int reentryGuard = 0;
  reentryGuard = guardReentry(reentryGuard, "main");
  // OUTPUT = 'Hello, world!'; (0)
  {
    descriptor_t *stringRHS;
    stringRHS = cToDescriptor(NULL, "Hello, world!");
    OUTPUT(0, stringRHS);
    stringRHS->inUse = 0;
  }

Installation of XCOM-I

Linux or Mac OS

On a Mac, you will need to install Xcode command-line tools before doing anything else.  If you instead want to install the entirety of Xcode, I'm sure that's fine too, but the only thing you'll actually need is the much-smaller package of command-line tools.  Note that my very old Mac (Xcode 4.6) hasn't been updatable for a long time now, but apparently the instructions you'll find throughout this page do work on Xcode 14.2 as well; whether they work on the current version of Xcode, I cannot say.

On Linux, you usually don't have to go out of your way to install a C compiler, but gcc or alternately clang will be available from your package repository if you do need to explicitly install either or both of them.

1. Download the entire Virtual AGC source tree.  For brevity, I'm going to assume that this ends up as a folder called virtualagc, but you need to substitute the actual name in the instructions below.  My personal suggestion would be to get it via git, as in the following command, because that makes it easy to keep up-to-date later via "git pull":

git clone --depth=1 https://github.com/virtualagc/virtualagc.git

2. Add the following folders to your PATH  (Hint:  In Linux, I usually do this by editing ~/.bashrc; in Mac OS, by editing /etc/paths.  But Google it if you're unsure!)  Afterward, you may need to close and reopen any terminal windows in order for the change to take effect.: 
   
• "virtualagc/XCOM-I"
• "virtualagc/yaShuttle/ported/PASS1.PROCS"
• "virtualagc/ASM101S"
• virtualagc/yaShuttle/Source Code/PASS.REL32V0"
3. Install Python 3, if not already installed.
4. Optionally, install clang-format.  This allows you optionally to "pretty print" the C code that XCOM-I generates. 
   

• Installation is easy on Linux (at least on Linux Mint), because clang-format is in the official package repositories. 
  
• On Mac OS, clang-format is apparently not included in Xcode.  You can do this:  Install homebrew, if you don't already have it.  Then use the command "brew install clang-format".
  

Cross-compiling from Linux or Mac OS to Windows

Cross-compiling XPL files on Linux or Mac OS to create EXE files for Windows seems like a niche option.  In fact, probably nobody but me would be interested, and I'm only interested because it saves me the trouble of firing up a Windows virtual machine to make EXE files for the HAL/S compiler!   But even if it's just for myself, I need to write down how to set it up, so here are the instructions.  Basically, you just need to install a C cross-compiler (mingw-w64) and a Windows emulator (WINE).

First, you have to install XCOM-I as described in the preceding section.

In Linux (at least Linux Mint), you should install mingw-w64 using your usual package manager.  Even if WINE is available in your package manager, though, I might suggest installing it directly from the WineHQ.org, in order to get a better outcome.  (For me, for example, that's the difference between WINE 6 and WINE 9.)

For relatively-new Mac OS versions, though, this should work:  First, install macports.  Then do this:

sudo port install mingw-w64 wine-stable

Aside:  As it turns out, my mid-2007 Mac Mini (Mac OS Lion) isn't "relatively new", and thus the steps just mentioned didn't work for me.  Nor did the seemingly very-attractive alternative of installing llvm-mingw in place of mingw-w64.  And as for WINE, its wiki says explicitly that it works only for Mac OS Catalina or later.  Oh well, no great loss!  I doubt that many Mac users want to build Windows EXE files anyway.

Windows Specific

The problem in Windows is not XCOM-I per se, which works as-is, but rather to set up some kind of reasonably-compliant environment for compiling the C files produced by XCOM-I.  That's more effort in Windows than in Linux or Mac OS, since you generally don't have to do anything to achieve it in Linux or Mac OS anyway.

Aside:  A choice that will likely be obvious to most people is to use the Windows Subsystem for Linux (WSL).  I have been unable to try it.  I purchased Windows 11 to do so, but I'm running Windows 11 in a virtual machine myself, and it turns out that you cannot run WSL on Windows inside of a virtual machine.  Too bad, though hardly surprising!  If you try WSL on a physical machine and make it work, let me know.

Option 1: MSYS2

1. Install MSYS2, which gives you a Linux/Mac-like command-line working environment for compiling stuff. 
   
2. Run MSYS2.
   
3. Install various packages with this command:  

pacman -S mingw-w64-ucrt-x86_64-gcc make grep diffutils python3 python3-pip git clang

4. Download the entire Virtual AGC source tree.  For brevity, I'm going to assume that this ends up as a folder called virtualagc, but you need to substitute the actual name in the instructions below.  My personal suggestion would be to get it via git, as in the following command, because that makes it easy to keep up-to-date later via "git pull":

git clone --depth=1 https://github.com/virtualagc/virtualagc.git

5. Add the following folders to your PATH:  "virtualagc/XCOM-I", "virtualagc/yaShuttle/ported/PASS1.PROCS", and "virtualagc/yaShuttle/Source Code/PASS.REL32V0".  In MSYS2, you'd do that by editing the file ~/.bashrc, which you can do with the command "nano ~/.bashrc".  Google how to permanently change the PATH in Linux if you're unsure what to do.  Afterward, you will need to exit MSYS2 and rerun it to see the change take effect.

It's worth noting that if you followed these instructions to the letter, then your "home directory" in MSYS2 will probably be something like C:\msys64\home\USERNAME, whereas your home directory for a normal command-line would be c:\users\USERNAME.  That's good to keep in mind if you need to transfer any files in or out of there!

Option 2: Visual Studio

Warning:  I have never used Visual Studio prior to this, or if I have then it was so long ago I no longer remember doing so.  Besides which, for XCOM-I I'm only using Visual Studio's command-line tools rather than its Integrated Development Environment (IDE).  The upshot of this that you shouldn't interpret my giving you Visual Studio instructions as meaning you're going to get the wonderful integration into Visual Studio that you might dream of.  You might, but you'll have to work it out yourself!

Note that almost everything I'm going to tell you to do, either here in the setup, or elsewhere on the page, is done from what's called the Developer Command Prompt for Visual Studio, which you can run from the Windows Start menu if you have Visual Studio installed.  You won't be working from a normal (non-developer) command prompt most of the time.  I think the main reason you have to do this is that the C compiler (cl) is in its PATH, but not in the PATH in a normal command prompt.  You'll have to 'cd' to some other directory, though, to get anything done, since you'll have no permissions to actually modify anything in the working directory this plunks you into.  Isn't that a fun feature?  It's your own choice, of course, but perhaps you could go to your normal command-line home directory.

python XCOM-I.py ... 

• Get the entire Virtual AGC source-code tree from GitHub. 
  
• Add the following folders to your PATH:  "virtualagc/XCOM-I", "virtualagc/yaShuttle/ported/PASS1.PROCS", and "virtualagc/yaShuttle/Source Code/PASS.REL32V0"..  If you've already opened a VS developer command prompt, then close it and start a new one to see the effect of the changed PATH.
• Install Python 3, if it's not already installed. 
  
• Permanently set the environment variable PYTHONUTF8=1 to enable Python's UTF-8 processing ability.  
• Obtain a zipfile of numerous UNIX-style utilities from here.  Extract the contents somewhere on your local computer, and add the subdirectory called "bin" to your PATH.  (Remember to close and then reopen any developer command line you may have open, in order to see the effect of the change to the PATH.)  
• The version of GNU make provided with these UNIX-style utilities is too out-of-date (version 3.81) to be fully satisfactory for use with XCOM-I.  So you'll have to install a newer version.  Here's a reasonably-easy way to do that:
• Download the latest source code from here (version 4.4.1 as I'm writing this) and extract it.
• From a Visual Studio developer command line, 'cd' in to the source-code directory just created.
• Run the batch script called build_w32.bat.  This creates the file gnumake.exe in the subdirectory called WinRel/.
• Rename gnumake.exe as just make.exe.
  
• Get it into your PATH somewhere that precedes the older version of make found in the UNIX-style utilities.  What I do is just copy it atop the older version in the UNIX-style utilities' bin/ folder.
• You can verify the version by running make -v.
  

Testing the Installation

We can do a very quick test of your setup and of XCOM-I itself by compiling and running the HELLO.xpl program I used as an example earlier.

• The XCOM-I --pp option:  This "pretty prints" the C-language files produced by XCOM-I, so that the indentation is nicer, the code doesn't trail out of view past the right-hand margin, and so on.  On the other hand, it may also mess up column alignment in comments.  You pays your money and takes your chances!  In terms of how the compiled program behaves, it makes no difference whatsoever.
  
• The -s in the make command:  Optional.  It just prevents make from giving you a running commentary on whatever commands it happens to be running.  No big deal!
  
• CC=clang EXTRA=-w in the Mac OS make command:  Probably not necessary.  The same make command as for Linux could have been used.  And clang is the default anyway in Mac OS anyway, I think, so adding CC=clang just makes it explicit.  In fact, to force it to use gcc, you'd have to use the command make -s CC=gcc EXTRA="-w -std=c99" -C HELLO.build.  In my experience, clang produces a lot of pointless (and incorrect) warning messages, which is why I disable all of the warnings by adding EXTRA=-w. 
  
• RDYNAMIC= in the Windows make command:  Disables a compiler command-line option that the Windows version of gcc doesn't support.
  

make -s -C Example-6.18.3.build
make -s -C Example-6.18.4.build
make -s -C Example-6.18.5.build
make -s -C Example-6.18.6.build
make -s -C breakCharactersDemo.build
make -s -C ANALYZER.build
make -s -C SKELETON.build
make -s -C bitsizeDemo.build
make -s -C bitsizeDemoCommon.build
make -s -C bitsizeDemoCommonIn.build
Files bitsizeDemo.build/out.txt and Tests/bitsizeDemo.regression are identical
Files bitsizeDemo.build/xxxxA and bitsizeDemoCommon.build/xxxxA are identical
Files bitsizeDemoCommon.build/xxxxA and bitsizeDemoCommonIn.build/xxxx are identical
Files bitsizeDemoCommon.build/COMMONa.out and bitsizeDemoCommonIn.build/COMMONa.out are identical
Files Example-6.18.3.build/out.txt and Tests/Example-6.18.3.regression are identical
Files Example-6.18.4.build/out.txt and Tests/Example-6.18.4.regression are identical
Files Example-6.18.5.build/out.txt and Tests/Example-6.18.5-and-6.regression are identical
Files breakCharactersDemo.build/out.txt and Tests/breakCharactersDemo.regression are identical
Files ANALYZER.build/out.txt and ANALYZER.build/out.regression are identical
======================== All regression tests passed! ========================

Emulating the IBM System/360 CPU

XPL or XPL/I programs compiled by XCOM-I are native to your own local computer system, and if XCOM-I has done its job properly, there should be no lingering dependencies on the IBM System/360 in the compiled programs.

Nevertheless ... XPL was designed to be a program for generating compilers, so a lot of legacy XPL or XPL/I code is actually the source code of a compiler for one or another computer language, and those compilers invariably spit out IBM System/360 object code. Suppose for example that we had an XPL program called X that was a compiler for a programming language called "X". It's true that if we compile the source code for X with XCOM-I that we'll get executable code for X that runs on our native Linux/Mac/Windows computer (rather than, say, an IBM 360). But it's also true that if we use that native-Linux, Mac, or Windows version of X to compile source-code in the X language, say for a program called Y, then the output of the compiler will still be an object file for the IBM System/360. So we still wouldn't be able to run program Y on our native computer, and would still need an IBM System/360 to run Y.

Which means that even with XCOM-I in hand, it's still very useful to have an IBM System/360 emulator in which to run not the programs produced by XCOM-I, but the programs produced by the compilers compiled by XCOM-I.  Yes, I know that makes your head swim a bit.  Well, wait until you get to the next section.

If you're already an IBM System/360 expert or even someone who wants to be an IBM System/360 user, then perhaps an IBM System/360 emulator such as Hercules might work well for you. As for myself, having tried to figure out without success how to use Hercules, I have to concede that I'm neither an expert nor want to be one, and thus I have no desire to confront that learning curve. I'd much prefer a lightweight IBM System/360 emulator, with essentially no learning curve at all. (And a lightweight emulator is far more in line with Virtual AGC's goal of integrating Space Shuttle computing support into spaceflight simulation programs than a full-blown mainframe emulator would be anyway.)

As it happens, there is an available lightweight IBM System/360 emulator. This emulator, called sim360, was written by the same Daniel Weaver who I've also mentioned earlier as the author of an earlier XPL-to-C translator for standard XPL. You can find the source code for sim360 as slightly modified for my purposes here in the Virtual AGC software repository, but the official place to get the most up-to-date version is Dan's own site.  If you follow the hyperlink to Dan's site, you'll find that it's disguised as a Pascal compiler, which it is.  But don't be confused: There's an IBM System/360 emulator in there!

Aside:  There is a limit, of course, to how "lightweight" an IBM 360 emulator can be while still remaining useful.  To run programs originally written in XPL or XPL/I, for example, just emulating the CPU's instruction set is not enough.  Such XPL or XPL/I programs depend on there being an operating system in place to allow operations like inputting data (from a files or keyboards) or outputting data (to a displays, printers, or files) to occur.  But an XPL program doesn't actually interact directly with the operating system.  Instead, an XPL program expects that there's a separate program called the submonitor running along side of it, and the XPL program makes its low-level requests for input, output, allocation of memory, and so on, to this submonitor program.  In fact, there will be a whole section later on that describes XPL's built-in functions, some of which rely on the submonitor, as well as another section covering the MONITOR procedure, all of whose functionality comes from the submonitor.  But the point is that to be useful to us, a lightweight emulator like sim360 must provide enough submonitor capability to respond correctly when the XPL program makes these requests of the submonitor, even though the IBM 360 operating system is entirely absent.  Fortunately, sim360 does so.  Thanks, Dan!

By the way, if in place of sim360 you were to use Hercules, an emulation of the XPL or XPL/I submonitor would not be built into it.  Rather, you would have to assemble the IBM System/360 Basic Assembly Language (BAL) source code for the submonitor, and presumably contrive the JCL needed to run the submonitor program along-side your compiled XPL program on the emulator.  Fortunately, although we don't need it if we are using sim360, we do have source code for the submonitor: 

• For A Compiler Generator's XCOM, it's XMON.bal.
• For SUNY's XCOM 4.5, which we've not discussed yet but which we'll get to in the next section, it's the slightly-different XPLSM.bal.
• For Intermetrics's unavailable XCOM, it's the collection of BAL source-code files known as MONITOR.ASM. 
  

What we don't have in this scenario is a BAL assembler, nor an IBM System/360 linker program, so in spite of this wealth of source code for submonitors, in the end we still have no way to build the submonitor programs that I'm aware of.

Let's build sim360.  Do this:

make CC=cl sim360.exe

sim360 -o0ET stdout -i0AT stdin Tests/HELLO.obj

Hello, world!

• -o0ET stdout: Causes any output to "device 0" from the IBM 360 program to be routed to stdout, and automatically translate it from EBCDIC to ASCII.
  
• -i0AT stdin: Causes input from "device 0" into the IBM 360 program to be routed from stdin, and automatically to translate it from ASCII to EBCDIC.

Insane Bootstrapping!

Aside:  To be clear, XCOM-I is not self-compiling.  It's written in Python, not XPL.  Rather, XCOM3 and XCOM45 are what I expect to be self-compiling once they're bootstrapped.  In the bootstrapping scenario, XCOM-I is that other compiler used to compile the initial version of XCOMx before an XCOMx executable exists.  Except that unlike the normal bootstrapping scenario, XCOM-I, XCOM3, and XCOM45 are full-blown compilers rather than being the initial weak-and-feeble versions thereof.

1. We'll use XCOM-I to compile the XPL source code for XCOMx. That will give us an executable which I'll call XCOMx-native that runs natively on our Linux, Windows, or Mac computer.
   
2. We'll then use XCOMx-native to compile the XPL source code for XCOMx again. That will give us an executable which I'll call XCOMx-360 that's native to the IBM 360. I.e., it only runs on an IBM 360 or a simulation thereof, producing IBM 360 object code as well.
   
3. Finally, we'll run XCOMx-360 under sim360 to compile the source code for XCOMx one last time. That will give us an executable which I'll call XCOMx-360A.  If everything worked perfectly, XCOMx-360A should be byte-for-byte identical to XCOMx-360.
   
4. Finally finally, we'll run the legacy XCOMx-YYYYMMDD executable under sim360 to compile source code for XCOMx one really-truly last time.  That will give us an executable which I'll call XCOMx-360C.  If everything worked perfectly, XCOMx-360C should be byte-for-byte identical to XCOMx-360 and XCOMx-360A.
   

make CC=cl

Bootstrapping XCOM3 -----------------------------------------
Files XCOM3-360.obj and XCOM3-360C.obj match within tolerance (2 <= 4 mismatches)
Cross-comparison to output of legacy compiler successful
Files XCOM3-360.obj and XCOM3-360A.obj match within tolerance (1 <= 4 mismatches)
Bootstrap of XCOM3 successful
Bootstrapping XCOM45 -----------------------------------------
Files XCOM45-360.obj and XCOM45-360C.obj match within tolerance (1 <= 4 mismatches)
Cross-comparison to output of legacy compiler successful
Files XCOM45-360.obj and XCOM45-360A.obj match within tolerance (1 <= 4 mismatches)
Bootstrap of XCOM45 successful

 XPL COMPILATION---SUNY STONYBROOK---XCOM4.5 VERSION OF  JULY 19, 1976.  CLOCK TIME = 21:5:40.16.

TODAY IS AUGUST 12, 2024.  CLOCK TIME = 15:6:40.38.

   1 I   /*  FIRST WE INITIALIZE THE GLOBAL CONSTANTS THAT DEPEND UPON THE INPUT      I  1286
   2 I      GRAMMAR.  THE FOLLOWING CARDS ARE PUNCHED BY THE SYNTAX PRE-PROCESSOR  */ I  1286
   3 I                                                                                I  1286
   4 I   /*  XPL PARSING TABLES  */                                                   I  1286
   5 I                                                                                I  1286
   6 I   DECLARE MAXTL LITERALLY '12' ;                                               I  1286
   7 I   DECLARE MAXNTL LITERALLY '26' ;                                              I  1286
   8 I   DECLARE STARTSTATE LITERALLY '112' ;                                         I  1286
   9 I   DECLARE NT LITERALLY '42' ;                                                  I  1286
  10 I   DECLARE NSY LITERALLY '92' ;                                                 I  1286
  11 I   DECLARE NSTATES LITERALLY '228' ;                                            I  1286
  12 I   DECLARE V(92) CHARACTER INITIAL ( '< DUMMY >', '<', '(', '+', '|',           I  1286
⋮

TOTAL TIME IN COMPILER   0:0:1.51.
SET UP TIME              0:0:0.03.
ACTUAL COMPILATION TIME  0:0:1.33.
POST-COMPILATION TIME    0:0:0.15.
COMPILATION RATE: 198451 CARDS PER MINUTE.

Aside:  By the way, XCOM3 has plenty of quirks specific to it, quite aside from any quirks XPL more-generally may have as a computer language.  XCOM3-native and XCOM3-360 inherit these quirks.  I mention this just in case you become excited about using XCOM3-native and start writing new XPL programs for it!  Here are a few quirks I've noticed:

• It only accepts upper-case for identifiers and keywords ... in spite of the fact that XPL is case-insensitive and that every speck of XPL source code in A Compiler Generator is printed in lower case!
  
• It requires an EOF token at the end of the source code.
  
• It won't allow you to have two division operations (including MOD as a division) in the same statement, instead emitting an error message saying that it requires a "busy register".

How to Build HAL/S-FC

DEBUG ¢0¢3¢4¢5¢6¢8¢9¢C¢D¢E¢F H(202)
 
 HELLO: PROGRAM;
    DECLARE I INTEGER;
    DECLARE MY_NAME CHARACTER(20) INITIAL('RON BURKEY');
    DECLARE INTEGER, J;
    REPLACE PRINTER BY "6";
    WRITE(PRINTER) 'THE BEGINNING';
    DO FOR I = 1 TO 5;
       WRITE(PRINTER) I, 'HELLO, WORLD!';
       DO FOR J = 2 TO 8 BY 2;
          WRITE(PRINTER) J, MY_NAME, 'SAYS ISN''T THIS FUN?';
       END;
    END;
    WRITE(6) 'THE END';
 CLOSE HELLO;

Aside:  By the way, XEXTRA is something you can use on a make command line to pass extra options to XCOM-I, while EXTRA can be used to pass extra options to the C compiler.  In particular, the --quiet in XEXTRA is a way to eliminate certain informative messages from XCOM-I that (after seeing them a hundred times) I personally no longer find quite so informative and would rather not see any longer.

⋮
Processing regression/HELLO ==========================================
Comparing ...
Files cA.rpt and pyA.rpt are identical
Files FILE1.bin and halmat.bin are identical
Files LISTING2.txt and listing2.txt are identical
Files pass1-new.rpt and pass1-old.rpt are identical
Files halmat.bin and regression/halmat.bin are identical
Files litfile.bin and regression/litfile.bin are identical
Files listing2.txt and regression/listing2.txt are identical
Files COMMON-PASS1-new.out and COMMON-PASS1-old.out are identical
Files optmato.bin and regression/optmato.bin are identical
Files auxmata.bin and regression/auxmata.bin are identical
Files pass2-new.rpt and pass2-old.rpt are identical
Files cards2.bin and regression/cards2.bin match within tolerance (17 <= 18 mismatches)
Files cards2.bin and regression/cards2.bin match
Files COMMON-PASS2-new.out and COMMON-PASS2-old.out are identical
Regression test passed.

1. PASS1[.exe]
2. FLO[.exe]
3. OPT[.exe]
4. AUXP[.exe]
5. PASS2[.exe]

Aside:  This isn't actually a full build of HAL/S-FC, but merely of the compiler passes needed to run the regression test.  You could have gotten the additional passes PASS3 and PASS4, along with the BFS versions of the passes (discussed later),  namely PASS1B[.exe], OPTB[.exe], PASS2B[.exe], and PASS3B[.exe], by using the make targets "all regression" rather than just "regression" in the instructions above.

The Peculiarities of Standard XPL and Intermetrics XPL/I

It is a truth universally acknowledged that there is no satisfactory introductory information available concerning programming in XPL.  Or at least, it would be universally acknowledged if anybody had ever heard of XPL and wanted to use it for anything. 

The best you can do, generally speaking, is to purchase an out-of-print, used copy of the book A Compiler Generator.  If you do, you'll find a book that's densely packed with information, but that that information is the source code for an XPL compiler (written in XPL), lots of BNF descriptions of the language, lots of theory on how to write a compiler-generator program, and very little of direct interest to a programmer who wants to come up to speed quickly on how to write or understand a program written in XPL. Not to mention the fact — though I am mentioning it! — that some of the most-critical counter-intuitive information is buried in easy-to-miss, easy-to-misunderstand comments made in passing, rather than as big, bold-face warnings.   As a bonus, the book provides an index of almost no use at all to a newby XPL programmer.  Beyond the book itself, most online information about XPL, in my experience, is simple condensing or other rehashing of A Compiler Generator, and adds little extra of value in a tutorial sense, since it's almost never written by anybody actually working with XPL.  With that said, you may find some useful online information in a couple of places:

• Daniel Weaver provides some documentation (in particular, his xpl.pdf) with his XPL-to-C translator program.
• The University of Toronto XPL website has a page with a small but useful transcription of some material from A Compiler Generator — albeit with unfortunate HTML formatting that make some of it almost unreadable.

And as for Intermetrics XPL/I ... well, from a tutorial perspective, it's orders of magnitude worse. From the standpoint of surviving documentation, XPL/I may as well never have existed at all.

Taking all of that into account, I wish I could provide a full tutorial here how to write XPL or XPL/I programs, similar in nature to Michael Ryer's book Programming in HAL/S.  But there's so little to do, and so much time.  No, scratch that, the other way around!  There's just not enough time, and maybe not enough skill.  So for now, I'll just cover some of the quirks of the language(s) that might be unexpected to programmers already skilled in other programming languages. Send in suggestions for improvement, if you like; I'm sure I can use them somehow to make the discussion even worse.

Character Set and Modern Character Substitutions

The most basic characteristic of a language is the character set in which the language is expressed.  Above all else, we need to recognize that IBM 360 computers encoded textual information using so-called EBCDIC coding, which defined somewhat-different characters than the ASCII coding common today, as well as entirely different numerical codes for the characters it did include.

Neither A Compiler Generator nor Intermetrics specifies the character set.  I've given it a lot of thought, and my conclusion is that the originally-supported character set was:

<space>
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9
_ % + - * . / | & = < > # @ $ , ; : ( ) ' " ! ? [ ] { }
¢ ¬ <eof>

Here, by <space> I mean a space character, and by <eof> I mean an end-of-file character.

You'll notice that several characters common today were not originally supported, including:

` ~ ^ \

Contrariwise, there are three characters (¢, ¬, and <eof>) that don't exist at all in the 7-bit ASCII character set that's the common core for the character sets typically used today when writing software source code.  The first two are EBCDIC characters, while the third isn't even EBCDIC, but merely something (a Hollerith code, I guess) which could be encoded on punchcards.  HAL/S-FC somewhat arbitrarily assigned the <eof> character the numerical code 0xFE, which I believe was otherwise unused in the then-available version of EBCDIC.

Therefore, when working with XCOM-I, we can use the fortunately otherwise-unused ASCII characters in place of the three unsupported characters:

• The ASCII characters ~ and ^ are used interchangeably with ¬, but XCOM-I prefers (and transparently replaces them by) ~. 
  
• Similarly, we use the ASCII character ` in preference to ¢. 
• Finally, the ASCII \ character is now used in place of <eof>, which was a character that could be punched on the punch cards, but had no printable representation.
  

Aside:  The <eof> character — not to be confused with the EOF token that appears at the end of the source code in an XPL or XPL/I program — is an unusual case.  It is not a special character in XPL programs.  However, it is used as a bookkeeping device by HAL/S compilers, or at least by HAL/S-FC, in padding that the HAL/S compiler automatically appends to a stream of input HAL/S source code.  Thus we have to have a way of representing it in the XCOM-I framework, even though it's not a character anyone would ever have occasion to use in either an XPL or HAL/S source-code file, unless that file was itself part of a HAL/S compiler!  To the best of my knowledge, <eof> appears only within a single string variable, INPUT_PAD, that occurs in the HAL/S-FC source-code file STREAM.xpl.  See the comments for INPUT_PAD in STREAM.xpl.

These substitutions allow us to completely translate the original XPL or XPL/I source code back-and-forth between the original EBCDIC and ACII without any loss of information, and without insisting that you adopt any specific "locale" like ISO-8859-15 or UTF-8 for your computer that's not optimal for your other (non-XPL'ing) activities.  With that said, I suspect that in most of today's common computer setups, you can probably use ¢ and ¬ in your XPL/I source code if you insist on doing so, but I neither recommend nor guarantee it.

Aside: All previously-existing XPL or XPL/I source code files I've found, or HAL/S source code for that matter, has already substituted ~ or ^ for ¬ anyway. And indeed, some early HAL/S documentation suggests the former substitution. I suspect that's because some IBM printers at the time printed ~ in place of ¬. But whatever the explanation, the substitutions I'm suggesting are not exactly daring in their originality. As far as the ¢ symbol is concerned, it's not actually used in XPL or XPL/I code, as far as I know, but it is used in HAL/S, so we still need to have a reliable way of dealing with it.

Case Sensitivity

XPL programs are not case-sensitive, except in so far as the contents of quoted strings are concerned.  E.g., lower-case or mixed-case symbols are treated as being identical to their upper-case correspondents, but quoted strings are case-sensitive.

Source-Code Formatting

Input to an XPL or XPL/I program (via the INPUT built-in function) is expected to conform to computer punch-card-like conventions. I.e., input lines are always exactly 80 characters long, and XCOM-I enforces this by truncating or right-padding input lines as necessary. If the input lines are longer than 80 columns physically — say, because they have punch-card sequence numbers in columns 81-88 — the extra columns are stripped off.

As for the source-code proper, other than being confined to columns 1-80, it is entirely free-form. I.e., line breaks are ignored; several statements may exist on the same input card, or conversely, a single statement may be split across multiple cards. Even though XPL CHARACTER strings are limited to 256 characters, there is seemingly no limit to the length of an XPL statement.

There are hints in the error messages of XCOM3 (and I believe, in A Compiler Generator text) that some contemporary computer systems may have treated column 1 specially, perhaps accepting some kind of non-blank control characters there. If so, it was a issue outside XCOMx proper and is irrelevant to XCOM-I. However, you do find that legacy XPL source code does often begin in column 2 rather than column 1, and I suspect that this hypothetical column-sensitivity is the reason for it.

Regarding this column-1 ambiguity, however, there is the practical question of what to do when a long quoted string is split across multiple cards. (Yes, that does happen.)  Does column 1 belong to the string or not? In XCOM-I, I take my cue from XCOM3 source code in this matter: In spite of the fact that XCOMx's source code generally avoids column 1 in all other cases, column 1 does belong to any multi-line quoted strings.

Identifiers

Identifiers cannot begin with a digit, but otherwise consist of any sequence of upper- or lower-case alphanumeric characters, or any of the characters @ _ # $.  For example, @_26$8ab# is a legal name for a variable.

Identifiers cannot exceed 256 characters in length.

The Basic XPL Datatypes

There are only three basic datatypes:

• FIXED is a 32-bit signed integer.  (Stored as 2's-complement, in big-endian byte order, vs the little-endian byte order used in most personal computers today.)
  
• CHARACTER is a variable-width character string, with a string-length limited to 256 or less.  Strings are stored as a 32-bit unsigned integer known as a descriptor, paired with a separate area from 1 to 256 bytes containing the individual characters of the string, encoded in EBCDIC.  The descriptor has 8 bits specifying the string length (minus 1) and 24-bits providing the starting memory-address of the character data.  The string descriptor 0x00000000 is a special case, and indicates an empty string.
  
• BIT(n), where n is from 1 to 2048, is an n-bit object.  The amount of storage varies by the precision:
• BIT(1) through B(8) are stored in memory as single bytes.
• BIT(9) through BIT(16) are stored as 2-byte "half-words".
• BIT(17) through BIT(32) are stored as 32-bit words.
• BIT(33) through BIT(2048) are stored similarly to CHARACTER variables: There's a 32-bit "descriptor", of which 8 bits is the number of bytes needed to store all of the bits, minus 1, and 24 bits area pointer to elsewhere in memory, where the bytes themselves are stored. Thus, a long BIT(n) like this uses up 4 bytes for the descriptor, plus ⌊(n+7)/8⌋ bytes (5 for n=33 through 256 for n=2048) for the data.

DECLARE COMPILING BIT(1);
...
IF (COMPILING&"80")^=0 THEN
    ...

Or in other words, first you declare COMPILING to have a single bit, and then later you check it to see what the value of its 8th bit is!  This particular bit of hilarity caused me months worth of trouble.

The storage formats in memory duplicate those that would have been expected on an IBM System/360 computer.  While the storage formats are not significant in abstract terms, they'll be seen to be quite significant in dealing with certain aspects of HAL/S-FC's source code, such as its so-called "virtual memory" system, and indeed I think it would be impossible to run HAL/S-FC unless these underlying IBM 360 storage formats were used.

Aside: It's easy to become confused and to imagine (incorrectly!) that you can treat a CHARACTER variable (as opposed to an array of CHARACTER variables) as an array itself, in order to access its individual characters.  You cannot!  In fact, the XPL language does not provide any syntactical means to access individual characters of a string.  For that, you must rely on built-in functions provided by the runtime library to directly manipulate memory.  The most-direct method is to use the BYTE function, which can either retrieve the EBCDIC numerical encoding of an individual character in a CHARACTER variable, or else to store a new EBCDIC numerical value at a given position in a CHARACTER variable.  Thus if we had a CHARACTER variable C which held the value 'HELLO!', then BYTE(C, 3) would return 211 (the EBCDIC encoding for the letter 'L'), while the assignment statement "BYTE(C, 3) = 198;" would change the contents of C to 'HELFO' since 198 is the EBCDIC code for the letter 'F'.  That sounds cumbersome, since very few of us have memorized the EBCDIC table, but it's really not.  You generally don't have to look up the EBCDIC encoding for anything, because you would actually have programmed operations such as this as "BYTE(C, 3) = BYTE('F');".  Another, less-generally-useful method would be to use the built-in SUBSTR function to retrieve a specific character position as a new CHARACTER object of length 1.

• BIT(1) through BIT(32): The rightmost bit corresponds to the least-significant bit in the byte at the highest address.
• BIT(32) through BIT(2048): The leftmost bit corresponds to the most-significant bit in the byte at the lowest address.

Aside:  Since I say that the bit-packing is undocumented, what's my justification for claiming that what I said just above is true?  The short answer is trial-and-error!  The longer answer is that one of the intermediate milestones in trying to get XCOM-I to the point of being able to compile the original source code for HAL/S-FC was first to be able to compile XCOM3 and run it with a verifiably correct result.  (Recall "insane bootstrapping" from earlier.)  But I couldn't get it to work!  After messing with it for days on end, I eventually got the answer in a dream, and then experimented with a couple of different bit-packing schemes before finally getting XCOM3 to run properly.  The packing scheme I describe above is the one that worked.

Aside: If all that wasn't bad enough, there's also a trap waiting for you if you're already used to doing bit manipulations with logical operators and shifts in other computer languages.  This trap is in the behavior of conditional tests in XPL's IF and DO WHILE statements.  Conditional tests in these statements depend only on the least significant bit; i.e., it as if any conditional test involves an extra "& 1" operation that you can't see.  Thus if you wanted to detect (say) that bit 3 of the BIT(5) variable A was set, a statement like "IF A & 8 THEN ...;" wouldn't help you at all, since the implicit "& 1" in the conditional would cause the test always to fail!  You would instead need to use a shift-right operation, such as "IF SHR(A, 3) THEN ...;".

declare x fixed, y fixed, z fixed;
x(0) = 1;
x(1) = 2;
x(2) = 3;
output = x(0) || ' ' || x(1) || ' ' || x(2);
output = x || ' ' || y || ' ' || z;

DECLARE F FIXED, C CHARACTER, B BIT(5);
DECLARE FS(10) FIXED, CS(10) CHARACTER, BS(10) BIT(5);

Aside: Standard XPL, as in A Compiler Generator, doesn't allow expressions when expressing array sizes, whereas XPL/I does allow them.  For example, the following is fine in XPL/I but is a no-no in XPL:

DECLARE BUFFER(3600-1) BIT(8);

This example will be continued in the next section, where it will make a little more sense.

DECLARE F FIXED INITIAL(22), F2 FIXED INITIAL("22"), F3 FIXED INITIAL("(8) 22");
DECLARE C CHARACTER INITIAL('Hello!');
DECLARE B BIT(5) INITIAL("(1) 10100");
DECLARE FS(10) FIXED INITIAL(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11);

• CHARACTER literals — i.e., text strings — are enclosed in single-quote (') characters. If a single-quote itself must appear within the string, you use two single-quotes in succession. For example: 'I am the ''king'' of the world!'.
• Integer literals for FIXED or BIT(N) have one of several forms:
• A sequence of decimal digits is interpreted as a non-negative number in base-10 in the usual way. Note that a leading minus sign (-) or plus sign (+) is not part of a numeric literal! In XPL, minus signs are only operators, and thus something like -10 is not a literal for the number -10, but is instead the unary-minus operator followed by the literal for the number 10. In most cases this is a distinction without significance, because XCOM-I (or the original XCOMx) automatically tries to perform all computations that are possible at compile time. Nevertheless, this distinction does cause some arithmetically-satisfactory expressions to be syntactically illegal in XPL. For example, the expression 5 + -5 isn't legal in XPL, and would need to be 5 + (-5) instead.  The plus sign is also only an operator, but unlike the minus sign (which can be either a binary or a unary operator), the plus sign is only a binary operator.  So even an expression like "(+5)" would be illegal.
  
• A sequence of hexadecimal digits enclosed in double-quote (") characters represents a hexadecimal number. Spaces are ignored within literals like this, and hence can be added at will for improved human readability.
• If a double-quoted string is preceded (within the quotes) by a parenthesized decimal number, then that number indicates the number of bits represented by each digit. XCOM-I supports only the following cases:
• "(1) ..." (with digits 0-1 and spaces) is a binary number.
• "(2) ..." (with digits 0-3 and spaces) is a base-4 number.
• "(3) ..." (with digits 0-7 and spaces) is an octal number.

Automatic Datatype Conversions

To a limited extent, XPL will convert one datatype to another when appropriate.  Table 6.8.2 in A Compiler Generator supposedly covers all conversions, if you can understand what it's telling you.  Here's the way I think it works:

• BIT(1) through BIT(15) are automatically up-converted to FIXED when the context calls for it.  After conversion, all of the converted values will be positive integers.
• BIT(16) is automatically up-converted to FIXED, with sign-extension of the most-significant bit.  I.e., the converted value may be either positive or negative.
• BIT(17) through BIT(31) is automatically up-converted to FIXED, but A Compiler Generator is not clear on the handling of the sign.  XCOM-I assumes, highly dubiously, that the most-significant bit is extended as if it were a sign.  In practice, BIT(17) through BIT(31) are seldom if ever used.
• The storage format of BIT(32) is identical to FIXED, so no conversion is needed.
• FIXED is automatically down-converted to BIT(1) through BIT(31) by truncating the most-significant 0, 16, or 24 bits.
  
• FIXED (or BIT up-converted to FIXED) can be automatically converted to a CHARACTER representation of a decimal number.  For example, 12345 → '12345'.  (Note that there is no automated conversion in the reverse direction.)

The final conversion just mentioned, conversion of integer datatypes to decimal-string representations, is quite problematic, because it is entirely ambiguous as to what the appropriate conversion may be.  The usual usages are something like the following:

DECLARE I INTEGER, C CHARACTER;
⋮
C = I;
OUTPUT = 'THE VALUE IS' || I;

I.e., assigning an integer value to a string variable, or concatenating an integer value with a string value.  Autoconversion seems logical and sensible at first glance, particularly in string concatenation.

Looking deeper, however, recall that CHARACTER values are passed around in the form of so-called string descriptors, which are themselves indistinguishable from FIXED.  Thus in C=I, should the integer I be converted to a string and assigned to C, or should it be treated as a string descriptor and the string it points to be assigned to C instead?  There's literally no way for the compiler to know for sure what was intended by the programmer.  XCOM-I makes an attempt to guess which is appropriate, and I assume so did the original XPL/I compiler.  But the rules to be used are undocumented, and the two compilers sometimes disagree.

In cases of disagreement between the intention of the programmer and the guess of the compiler, there's a built-in function (STRING) that can be used to communicate the programmer's intention to the compiler.  As near as I can tell (with A Compiler Generator, perhaps no longer shockingly to us, being unclear about it), STRING(I) would tell the compiler to treat I as a string descriptor, while STRING(C) might tell it to treat the string descriptor of C as if it were an integer.  Thus judicious use of STRING can be used to override the compiler's guesses as to where automatic conversions are needed.  Unfortunately, at present, there are some (though fortunately very few) cases in which XCOM-I would require the use of STRING while the original XPL/I compiler did not, or vice-versa.  In these cases, the only recourse at present is to insert and extra STRING (or to remove one) from XPL source code, which in my world-view is a very undesirable thing to do in legacy XPL source code such as that of HAL/S-FC.

XPL/I ARRAY and BASED Data ... and an Exception

Aside: For the original XPL/I compiler, there was a distinction in the way ARRAY variables were stored in memory vs DECLARE array variables.  The distinction was that a pointer was stored in the variable and the data for the array was stored elsewhere, much as a string descriptor is used for character values.  That was apparently a trick get around limitations on IBM 360 memory-segment sizes.  At the present time, I don't see this distinction as being operationally significant, so XCOM-I treats the two keywords identically.  This is subject to change, if I discover my thinking was in error.  As, unfortunately, I often do.

BASED FB FIXED;
BASED RB RECORD:
    F FIXED,
    C CHARACTER,
    A(10) BIT(5),
END;

"Dope Vectors"

When I said that a BASED is a "pointer to an array", I was glossing over the fact that to be useful a BASED must track a lot more information about the BASED than just its data's location in memory.  In fact, a BASED is stored as a 28-byte structure plus the separately-positioned data for the array.  The 28-byte structure is referred to as a "dope vector".   In other words, if you had a BASED called (say) MYBASED and you executed the built-in function ADDR(MYBASED), it would return the address of the dope vector for MYBASED.  As usual, the HAL/S-FC documentation and source code do not actually provide any useful facts about this setup, but various factoids can be inferred from HAL/S-FC source code, to a greater or lesser degree of confidence, and here are my feeble inferences about the fields of dope vectors.  The fields with green highlighting are those of conceivable interest to an application programmer, though all of them are managed entirely transparently in most cases.

• Offset 0:  FIXED pointer giving the address of the actual data.
• Offset 4:  BIT(16) giving the size in bytes of each record.
• Offset 6:  BIT(16) gives the number of CHARACTER or BIT(≥32) fields in each BASED RECORD, or 1 if it's a BASED CHARACTER or BASED BIT(≥32), or 0 otherwise. The point is that it's the number of "string descriptors" associated with each element of the BASED array. This information is used by COMPACTIFY. (See below.) While I don't fully understand the calculations being performed, I'd venture the opinion that for COMPACTIFY to use this information efficiently, it's necessary for the XPL/I compiler to rearrange the fields of BASED RECORD from their declared order in such a way that all of string-descriptor fields come first in the record. XCOM-I does indeed perform this rearrangement.
  
• Offset 8:  FIXED giving the total number of array entries for which space has been allocated.
• Offset 12:  FIXED giving the total number of array entries actually used so far.
• Offset 16:  FIXED.  The dope vectors for all BASED variables for which memory has been allocated form a linked list.  The global variable FIRSTRECORD gives the address of the first dope vector on the list, and this field at offset 16 in the dope vector points to the next dope vector, or 0 if there is no next one (or if space hasn't been allocated).  The ordering is TBD, since I haven't traced through the code in all of its gory detail, but I believe they are ordered according to decreasing address fields (i.e., the field at offset 0), which at least initially is also the order in which the RECORD_CONST or ALLOCATE_SPACE macros (see below) were executed to allocate memory for them.
  
• Offset 20:  FIXED. It appears to me that this field supplies some properties of the BASED in the form of bit fields. It is laid out as follows:
• Bit 24 indicates the BASED is "constant", which appears to mean that you cannot incrementally grow it. (The macro NEXT_ELEMENT(based) is used to increase the size of the specified based by 1 record, an operation which fails if the based is "constant".)
• Bit 25 indicates the BASED is "unmoveable". If a based is "unmovable", it means that an operation like NEXT_ELEMENT(based) (see above) will succeed only if there is enough free space immediately following the allocated memory that can be "stolen". Whereas if it's not unmovable, then the based may migrate in its entirety to a newly-allocated block elsewhere and the space it originally occupied may thus be freed.
• Offset 24:  BIT(16) of purpose TBD. It is referred to as "global factor".
• Offset 26:  BIT(16) is referred to as "group factor". As far as I can see, all uses of this are commented out in HAL/S-FC, so perhaps it ended up being unused.
  

The XCOM-I implementation mimics this dope-vector structure, though only the fields I've highlighted in green are significant in XCOM-I ... which is fortunate, since they're the only ones I imagine I understand somewhat.

With that discussion in mind, in understanding some of the things that need to happen with BASED variables in actual XPL/I source code (and in particular, in HAL/S-FC), let's consider various space-management macros and/or procedures used:

• RECORD_ALLOC(based), used in expressions, returns the number of records allocated in BASED variable based.
• RECORD_USED(based), normally used on the left-hand-side in assignments, sets the number of records used so far in based. Its most-common usage is RECORD_USED(based)=RECORD_ALLOC(based), but it can also be used with something other than that on the right-hand side to truncate the array or to skip past the lowest indexes. And it can be used in expressions or conditionals, though that happens relatively seldom.
• RECORD_TOP(based), as you might expect, simply returns RECORD_USED(based)-1.  I.e., the highest allowed index into the array.
  
• ALLOCATE_SPACE(based, top) allocates enough space for based to insure that it contains at least top+1 records in total. It will fail if we already have RECORD_ALLOC(based)>0.
• NEXT_ELEMENT(based) increments RECORD_USED(based) by 1, stealing the space from adjacent free memory or else reallocating and moving the entire array if necessary to do so. This can only be used if space for based had been allocated by ALLOCATE_SPACE, and will instead cause an abend if based had been allocated by RECORD_CONSTANT (see below).
• RECORD_FREE(based) frees the data for based, returning the allocated space to the free pool.
• RECORD_SEAL(based), RECORD_UNSEAL(based): Enables or disables the "constant" attribute of the based.
• RECORD_CONSTANT(based, top, moveable) Like ALLOCATE_SPACE(based, top), but additionally enables the "constant" property, and optionally enables the "unmovable" property.
• RECORD_WIDTH(based) returns the record width of the based.
• RECORD_LINK() prepares the data for transferring COMMON memory to the next program loaded.

BASED MYVARIABLE FIXED;
...
RECORD_CONSTANT(MYVARIABLE, 25, MOVEABLE); /* OR UNMOVEABLE */
RECORD_USED(MYVARIABLE) = RECORD_ALLOC(MYVARIABLE);

BASED MYBASED FIXED;
...
ALLOCATE_SPACE(MYBASED, 30);
RECORD_USED(MYBASED) = 25;

NEXT_RECORD(MYBASED);

BASED MYNEWBASED RECORD:
    F FIXED,
    C CHARACTER,
    A(9) FIXED,
END;
...
RECORD_CONSTANT(MYNEWBASED, 30, MOVEABLE);
RECORD_USED(MYNEWBASED) = 25;

MYNEWBASED(6).F = 12;
MYNEWBASED(10).C = 'XPL is where it is at!';
MYNEWBASED(20).A(6) = 15;
X = MYNEWBASED(6).F;

There is one — count 'em, one! — exception I've found to the somewhat-documented behavior I've described above.  This undocumented use is seen in the IND_STACK variable found in PASS2 of HAL/S-FC.  IND_STACK is declared using the following bastardized mash-up of a DECLARE statement and a BASED statement:

   DECLARE IND_STACK(STACK_SIZE) RECORD:
         I_CONST        FIXED,
         I_INX_CON      FIXED, 
         I_STRUCT_CON   FIXED,
         ...
         I_LIVREMT      BIT(8),
         I_NAMEVAR      BIT(8),
         I_STRUCT_WALK  BIT(8),
         I_AIADONE      BIT(8),
      END;

Unlike a BASED declaration, in which specifying an array size at compile time is illegal, or as IR-182-1 states, "ignored if present", the array size is indeed found in this declaration.  Nor is there any runtime adjustment to the size, as would be expected with a BASED.  Nonetheless, IND_STACK is subsequently accessed by the dotted, structure-type notation used only by BASED variables.

Rather than implement an entirely new but undocumented class of structure objects to handle this single instance, XCOM-I implements this case as if it were a BASED declaration followed by an appropriate RECORD_CONSTANT operation.  IND_STACK is one of the very few objects stored in region 7 of the XPL memory model, and by far the largest of them.

LITERALLY and Macros

DECLARE ARRAYTOP LITERALLY '255';
DECLARE MYARRAY(ARRAYTOP) FIXED;

DECLARE MYARRAY(255) FIXED;

DECLARE RECSIZE LITERALLY '3600';
DECLARE BUFFER(RECSIZE-1) BIT(8);

DECLARE DEVICE LITERALLY '6', OUT LITERALLY 'OUTPUT(DEVICE)';
OUT = 'My message';

OUTPUT(6) = 'My message';

DECLARE MYBLOCK LITERALLY 'DO; X=1; Y=X+3; END';
...
IF X=7 THEN;
    MYBLOCK;

IF X=7 THEN;
    DO; X=1; Y=X+3; END;

DECLARE MYMAC(2) LITERALLY '%1% = %2%';

Warning:  As with macros in any other computer language, this can quickly get out of hand.  XCOM-I, for example, won't detect recursive, endlessly-expanding macros.  There's also no guarantee when multiple macros are in play that XCOM-I will necessarily expand macros in the same order that XCOMx would have.  Neither A Compiler Generator nor Intermetrics documentation makes any mention of what that ordering should be.

Warning:  The scope of macro definitions is also different in XPL vs XPL/I.  In XPL, macro definitions don't respect any nested scopes they appear in; i.e., any macro definition will simply remain in effect until the end of the source code.  In XPL/I, macro definitions remain in effect only until the end of the procedure in which they're defined, including embedded procedures.  In neither case does a macro definition have any effect on source code prior to it.  XCOM-I uses the XPL/I scoping convention, even when the command-line switch --xpl is used, and thus is not compatible with standard XPL in this respect.  I've so far found no instance in legacy XPL code in which this caused any problem.

Perplexing Multiple Assignments

XPL allows multiple variables to be assigned the same value in a single assignment statement, with a syntax like:

X1, X2, ..., XN = Y;

According to A Compiler Generator (p. 137), these assignments are performed in right-to-left order.  In other words, it should be equivalent to:

XN = Y;
.
.
.
X1 = Y;

In most cases, the ordering of these assignments is of no significance, and in fact I've found no legacy standard XPL programs in which the ordering matters.  However, there are instances in XPL/I code, specifically in PASS1 of HAL/S-FC in which the order matters a lot.  That's when you have assignments of the form:

Y(I), I = J;

because obviously

I = J;
Y(I) = J;

is a lot different than

Y(I) = J;
I = J;

Unfortunately, in the cases I've encountered, the XPL/I code seems to rely on the latter interpretation.  I.e., it seems to believe that the assignments are performed in left-to-right order, in direct contradiction to A Compiler Generator.  What's going on here?

Of course, we have no way of knowing what Intermetrics XCOM did with this, but as far as A Compiler Generator's XCOM3, it appears (thanks to Dan Weaver for this explanation) that while the assignments are indeed performed in a right-to-left manner as documented, the peculiarities of the IBM 360 object-code generation in XCOM3, shove the index I into a CPU register and reuse it without change throughout both assignments.  So seemingly by accident, the net result is that the assignments appear to have been done in a left-to-right order.  I would also note that there is an example in A Compiler Generator (Fig. 6.18.1) that shows assembly code generated for the XPL expression

I, J, K = 2;

which supports the documented right-to-left order, namely,

LA	1,2(0,0)	[This gets a 2]
STC	1,1346(0,11)	[This sticks it into K]
STH	1,1344(0,11)	[This sticks it into J]
ST	1,1340(0,11)	[This sticks it into I]

Vis-à-vis Dan's explanation, it doesn't appear to me to be a reliable design choice in XCOMx:  what if there are more than 2 assignments on the left-hand side, and if they don't all use the same index I?  What would the order appear to be then?  Well, perhaps XCOMx does something else in that situation.

Aside:  When I wrote this, perhaps I didn't have a working XCOM3 yet, or perhaps it did not occur to me that I could just use XCOM3 and compile any test cases I like, so that I could see what the assembly code did.  Well, I realize it now.  I still haven't done it, but perhaps I should.  Some day.

To sum it all up:

Important Note:  XCOM-I ignores the statement in A Compiler Generator that assignments are performed in a right-to-left manner.  Rather, it performs them in a left-to-right manner, and any indices of arrays are computed at the moment the assignment is performed.  I'm keeping my fingers crossed that this doesn't explode in my face.  If it does, I would be tempted to just fix the XPL source code that was doing it, rather than to fix XCOM-I.  We'll see.

Logical Expressions

XPL's logical operators are &, |, and ~ (¬, ^), for "and", "or", and "not", respectively. The documentation in A Compiler Generator is maddeningly unclear as to what these operators do. True, table 6.8.1 calls them "logical and", "logical or", and "logical complement", but the word "logical" isn't defined ... just as my sloppy usage of the word "logical" at the beginning of this paragraph makes no distinction. Which leaves open a few loopholes that have to be closed up. The issues which we must understand are:

• Are these operations bitwise or narrowly "logical". I.e., do they operate in parallel on each bit position in a numerical value, or do they simply produce bipolar results of 1 (true) or 0 (false)?
• Do these operations short circuit? E.g., if we have an expression of the form (say) expression1&expression2, and expression1 evaluates to 0, is expression2 even evaluated?

In case you're not in the mood for a technical discussion of the matter, I'll give you the short answer up front, and having read that, you can either bypass or proceed to the long discussion according to your desires:

• The logical operations are indeed bitwise.
• The preponderance of evidence is that XCOMx did not short circuit. XCOM-I does not at present attempt any short-circuiting.
  

As for how I came up with this "information", there are several places we can look for guidance in guessing the answers. For one thing, according to A Compiler Generator's account, the XPL language was derived from the PL/I language, so we can look at PL/I documentation and hope that it applies to XPL. Of course, we can look at the source code for A Compiler Generator's XPL compiler (XCOM3) as listed in the book, and see if there are any hints there. Or we can even examine the IBM 360 object code that XCOMx generates for these operators. (All the while wondering how things came to this, that we have to resort to lame measures like consulting object code to figure out the basic features of the language?)

As far as PL/I is concerned, IBM's PL/I Language Reference (2017) tells us on p. 66 that for the &, |, and ¬ operators, "bit operations are performed on a bit-by-bit basis". As far as object code produced by XCOM is concerned, A Compiler Generator (p. 150) shows an example in which object code for the expression "SHL(K,1) & SHR(I,J)" is produced, and we do find that it simply uses the IBM 360 NR ("And Logical") instruction:

Although I had to consult more than one IBM assembly-language manual to find the answer to the seemingly-simple question of what NR does, IBM's z/Architecture Principles of Operation (p. A-8) does tell us that the NR (and its cousins N, NC, and NI) are indeed bitwise operations.

Aside: Figure B-2, "Instructions Arranged by Mnemonic" of the latter document is very helpful in trying to decipher such listings of IBM 360 object code.

Short-circuiting is a natural consideration for strictly bipolar operands and operators, but is a bit trickier to consider once we've concluded that the logical operators operate bitwise rather than in a bipolar fashion. Certainly the object-code example from A Compiler Generator that was mentioned in the preceding paragraph shows no signs at all of short circuiting: Both of the operands of the & operator in that example are evaluated, with no attempt at checking the value of the first operation before proceeding to the second one. On the other hand, that example of object-code generation by XCOMx happens to be for an assignment statement rather than for the conditional expression of an IF, DO WHILE, or DO UNTIL. Perhaps the evaluation of a conditional expression might be very different in those contexts. One reason to believe that it might be different is that the final result of a conditional expression is masked to just the least-significant bit, and thus (eventually) is indeed a bipolar value; i.e., even if all of the bits were involved in the computation, all but one of them is discarded in the end, so perhaps the extra bits are discarded at the beginning rather than at the end of the computation, even though it's more work to do so. Moreover, the PL/I Language Reference document mentioned earlier does cover short-circuit evaluation (see p. 245), and it says that short-circuiting is only in the context of the conditional of an IF statement (versus assignment statements). Plus, even then the short-circuiting occurs only in certain special circumstances, such as the leading operand being a BIT(1) literal or constant variable, which leads one to believe that the value of the leading operand has to be determined to be 0 or 1 at compile-time rather than at run-time for the short-circuiting to occur.

Unfortunately, the example of object-code generation in A Compiler Generator doesn't show us how an IF statement would compile. But as we saw earlier, we have been able to use XCOM-I to create a working copy of A Compiler Generators's XCOM3, so we can make our own example of IF, compile it with XCOM3 and see! Imagine we have the following ridiculous little XPL program:

 DECLARE I FIXED;
 DO I = 1 TO 10;
     IF (I * I) & (100 - I * I) THEN OUTPUT = 'hello';
 END;
 DO I = 1 TO 10;
     IF 0 & (100 - I * I) THEN OUTPUT = 'hello';
 END;
EOF

  20 |    IF (I * I) & (100 - I * I) THEN OUTPUT = 'hello';                           |  1314 C7 = 10.
                                                                      1314: CODE = L    1,1340(0,11)
                                                                      1318: CODE = M    0,1340(0,11)
                                                                      1322: CODE = L    2,1340(0,11)
                                                                      1326: CODE = L    3,1340(0,11)
                                                                      1330: CODE = MR   2,2
                                                                      1332: CODE = LA   2,100(0,0)
                                                                      1336: CODE = SR   2,3
                                                                      1338: CODE = NR   1,2
                                                                      1340: CODE = N    1,164(0,11)
                                                                      ...
 24 |    IF 0 & (100 - I * I) THEN OUTPUT = 'hello';                                 |  1400
                                                                      1400: CODE = L    1,1340(0,11)
                                                                      1404: CODE = M    0,1340(0,11)
                                                                      1408: CODE = LA   2,100(0,0)
                                                                      1412: CODE = SR   2,1
                                                                      1414: CODE = N    2,1300(0,1048571)
                                                                      1418: CODE = N    2,164(0,11)
                                                                      ...

Not shown above is that the symbol table tells us variable I is stored at address 1340(11), which is why all of the 1340(0,11)'s appear above. I don't understand IBM 360 assembly language too well, but what I think the code probably does is:

• The first IF:
• 1314 through 1318: Compute the left-hand operand of the & operator; i.e., I*I.
• 1322 through 1336: Compute the right-hand operand of the & operator; i.e., 100-I**.
• 1338: Perform a bitwise-AND of the two operands.
• 1340: Mask off all but the least-significant bit.
  
• The second IF:
• 1400-1412: Compute the right-hand operand of the & operator; i.e., I*I.
• 1414: Perform a bitwise-AND with 0. Admittedly, I'm not quite sure why it would be 0 it's AND'ing with, but it's certainly AND'ing with something.
• 1418: Mask off all but the least-significant bit.

But whether or not my interpretation is 100% correct, at least in this example there's no evidence of short-circuiting. The 2nd IF in particular is pretty shocking. Perhaps there's supposed to be some subsequent optimization I'm not aware of that would have cleaned it up.

COMMON Memory

• Using the keyword COMMON in place of the keyword DECLARE.
• Using the keyword-pair COMMON ARRAY in place of the keyword ARRAY.
• Using the keyword-pair COMMON BASED in place of the keyword BASED.

Another distinction is that CHARACTER variables cannot declared in COMMON, though CHARACTER variables can appear as fields in COMMON BASED RECORD variables.

Each cooperating application running in succession needs to declare COMMON in exactly the same way, using exactly the same ordering of variables and the same datatypes, at the same addresses.  When "dope vectors" of BASED variables were discussed earlier, it was mentioned that BASED variables could be set as "constant" or "unmoveable", and this necessity for remaining at the same location when successor programs are executed is part of the reason for this feature.

Aside:  If BASED variables were always manipulated by an XPL/I program as intended by the original compiler design — i.e., allowing SPACELIB to manipulate them via their dope vectors — there would be no reason for these restrictions on moveability.  However, the "virtual memory" system employed by HAL/S-FC, as implemented by the files HALINCL/VMEMx.xpl, bypasses the dope-vector system.  Specifically, "pages" of memory managed by the virtual-memory system are tracked only by an array of addresses (rather than dope vectors).  But SPACELIB manages dope vectors, and has no cognizance of arrays of addresses whose interpretation exists only in the mind of a programmer.  It is therefore necessary to make sure that the virtual-memory system's pages of memory never move.  One might ask the programmer why they didn't use arrays of dope vectors rather than arrays of addresses?  But that might be interpreted as being rude.  On the other hand, I've known enough engineers to be pretty sure they'd think that since it worked for them, it was good enough.

Aside:  Speaking of the relationship between BASED variables and COMMON memory, it's important to note the following points:

• For BASED variables appearing in COMMON, space is allocated for them via RECORD_CONSTANT or ALLOCATE_SPACE only in the first of the chained programs that needs to use them.  Subsequent programs in the chain use them as-is, without allocation.
• Memory for BASED variables not appearing in COMMON must be explicitly freed via RECORD_FREE prior to exiting whatever program of the chain allocated their memory.  If this is not done, then the memory-management system (Intermetrics SPACELIB) will abort the program with an error and fail to prepare the COMMON data for use by the next program in the chain.
  

Of course, XCOM-I makes no effort at all to pass COMMON data from one application to another using actual memory. Rather, each XPL/I application program compiled by XCOM-I can optionally (depending on its command-line options) load a file containing data into its COMMON areas of memory; and similarly, it automatically writes out its COMMON areas of memory into a file upon termination. By using the --commoni and --commono command-line switches of the application, a close degree of control can be exercised over which previously-saved COMMON blocks, if any, are passed to which application programs.  The command-line options are needed because by default, an executable produced by XCOM-I does not read in a COMMON file at startup, and outputs a file literally called "COMMON.out" upon termination.  The command-line options override either or both of those defaults.

COMMON files are in a human-readable format.  See the comments for the writeEntryCommon function in the runtime-library file runtimeC.c if you have any interest in the file format.  It's actually pretty useful for debugging certain kinds of problems.

Memory Model for a Compiled XPL Program

Understanding COMPACTIFY

At runtime, changes to sizes of BASED arrays may cause those arrays to move around within memory region 6 (see the preceding section). Depending on the type of changes, this can cause "holes" of unused memory to develop in memory region 6. Similarly, operations on CHARACTER variables such as INPUT or string concatenation (||) can cause holes of unused memory to develop in memory region 5. As long as FREEPOINT is comfortably less than FREELIMIT, these holes don't cause any problem and can just be ignored. However, it may eventually become necessary to repack these memory regions to consolidate the free space and eliminate the holes.

As far as memory region 6 is concerned, that's handled transparently by the tools already discussed in the section on BASED variables earlier, and won't be discussed here. The COMPACTIFY procedure which handles this for memory region 5, unfortunately, does require some clarification, even though you typically don't need to explicitly call COMPACTIFY yourself, and can assume it will just be called automatically whenever needed.

Most importantly, while COMPACTIFY doesn't handle memory management of BASED variables, the existence of based variables does affect how COMPACTIFY operates, with the result that different versions of COMPACTIFY must be used for XPL code than for XPL/I code. But there's a quirk in XPL's implementation: While COMPACTIFY is considered a "built-in" function of the XPL runtime library, in point of fact it's not present in the XPL (or XPL/I) runtime library, and must instead be explicitly provided in the form of XPL source code.

Fortunately, that doesn't entail any effort on your part, since XCOM-I can usually figure out what to do on its own. But still, it may be helpful to understand what's going one behind the scenes, for those cases in which you need to intervene. The XCOM-I approach to the inclusion of COMPACTIFY is that prior to loading any of the XPL or XPL/I source-code files you explicitly specify, it automatically preloads a "library file", which is an XPL or XPL/I source-code file containing at least the source code for COMPACTIFY. The library file it chooses is governed by the following rules:

1. By default, it chooses SPACELIB.xpl, which is a duplicate of the Intermetrics file of the same name provided with the XPL/I source code for the HAL/S-FC program. It should be good for compiling all XPL/I source code.
2. But if the XCOM-I command-line switch --xpl is used, the default library changes to XPL.LIBRARY.xpl. This a duplicate of the library of the same name provided with the source code of XCOM3 program. It should be good for compiling all standard XPL source code.
3. But if the XCOM-I command-line switch --lib-file=FILENAME is used, then FILENAME is used in place of the default library file. This gives you the option of using some other version of COMPACTIFY, perhaps experimenting with it yourself. If used along with an --xpl switch, then --lib-file must follow --xpl on the command line.
   

DECLARE DX_SIZE LITERALLY '500', DX(DX_SIZE) BIT(16);

• While the built-ins called DESCRIPTOR and NDESCRIPT still exist, and still provide exactly the same info about memory region 3, they are no longer relevant to COMPACTIFY.
• DX is now a BASED FIXED, because it contains pointers to the string descriptors (i.e. it contains memory addresses of the string descriptors) rather than just indices into region 3.
• DX_SIZE gives the number of elements of DX, but as it's nowhere DECLARE'd in the library, I suppose it must be an undocumented XPL/I built-in.

Aside: I'd venture the guess that DX_SIZE is initially set and space for DX is initially allocate by the XPL/I compiler to handle the string descriptors which are DECLARE'd and therefore known at compile time. At runtime, SPACELIB would then take over the task of maintaining DX and DX_SIZE to handle additional string descriptors that come into existence (or leave it) when BASED variables containing them grow (or shrink).

Structure of an XPL Program vs XPL/I

An XPL program consists of any sequence of XPL statements, followed by the token EOF.  In particular:

• Declarations of variables can be intermixed with active statements such as assignments.
• Active code can exist at the global level, outside of any PROCEDUREs.
  

Note:  The original XPL compiler, XCOM3, performed a single pass.  It required that the declaration of any particular identifier as an object (such as a variable) had to precede the use of that identifier, although there were provisions for making a forward declaration for a PROCEDURE, so that the PROCEDURE could be used before it was defined.  XCOM-I relaxes this requirement.

Each of the sample programs I've encountered in standard XPL so far has been contained in a single relatively-small file. For example, ANALYZER has a little over 1500 lines of source code, while XCOM has a little over 4200 lines.

In contrast, the XPL/I source code for Intermetrics's HAL/S compiler HAL/S-FC has over 120,000 lines of source code spread across over 600 files, though any individual pass of the compiler has no more than around 35,000 lines. This necessitated different methods for managing that source-code base in XPL/I vs XPL, and some of those methods are reflected by compiler directives embedded within the source code. Insofar as HAL/S-FC and its related applications are concerned, the top-level source-code file (##DRIVER.xpl) for each application always contains all of the necessary directives for compiling the other source-code files needed, in the correct order, so in using XCOM-I to compile these applications you don't need to worry about any file other than ##DRIVER.xpl itself.

Aside: Well, the comment about ##DRIVER.xpl isn't exactly right. Any XPL or XPL/I program will expect that there's a separate "library file" containing source code for the COMPACTIFY procedure, but the XPL/I source code for the program won't explicitly include the library file. That's the compiler's responsibility.

Aside: Due to the lack of relevant Intermetrics documentation, what I'm about to describe is not only speculative on my part, but also represents certain pragmatic compromises that I don't believe literally existed in Intermetrics's XPL compiler or development procedures. But if it will work for us using XCOM-I on HAL/S-FC and if there are no other lurking XPL/I programs that we need to worry about, why complain?

Compiler Directive Type:  /?c ... XPL/I source code ... ?/

Aside: This implies that you don't just compile HAL/S-FC once to get a HAL/S compiler that works for all HAL/S programs. Rather, you compile HAL/S-FC twice, once to get a version of the HAL/S compiler that works for the primary flight software, and once to get a version of the compiler that works for the backup flight software.

Compiler Directive Type:  /%INCLUDE module %/

Compiler Directive Type:  /* ...comment... $%module */

Compiler Directive Type:  /**MERGE module procedure */

Aside: Procedure names don't match the filenames, usually, because the naming conventions for System/360 files were severely limited vs identifiers in XPL.  Thus the filenames were normalized, truncated forms of the procedure names.

PROCEDUREs, RETURNs, and Their Peculiarities

In a view from a height, an XPL program consists of PROCEDURE definitions and of code that uses those definitions.

A procedure definition looks something like this:

label:
PROCEDURE(... parameter list ...) ReturnType;
	DECLARE ... for the parameters ...;
	DECLARE ... for local variables ...;

	... code ...;
END label;

A lot of this is optional. Thus while the initial label: is required (since it's the name of the procedure), the label at the end of the definition is optional, and is really there only for readability purposes. If the procedure needs no parameters, then the parameter list, including its enclosing parentheses, is omitted. If the procedure returns no value, then ReturnType is omitted; if present, it is one of the basic non-subscripted datatypes FIXED, BIT(n), or CHARACTER. Each parameter in the parameter list must have a declaration within the body of the procedure, and while those declarations don't technically have to precede the declarations of the local variables as shown above, it was apparently customary to do so.

            IF NODE_B(3) > 0 THEN DO; 
 	       /* IN CASE OF SYMBOL XREF EXTENSION CELL, SAVE ADDR OF SYM DATA CELL */
               DECLARE SYM_DATA_CELL_ADDR FIXED; 
               SYM_DATA_CELL_ADDR= COREWORD(ADDR(NODE_F)); 
               CALL PRINT_XREF_DATA(SHR(NODE_B(3),1));
            END; 

            IF ASIP_FLAG & (SCLASS=1 | SCLASS=2 & NAME_FLAG) THEN DO;
 	       /* RESTORE NODE_F TO SYM_DATA_CELL_ADDR BEFORE GETTING AUXILIARY INFO. */
               COREWORD(ADDR(NODE_F))  = SYM_DATA_CELL_ADDR;
               IF STYPE=16 & ^NAME_FLAG THEN
                  CALL FORMAT_NAME_TERM_CELLS(I,NODE_F(-2));
               ELSE OUTPUT = X10||'INITIAL(NAME('|| 
                  FORMAT_VAR_REF_CELL(NODE_F(-2)) || '))';
            END; 

Important:  All variables local to a PROCEDURE definition retain their values after the PROCEDURE returns.  If the PROCEDURE is re-executed, those local variables retain the values they previously had in the prior invocation of the PROCEDURE.  The values of those local variables, though retained, are inaccessible to code outside of the PROCEDURE, because the namespaces of the local variables prevent it. (In C code, this would be the same thing as saying that every local variable of every function is automatically declared as static.)

Very important: Any (or even all) parameters at the trailing end of the calling list of a PROCEDURE can be omitted from when calling the PROCEDURE, and if omitted, they too retain the same values as the last time the PROCEDURE was invoked or the values previously assigned to those parameters from within the PROCEDURE itself!  (In essence, this is like saying that parameters of a PROCEDURE are not passed to the PROCEDURE, but rather that they are just aliases for some set of global variables dedicated to the PROCEDURE.)  That's so weird that we need to see an example.  Consider the following XPL PROCEDURE definition, and CALLs to it:

weirdo: 
procedure(x, y, z);
    declare (x, y, z) fixed;		/* Declare x,y,z as integers */
    output = x || ' ' || y || ' ' || z; /* Print out x,y,z */
    x = 29;
    y = y + 1;
end weirdo;

call weirdo(1, 2, 3);
call weirdo(4, 5);
call weirdo(6);
call weirdo();
call weirdo;

The five calls successively print out the following:

1 2 3
4 5 3
6 6 3
29 7 3
29 8 3

PROCEDUREs cannot be recursive, either directly or indirectly.

Taking these facts altogether, XCOM-I implements both parameters and local variables of PROCEDUREs essentially as global variables in they way they are stored:  i.e., each parameter and each local variable of each PROCEDURE has its own static address (in the global memory model), assigned at compile time and unchanging thereafter.  The compiler enforces the logical scoping of these variables by means of their namespace.  There will be more on this when the concept of "mangling" of identifiers is discussed.

Regarding the RETURN statement, A Compiler Generator explains that it is used to exit from a PROCEDURE and optionally to return a value. Furthermore, the calling code can either use that return value or else ignore it. Which makes perfect sense. But as usual, there are some documented and undocumented peculiarities to the RETURN statement as well:

• A Compiler Generator explains (p. 146) that calling code can use returned values from PROCEDUREs even if the PROCEDURE has no RETURN statement, or the RETURN statement specifies no value. In this case, we are told, the return value is simply some unpredictable value from some unspecified System/360 register. Which is rotten, of course, but so what? This is never going to happen, right? Wrong! Actual XPL code does this from time to time. XCOM-I, on the other hand, always returns a well-defined value from a PROCEDURE, whether or not there are any RETURN statements specifying a return value; the returned value in this case is 0 if FIXED, a BIT value of the appropriate width evaluating to 0, or else the empty string for a CHARACTER.
• A Compiler Generator (probably!) does not mention that RETURN statements may exist at the global level, outside of the scope of any procedure, and may return a value when they do. But they can. XCOM-I treats these as exits from the program back to the operating system, with the returned value being the program's exit code. It thus expects the return value to be a program status code.

Blocks and Loops

Compound statements in XPL are groupings of simple statements (such as assignments or if-then-else statements) enclosed within a DO ... END block:

DO ...;
    ... simple statements ...
END;

There are five different kinds of DO ... END blocks. First, there is a stand-alone simple grouping as shown above.

Then there are 3 different kinds of loops:

DO COUNTER = START TO END [BY STEP];
    ... simple statements ...
END;

DO WHILE CONDITION;
    ... simple statements ...
END;

DO UNTIL CONDITION;
    ... simple statements ...
END;

Note that DO UNTIL is new in XPL/I and is not present in standard XPL.

In these loops, COUNTER, START, END, and the optional STEP are all integers. STEP defaults to 1, but must be positive. START, END, and STEP may be expressions, but if so they are evaluated only a single time, at the start of the loop, and are not reevaluated thereafter. CONDITION, on the other hand, is an expression evaluated on each loop; it is treated as "true" if its least-significant bit is 1, or "false" if its least-significant bit is 0.  When I say they are "integers", I don't mean that they are necessarily FIXED; they could also be BIT(≤32), which evaluate to integers.

The fifth kind of DO ... END block is:

DO CASE EXPRESSION;
    STATEMENT0;
    STATEMENT1;
    STATEMENT2;
    ...
END;

The EXPRESSION must also evaluate to an integer. If 0, then STATEMENT0 is executed; if 1, then STATEMENT1 is executed; and so on. At most, a single statement is executed, and there is no "fall through" from one statement to the next. If the EXPRESSION is negative or beyond the number of available statements, A Compiler Generator tells us that "a random jump is executed". In XCOM-I, no statement is executed under those circumstances, and control passes to the next statement after the END.

ESCAPE and REPEAT

The ESCAPE and REPEAT keywords appear to be undocumented XPL/I features not present in standard XPL. Unfortunately, from the available material I can't think of any way to be sure what they do, so I can only speculate.

ESCAPE appears in two different forms:

• ESCAPE;

• ESCAPE LABEL;

It should be noted that the HAL/S language has the keyword EXIT, which also has these two forms. (See Ryer, p. 5-12.) EXIT has the following behavior in HAL/S:

• EXIT; — Exits from the innermost enclosing DO ... END block. I.e., it's essentially a GO TO to just after the closest enclosing END.
• EXIT LABEL; — Exits from an enclosing DO ... END that isn't necessarily the innermost one, but rather the one which instead has the specified LABEL attached to it. By "attached to it", I mean that they're directly adjacent, as in "LABEL: DO ...".

Until a more-plausible explanation comes along, my assumption is that ESCAPE in XPL/I has the same behavior as HAL/S's EXIT.

For example, consider the following XPL/I code:

...
MYBLOCK:
DO ...
    ...
    DO ...
        ...
        ESCAPE; /* Escape #1 */
        ...
        ESCAPE MYBLOCK; /* Escape #2 */
        ...
    END;
      /* Escape #1 comes here! */
    ...
END;
/* Escape #2 comes here! */
...

• REPEAT;

• REPEAT LABEL;

• REPEAT; — "Repeats" the smallest enclosing DO ... END. In case the smallest enclosing DO ... END is a loop — DO WHILE or DO UNTIL or DO I = X TO Y — saying that it "repeats" has a pretty clear meaning: XPL/I REPEAT is like a Python or C continue statement. On the other hand, if the immediately-enclosing DO ... END is not a loop, the expected behavior is less clear; nor are there any instances of REPEAT in non-loops in legacy XPL/I code from which we might get a clue.  XCOM-I implements REPEAT without a label simply as a jump to the beginning of the block, which means that you could form an infinite loop if there were no other code (like ESCAPE or GO TO) to exit the loop.
• REPEAT LABEL; — Breaks out of inner loops as needed, until reaching an enclosing DO ... END loop that has the attached LABEL. That's the block that it repeats.

Aside:  The way my XPL/I implementation is different from HAL/S is that in HAL/S, REPEAT (without a label) goes to the beginning of the innermost enclosing loop (DO WHILE or DO UNTIL or DO I = X TO Y) rather than the innermost enclosing DO ... END.  Which makes sense, since that's what you'd normally want.

Counter Value After Normal Loop Termination

DO I = 1 to 100;
    ...
END;

Aside:  This would match the behavior of C.  Whereas in Python, for example, the counter for an equivalent loop would have the value 100 rather than 101 after normal termination of the loop.

Order of Evaluation of Procedure Operands

In an XPL string-concatenation expression like this one,

EXPRESSION1 || EXPRESSION2

while I've found no documentation concerning it, XPL/I expects the operands to be evaluated in left-to-right order.  I.e., EXPRESSION1 is evaluated first, then EXPRESSION2 is evaluated, and then they are concatenated.  Normally this ordering doesn't matter, but it matters if as a side-effect of EXPRESSION1 some values needed to compute EXPRESSION2 are altered, or vice-versa.  I know that this is the expectation because I have found code in PASS2 of HAL/S-FC that does not work as needed if a right-to-left order of evaluation is used.

Presumably, the left-to-right ordering should be enforced on all calls to procedures, and possibly all operator expressions, rather than just this string-concatenation example.  XCOM-I already enforces left-to-right order in calls to all XPL procedures, but prior to this finding it did not enforce an ordering on parameter evaluation in calls to runtime-library functions.  Frankly, it had never occurred to me that a C compiler might evaluate such arguments in right-to-left order.  But some do, under some circumstances, and can do so because the C standard apparently does not specify the ordering.

As it stands presently:

• XCOM-I enforces left-to-right evaluation of arguments of XPL procedures.
• XCOM-I enforces left-to-right evaluation of operands for operator expressions (such as op1||op2 or op1+op2).
• XCOM-I does not presently enforce any particular order of evaluation on other runtime-library functions (such as SUBSTR or SHL), but may be changed to do so in the future if problems become apparent.
  

In A Compiler Generator, Fig. 6.18.1 depicts the assembly code generated for a sample XPL program having several library-function calls with 2 or 3 parameters.  In those examples, the code for the arguments is always in left-to-right order.

The example:  Look at this line in HAL/S-FC's PASS2.PROCS/OBJECTGE.xpl:

S2=GENERATE_OPERANDS || TAG_NAME;                                07411000

Here, GENERATE_OPERANDS is a PROCEDURE, and calling it has the side-effect of setting the TAG_NAME appropriately.  GENERATE_OPERANDS therefore must be executed before TAG_NAME is used.  (And it does no good to argue that TAG_NAME is merely string descriptor rather than a string so that it doesn't "evaluate" anything, since when GENERATE_OPERANDS modifies the string it modifies the string descriptor too.)  In XCOM-I, the string-concatenation operator is implemented as a function, xsCat, so the two string operands are the two function parameters.  I found that clang evaluated the function parameters left-to-right in a test program I created, while gcc evaluated them right-to-left.  Whether they do that consistently or not, I don't know, but if they ever do it right-to-left it's a problem!  If the ordering were not forced by XCOM-I to be correct, clang would print assembly code in PASS2 correctly, such as

00000 E8F3 0000                              LHI    R0,0()              TIME: 0.25; @0HELLO

while gcc would print it incorrectly, such as

00000 E8F3 0000                              LHI    R0,0()              @0HELLO

The printout that includes the TIME must be the correct one, because otherwise the elaborate XPL procedure that constructs the time (EXECUTION_TIME) would have no effect at all.

Program Options

As far as I've been able to determine, the standard XPL of A Compiler Generator did not support program options presented at program-start time.  The submonitor program provided with XCOM3 did have some such program options, but they were consumed only by the submonitor and were not passed to the XPL program.

In XPL/I, on the other hand, the XPL program could indeed accept a rich set of options, from a JCL object known as the "PARM field".  One of the XPL built-in functions, the MONITOR function, was provided to access submonitor services, but A Compiler Generator left it basically undefined, presumably since any functionality it provided would be highly program-specific.  In XPL/I, on the other hand, the MONITOR function is indeed defined with a rich set of features, and in particular, MONITOR(13) passes the submonitor the name of an "options processor", and the submonitor returns a set of program options (as interpreted by the selected options processor) back to the XPL/I program.

In HAL/S-FC, there are three different options processors defined, "COMPOPT" for pass 1 of the compiler, "LISTOPT" for pass 4 of the compiler, and "MONOPT" for the submonitor.  At the present time, the XCOM-I runtime library supports these three options processors, and none other.  Since the particular program options supported are specific to HAL/S-FC, they won't be covered here, but are covered in great detail on the HALSFC page.

• $B — Interlist code bytes in hexadecimal.
• $D — Print compilation statistics and symbol table at the end of compilation (initially enabled).
• $E — Interlist emitted code (assembly format) and data.
• $I — Print Impact summary, indicating variables outside the scope of any procedure which were referenced, plus procedures called.  (Default = Off.)
  
• $L — List the compiled program (listing is initially enabled).
• $M — List program without auxiliary information (speeds compilation by minimizing string storage usage).
• $N — Produce a warning message if a procedure is called with fewer actual than formal parameters.
• $Q — This toggle seems to have been available for a while and then discontinued.  It may have caused compilation to terminate after processing the library file.
  
• $R — Collect cross-reference data for each symbol (based on statement numbers) and print with symbol table. (Default = On.)
  
• $S — Dump symbol table at the end of each procedure, if any local data is declared. (Default = Off.)
  
• $T — Begin tracing execution of XCOM at this point, during compilation.
• $U — Terminate tracing of XCOM.
• $V — Expand variable cross reference to include names of procedures referencing data and names of procedures calling other procedures.  (Default = Off.)
• $X — Do not abort compilation when ceiling on count of severe errors has been exceeded.
  
• $Y — Use '|' as margin marker rather than 'I'.
  
• $Z — Allow the compiled program to execute in spite of severe errors.
• $| — Set margin.  The portion of succeeding cards starting from the column containing the | will be ignored.  Note that this can only be used to make the card width shorter than 80 columns.
  

Aside:  Again, to be clear, XCOM-I itself has no cognizance whatever of control toggles.  Control toggles, if available, are provided to and used by legacy versions of XCOMx which XCOM-I may have compiled for you!

Built-In Runtime-Library Functions

Standard XPL has a variety of so-called "built-ins", comprising runtime-library functions callable from XPL code. Some of these bullt-ins can appear on either the right-hand or left-hand side of assignments, and some have to be CALL'd like user-defined PROCEDUREs. XPL/I has roughly the same built-ins, plus-or-minus a few, mostly (but not entirely) defined to have the same functionality. The compiler recognizes these built-ins, and there is no need for them to be declared in any way prior to use. Since these built-ins were mostly written originally in IBM System/360 basic assembly language, the runtime library supplied with XCOM-I has been entirely written in C, without any reference to the original runtime-library source code.

The list below is from A Compiler Generator (p. 140-142), with some alterations due to XPL/I, and some hopefully-helpful notes from me. The parameter descriptions in the list below identify the datatypes of parameters by the following convention:

• FIXED — NEx
• CHARACTER descriptor — DEx
• Symbolic name of a variable, with or without a subscript — V

I suppose I should make it clear that the XCOM-I environment is not precisely like that envisaged in the original XPL language as confined to an IBM 360 runtime environment, and as such, built-in functions don't work exactly the same way either. What's described here is how the XCOM-I runtime library's functions corresponding to the original built-in functions work.

The MONITOR Built-In Runtime-Library Function

As mentioned before, an XPL/I program obtained various services outside what the XPL/I language proper or runtime library could provide, by instead making requests to the separate "submonitor" program. The mechanism was a call of the MONITOR procedure. For XCOM-I, on the other hand, there is no separate submonitor program, and we may as well think of MONITOR as being just another built-in runtime-library function. Well, not just any runtime-library function. A big difference is that it provides a very large number of functions, each one of which can require its own unique syntax, thus necessitating a somewhat more-flexible discussion of how to use it.

The only uniform feature among the many aspects of MONITOR usage is that each separate function it provides is identified by a number, and such a function number is passed to MONITOR as its first parameter. My explanations in the table below are mostly pulled from Chapter 13 of IR-182-1, and then altered according to my understanding (or lack thereof). Functions 24 through 32 are deduced, poorly, from the HAL/S-FC BAL source-code file for the submonitor program (which happens to be called "MONITOR").

A number of the MONITOR(...) functions work with what's called "IBM hexadecimal floating-point" format, and specifically to the 64-bit (double-precision) version of that format, as opposed to the 32-bit (single-precision) version of it. To make the discussion more concise, I'll just refer to it as "DP floating point".

To be perfectly clear, there is no floating-point datatype in XPL/I, there are no floating-point literal constants, and there is no provision whatever to make it convenient for you (the programmer!) to hard-code such constants into your XPL source code, nor to interpret any such hexadecimal constants you find within legacy source code. Rather, you must somehow obtain the hexadecimal equivalents for whatever floating-point constants you wish to use, and then hard-code those hexadecimals into your code. For your convenience — or more accurately, for mine — I've included a little utility called ibmHex.py that you can use to convert back-and-forth between human-readable floating-point numbers and DP floating point. Just run ibmHex.py --help for instructions. This little utility can either be run in a stand-alone fashion, or else you can import it as a Python module. But I digress!

To understand DP floating point, imagine 8 groups of 8 bits each:

SEEEEEEE FFFFFFFF FFFFFFFF ... FFFFFFFF

where S is the sign, E is the exponent, and F is the fraction. (SP floating point is the same, but with 3 FFFFFFFF-groups rather than 7 of them.) The exponent is a power of 16, biased by 64, and thus represents 16^-64 through 16⁶³. The fraction is an unsigned number, of which the leftmost bit represents 1/2, the next bit represents 1/4, and so on. As a special case, 0 is encoded as all zeroes.

For example, the 64-bit hexadecimal pair 0x42640000 0x00000000 parses as:

• Sign = 0 (i.e., positive)
• Exponent = 16^0x42-0x40 = 16² = 2⁸.
• Fraction = 0.0110 0100 ...

or in total, 1100100 (binary), or 100 decimal.

As in the preceding section, I want to make it clear that the descriptions given here are how the XCOM-I runtime library's MONITOR functions work, and not how the original MONITOR functions as confined to an IBM 360 runtime environment worked!

Debugging XPL Programs

It is admittedly unlikely that many people will be writing new XPL or XPL/I programs nowadays, and will instead be compiling only legacy XPL or XPL/I programs ... assuming that I haven't already compiled all of them first! (Which is a distinct possibility.) Since such legacy programs will presumably all have been debugged decades before, there's not as much need for a debugger as there is for computer languages in which there are many active developers. Nevertheless, XPL or XPL/I programs can be run under a debugger to a certain extent.

XCOM-I.py --xpl --debugging-aid SOURCEFILE.xpl
make EXTRA=-ggdb -C SOURCEFILE

• char *getXPL(char *identifier) — Returns (as a C string) the value of a single XPL variable whose name is given by the identifier. The identifier string can be any identifier expression that's syntactically correct in XPL/I, provided that subscripts consist entirely of decimal digits, possibly with a leading minus sign, or else of integer XPL variables. This includes expressions like "V", "V(5)", "B(3).V", or "B(I).V(-5)". Recall that in XPL/I, subscripts can be applied to scalar variables. If you are querying a BASED RECORD, then be sure to include the desired field, since while getXPL knows how to print an individual field of a RECORD, it does not know how to print a collection of fields such as a RECORD. In a gdb console, you could use getXPL with a command like print getXPL(...). For example, print getXPL("C1(4)").  But expressions for the subscripts are not understood.
  
• int ADDR(char *base, int bIndex, char *field, int fIndex) — Given the name of a BASED variable (or 0 if none), an index into it, the name of field (or 0 if none), and an index into it, returns the "address"; i.e., the offset into the memory array.