2006-12-04: Smalltalk, the Image and Multiple CPUs

First posted on 2006-12-04:

Runar Jordahl posted about VisualWorks and multiple CPU cores, which is a subject that has interested me, so I felt I wanted to write something in response - not that I disagree with Runar at all, I am hopefully just re-enforcing what Runar has already said.

It used to be simple. A computer had a CPU that did the work supported by a host of other bits and pieces like memory, disks etc... Today computers are tending to have many CPUs. In the future they may well have swarms of CPUs each.

Smalltalk took shape in the 1970s, well before the advent of multi-core chips and most Smalltalk implementations, even today, run on a virtual machine that runs in a single operating system process using "green threads"; which means that the VM schedules the work in the image with only one Smalltalk process running at a time.

The memory space occupied by a running Smalltalk system is called an image and is much like the memory space occupied by any running program regardless of language , i.e. there is code and data in there. In Smalltalk it's all objects with some of the objects being code (byte codes). (As a bonus a Smalltalk image can be saved in a running state to be later resumed.)

Having a single image make use of many CPUs requires that the VM run more than one Smalltalk process at a time (typically as native threads) and allow the operating system to do the scheduling (sharing the CPUs among the processes). The problems here are that the running processes are operating on the same set of Objects (i.e. in the same image) so great care must be taken to ensure that the activities of one process do not interfere with or corrupt another and also that the timing of execution is out of the hands of the VM so things may not happen in a repeatable order (i.e. the system may become non-deterministic).

One way to address these issues is to avoid them. Don't run many processes in one image but rather run many images. In this way each image continues to run in a (largely) deterministic fashion using green threads. This solution can be inefficient in it's use of memory. The set of data being actively operated upon in an image is often a very small subset of the whole image so having many images means duplicating things unnecessarily, and while running two such images might be tolerable, running 10s or 100s would be crazy (I think we can expect servers to have 100s (or more) CPUs in the not too distant future).

Another approach is to use flags or semaphores to stop processes from stepping on each others toes. So, we could lock the root of an entire object model while our process is operating on it, thereby preventing any other process from making changes until our guy has finished - but this probably means processes will have to queue to get access to the model (even read access because we want to have read consistency) which means we really only have one process actively doing anything at a time. Back to square one. A variant of this is to make the locks more fine grained so that each process will only lock a part of the model allowing others to work on other parts of the model but this makes a system harder to code and debug, perhaps much harder.

How about if we could mix the above two solutions and have have many operating system processes all running Smalltalk in the same image and have a ready made mechanism to help avoid processes treading on each others toes?

Such a system already exists in GemStone which is both a Smalltalk implementation and a DBMS.

A running gemstone instance is a single Smalltalk image which is backed securely to disk (that's part of the database-ness of GemStone). Many operating system processes can be started in the context of this image and all of these share the same pool of objects via shared memory (this is the Shared Page Cache). Each of these processes may run at the same time on any number (well, up to ~2000 with the version of GemStone we use) of CPUs. Transactions prevent one process from treading on the toes of another (another bit of database-ness).

GemStone is an excellent environment for running headless applications such as web servers (e.g. the OpenSkills SkillsBase). If you need a GUI and also want to use lots of CPUs, GemStone can't help ... which brings us to exactly the same conclusion that Runar reached.

As GemStone demonstrates, transactional shared memory is a viable way to go. I too hope VisualWorks moves in this direction.