A happy ending (and future) for Plua

In my previous article on Plua, a port of Lua to classic PalmOS, I mentioned I was hopeful we'd have a happy ending where there was a clean-room implementation, or better still, that the original author would get in touch with me and we could redistribute it.

Well, I'm pleased to announce that Marcio himself did get in touch (he's got a lot of fun Palm and retro projects still underway, incidentally) and graciously consented to relicense Plua2c, the "cross-compiler" portion, under the same MIT license as Lua itself. You can now download the source code from Github, which I'll keep maintained.

As for Plua-the-Palm-app and its various components, I will still be hosting the .prcs on Plua Revisited indefinitely, and Marcio and I have been discussing how we can add some improvements such as expanded screen size support and the like. This means Plua has a future again, and I couldn't be happier. Thanks, Marcio!

RIP Sir Clive Sinclair

In the US the name Sinclair is more associated with gas stations and partisan media outlets than computers. We had only the Timex Sinclair series States-side, of which the major models were the T/S 1000, a rebadged ZX-81, and the 2068, which was an upgraded but partially incompatible ZX Spectrum. (There was also the T/S 1500, a more upmarket version of the T/S 1000 roughly analogous to calling the Cimmaron by Cadillac a more upmarket Cavalier, and two minor non-American spinoffs of the T/S 2068, the TC2068 and UK2068.) These sold quite poorly in the United States because of the dominant position of the obviously superior Commodore 64 (I'm bracing for the comments from Martin), despite at least one retailer selling T/S 1000s at firesale prices so customers could get a rebate on a Commodore purchase. (Commodore reportedly used some of them for doorstops.) In the UK and Europe, however, they were a hit because they were cheap, and they introduced a generation to computers that may not have been able to afford them otherwise. My T/S 1000's keyboard has crapped out and I'm not even sure where my 2068 is, but on the announcement of Sir Clive Sinclair's death from cancer at 81, hats off, gentlemen, and godspeed.

The Incredible KIMplement 0.2b: a KIM-1 emulator for the Commodore 64 (and the Superkim)

As I manage to eke out a little more time now for personal projects, since TenFourFox is coming to a close and the OpenPOWER JIT is developing at record pace, it's time to dust off some older items I've really neglected. This last week I started by updating the Incredible KIMplement, my MOS/Commodore KIM-1 emulator for the Commodore 64.

No, that's not a joke. The KIMplement runs real KIM-1 code using a software 6502 core I've christened "6o6" (6502-on-6502). 6o6 implements protected memory, exception handling and all legal NMOS instructions. In addition, the KIMplement not only emulates those famous six seven-segment LEDs and the hex keypad, but also is one of the few KIM-1 emulators that emulates a TTY connection (an old-school ASR-33) and a KIM-4 expander with 16K of RAM, allowing you to run "big programs" too.

The KIM-1 is one of the earliest single-board computers, at least in the sense we conceive of them today. It was introduced in 1976 as an enticement for engineers to play with the MOS Technology 6502 which was then the cheapest microprocessor on the market. MOS had just gotten its pants sued off by Motorola, who did not like the prospect of an inexpensive drop-in replacement for their 6800 CPU; the MOS 6501 was completely pin- and bus-compatible, and where the 6800 cost $179 the 6501 was priced at just $25. The "lawsuit compatible" 6502 was just as cheap, but because the pins had been rearranged, couldn't be substituted without additional design work. The KIM-1 provided a platform for engineers to become more accustomed with the 6502, featuring a 1MHz CPU, 1K of RAM, cardedge I/O, a keypad, LEDs and in-ROM support for teletypes, punch tape and cassette tape, for only $245.

MOS expected this would be a low-volume item mostly of interest to circuit designers. Instead, hobbyists bought them in large numbers because it was easily the least expensive microcomputer you could purchase at the time. With a KIM-1 at its center, you could have a full system with teletype, power supply and cassette storage for around $500. No other system came close to competing on cost. When Commodore Business Machines bought the ailing MOS in 1977, they wisely kept producing the KIM-1 until 1979 even after the introduction of the PET. Several clone systems exist, most notably including the Synertek VIM-1/SYM-1, as well as one unusual clone I'll talk about in a moment. I am the proud owner of four KIMs (including an original pre-Commodore KIM-1) and they all work.

I don't know if KIMplement's CPU core could be truly considered "virtualizing" the 6502, but it's more than just a naïve emulation. Rather than manually setting results and flags, the core looks at the guest instruction and runs the same instruction (or a safe variant) in the core context so that all the side effects, in particular changes to the status register, occur "for free." There is no way that a Commodore 64 at 1.0225MHz (or, worse, 0.978MHz for PAL) can do full-speed emulation of a KIM-1 running at 1MHz, but because there is much less code running per instruction, I think this scheme is probably near the fastest way a 6502 can run "untrusted" 6502 code. In practice it is about 35-50 times slower than native code, and this upsets programs that use tight timing or cycle counts, but it's still absolutely enough to actually "do things."

What sorts of things? Besides a couple LED-based games (originally Jim Butterfield's version of Lunar Lander, and I also added the misère game variant Black Match) and toy applications, you can run Tom Pittman's Tiny BASIC in the TTY, and with the bug fixes in 0.2b now you can successfully run FOCAL-65:

This screenshot in VICE shows the emulated 64 running a full FOCAL-65 program to manually compute square roots using Newton's method. It takes it a few seconds to do each iteration, but it all works. This is also a good demonstration of the FOCAL programming language itself including its unusual "floating point" line numbers (actually module and line), which it inherited from its more complex ancestor JOSS.

For this version of the emulator (0.2b), I finally finished some performance improvements to the CPU core that had been gestating in my mind for literally years -- the last version of the KIMplement was released in 2006! -- and also fixed a problem with the TTY emulation where typing characters could get out of sync under CPU load. It can still drop keystrokes if you overflow the Kernal keyboard buffer, but it's a lot smoother generally. I also worked around a bug in VICE where, if you try to load files from a directory on the host machine, the RENAME-the-file-to-itself test used to check for the file's presence doesn't work (a real 1541 would respond with error 63 FILE EXISTS but VICE says 0 OK).

The other bug I fixed was caused by the CPU core, but can't be fixed in it. The 6502 has a decimal flag which can be set in the status register and causes add and subtract instructions to operate in binary coded decimal (e.g., $90 - $01 normally is $8F, but in BCD mode it's $89). Famously, or perhaps infamously, the Commodore 64 Kernal IRQ doesn't turn off the decimal flag, and there is at least one SBC in the normal execution path. Because 6o6 executes instructions for their side effects, if a program had previously set the decimal flag (and this is not at all uncommon in KIM-1 code) it needs to be on for those math operations. The usual solution is to turn on the interrupt flag first with SEI to suppress IRQs while decimal mode is on, but the normal state of guest code is to have the interrupt flag clear because the KIM-1 doesn't have this problem. If an IRQ hits right that moment, the IRQ will be executed with the decimal flag on, and possible unexpected behaviour could result.

This is an extremely infrequent occurrence, but in a long-running system "infrequent" is a synonym for "inevitable." This can't be efficiently solved in the core because there is no atomic method for controlling two flags at once. A better solution is very simple: we just make a patched IRQ that clears the decimal flag explicitly, and calls the normal Kernal IRQ. I did the same for NMIs as a belt-and-suspenders approach.

The eventual goal is to open-source the KIMplement, and in particular 6o6, but I want to have another demonstration application for 6o6 as well before I do. A small multitasking general-purpose kernel sounds like an ideal way to show off how it works.

On my main KIM-1 site I also put in a few words about the microproducts Superkim. This 1979 variant of the KIM-1 was developed in Redondo Beach by Paul Lamar, who had been using KIM-1s for automotive performance testing but was unsatisfied with their expansion capacity. Unlike the hobbyist audience of the KIM-1, the Superkim is all business. The unit in my possession was clearly in a card cage; the hex keypad was never even fitted. It uses a custom board rather than a modified KIM-1 with sockets for RAM, ROM and up to four 6522 VIAs (this unit has 4K of static RAM and a Rockwell R6502P), and sports a full prototyping breadboard on-unit. But its most noticeable (and, I might add, intimidating) characteristic is the 200 gold wirewrap pins penetrating the board:

I have no idea what the sam heck this board was doing, but it clearly did something major in whatever its application was. Don't ask me to power this thing up because I'm worried I'll short something out somewhere. I don't know how many of these were ever sold and Commodore and MOS seem to have had nothing to do with its development.

Of the clones I find the Superkim the most interesting. While many people mention the Rockwell AIM-65 as a clone (my unit is above), and it is strongly based internally on the KIM-1, its substantial expansion (wide alphanumeric LED screen, full keyboard, printer) would not make it generally recognizeable as such. I think the Synertek VIM-1/SYM-1 are more characteristic, though that is one unit I don't personally possess.

In the future, and hopefully that future isn't in another 15 years, I want to add actual cassette support (right now you just dump memory to and from disk) and maybe support for one of the hi-res video boards like the Visable. It may also be worth trying to port the KIMplement to a faster 6502-based system like the Commodore Plus/4 or the Commodore 128 in 80-column mode, or maybe even the Apple IIgs, though all of these would need a solution for the sprites I currently use for the LEDs. (Okay, you Atari freaks, I know, I know.) The KIM-1 is a great little machine and surprisingly capable. The fact all of mine have survived over four decades proves they don't make them like they used to.

You can download a .d64 or .sdas of the Incredible KIMplement and its demo applications, or read more about the KIM-1.

More Tomy Tutor homebrew

Even the more obscure machines have their homebrews, and for the Tomy Tutor, I'm not just talking about third-pary tapes (though those are a thing too). In addition to 2018's Team Europe bounty (two multicarts and a reproduced "Game Adaptor" for those cartridges requiring additional addressing lines), a couple gamepads wired for the Tomy Tutor turned up on eBay. They are available in singles and doubles to replace either the Joy Stick or the Joy Controllers. Since the Tutor uses a unique pinout, these must indeed be made specifically for it; they all work fine with the Pyūta as well. (Not affiliated, just satisfied.)

The Commodore Plus/4, 3-Plus-1 and computer literacy

I'm testing lots of units in storage, including a whole mess of Commodore 264-series machines, mostly C16s (in both the domestic U.S. and Mexican Sigma variants, which are exactly the same internally), but also a couple of Plus/4s. The Plus/4 was beaten up in the United States press as the "Minus/60" because of TED's deficiencies (more colours but no sprites and worse sound) and its intentional incompatibility with the Commodore 64's large software library. Indeed, that's probably why my family upgraded to the 128 instead and completely skipped it.

Another aspect of the +4 that was mercilessly derided was the 3-Plus-1 pack-in software. Based on an integrated suite called Trilogy by Pacific Tri-Micro, it included a word processor, database and spreadsheet; the fact they were intended to be basic applications did not prevent critical displeasure. Popular Computing Weekly pointed out how small the working space was and their limited features, and InfoWorld complained that "[t]he word processor is the worst I've ever seen," but The Transactor's editor Richard Evers was particularly barbed, famously observing that "[t]he word processor is barely that, the data base [sic] defiles the name and the spreadsheet has little spread." But while this quote got wide currency, the rest of the article is actually far more complimentary, adding, "Each package is well written, taking into consideration the limitation of trying to make them all work within the confines of each other. Running two packages in tandem is possible with this system ... think of the software as an almost free bonus, and accept its limitations."

I certainly thought so. While I didn't pick up my first +4 until I was a starving medical student, it was a cheap thrift store purchase and I made good use of the spreadsheet as a household budget. The word processor was just good enough for letters (school papers I typed in Pocket Writer on the 128D, until I got a Macintosh IIsi) and I barely used the database at all, but the spreadsheet actually worked fairly well for basic figures. Best of all, it loaded instantly because everything was in ROM, and you really could switch back and forth from one to the other and exchange data without losing anything, neither of which was true of the programs I had on the 128. In some respects it was even more "integrated" than bigger integrated packages (think Lotus Symphony, etc.) on the PCs of the time. It is debatable how useful this feature would have been versus simply having the applications run one at a time for a larger workspace, but it did work.

Something else that worked was this Plus/4 that I dredged out of storage to test. While checking the contents, I found it had this letter in the box which I don't even remember noticing before:

The previous owner, in August 1986 (granted about a year and change after the +4 had flamed out), was invited to a computer literacy class, "an introduction to computers, teaching you basic data entry terminology, to help you feel confident working with computers. You will recieve [sic] a Commodore Plus 4 [sic] computer that adapts to your television for passing the class." The class was administered in Oregon.

There is no mention of any peripherals being included, and no Commodore could directly connect to a standard cassette deck for storage. Furthermore, it is likely the Plus/4 was selected solely because they got donated stock that didn't sell. Still, here was a computer that was cheap enough to just give somebody and connect up to their television. You pressed a key and almost instantaneously you got a word processor, a spreadsheet and a database that came built-in. You could type letters, do a household budget and maintain an address book. If you picked up a 1541 disk drive, which by then was selling for under $200, you could save files. If you picked up any of the cheap Commodore 1525-compatible printers on the market, you could print letters. If you cared to crack out the manual, you could learn to write your own programs.

For this person in western Oregon, this Plus/4, as idiosyncratic and artificially limited as it was, may have been their gateway to computer literacy — and at that time it very likely was all the computer they actually needed.

Plua revisited: Lua for PalmOS (and resurrecting plua2c)

I was not an early adopter of Palm PDAs, but my experience with handheld computers is actually rather longer; my first handheld was a Tandy Pocket Computer PC-4. It had 544 free bytes (1,568 with the 1K expansion pack) and you programmed it in BASIC. Other than the TI-85 in college my next step up from there was an HP 95LX, which was the first of Hewlett-Packard's well-regarded handheld DOS machines. But Palm was the rage in the mid-to-late-1990s when I was in medical school, and I picked up a brand-new colour m505 in 2001. From there I upgraded to the Zire 72 in 2005, which I kept using until the iPhone 3GS arrived because it had better apps and it could record video. Later on I got a used AlphaSmart dana, which is the closest thing to a PalmOS laptop; it had a wide screen, full keyboard, built-in word processor, a WiFi option, two SD/SDIO slots, and USB to connect it to a printer or to have the unit act as a keyboard for a PC or Mac. Just connect it to the computer, start your favourite word processor app, and the dana would "type" your entire document into the larger computer. No drivers necessary! Although I have a few other Palm units in my collection, including an original Palm 1000, a Tungsten T|X and a couple Centro phones, plus a Pre 2 and Veer from the webOS days, I still use my Zire 72 and Dana now and then for specific tasks.

The classic Palm OS (also known as Garnet in its final revisions), not to be confused with Palm's later and technologically unrelated webOS, actually feels a lot like classic MacOS. (The "Classic" mode in webOS 1.x for running Garnet apps doesn't seem like a coincidence to me, either.) Besides the common original architecture (68K), the heavy reliance on structured resources for both applications and data storage is very reminiscent of the Mac. When ARM-based Palm OS 5 devices emerged, not only was there a 68K emulator like the Power Mac's for running older software (called PACE, the Palm Application Compatibility Environment), but the normal state of the system was to be running 68K code.

I got a lot of wear out of pre-programmed Palm apps but I'm a nerd at heart, and I like to program things. The PC-4 was easy: it was BASIC, and it had 10 segmented program spaces, so I wrote simple games and tools for school classes. The 95LX ran DOS programs, and would happily run anything I wrote in Turbo Pascal 5.5 (though optimally if formatted for the smaller screen first). However, Palm development was primarily proprietary at the time, officially requiring CodeWarrior with specific Palm support and the appropriate hardware. I used Macs (at the time, a hand-me-down Power Mac 7300), so the hardware was no problem, but I was a starving student back then and CodeWarrior wasn't cheap.

So late in 2001 it was a real boon to discover a beta Palm OS 3.1 port of the Lua programming language to Palm OS, written by Marcio Migueletto de Andrade. The part I liked best (well, other than the fact it was free!) was it was fully self-hosted, with what today we would recognize as a simple IDE, such that you could develop right on the device. In those days Plua was based on Lua 4 and offered easy graphics, serial and UI support, so I used it for writing my own internal calculation apps which (thanks to an external separate runtime, the Palm's ubiquitous IR beaming, and everyone having a Palm device) everyone on the clinical team ended up using. Eventually Plua evolved into a full-fledged 1.0 release in 2003 instead of a time-limited beta.

Plua also included a small "cross-compiler" (really, a bytecode dumper) based on luac, though with additional code to link resources as well as emit a stub PRC header to call the runtime. This allowed you to develop on a desktop PC and build the PRC there, and then HotSync it over. More about that in a moment.

Plua was already pretty great by then, but what really moved it forward was support for TCP networking in Plua 1.1. Unfortunately, networking in Plua 1.1 had several significant bugs and Marcio was already working on Plua 2.0, which was based on Lua 5, so these weren't fixed. (One of my early apps that got bitten by this was Port-A-Goph, a gopher client for Palm OS. I got a mention in Wired and the code really did exist, but the socket bugs were difficult to work around. I probably have the source code around here somewhere.) Plua 2.0 also required Palm OS 3.5 and wasn't source compatible with Plua 1.1; the functions were similar, and many function calls could be rectified with text search-and-replace, but it still had some important differences plus the jump in the core language as well. For me personally it took awhile to convert over, but Plua 2.0 was a definite improvement and the bugfixes made it a very solid package.

This screenshot, of POSE (Palm OS Emulator 3.5) running under Mac OS 9 (emulating the emulator on my Raptor Talos II), shows that Plua 2 was perfectly capable of native controls and a standard application UI. With the socket issues resolved, networking was actually stupidly easy.

At the time I was an active participant in the Yahoo! Group for Plua (now gone, along with the rest of Yahoo! Groups), which was the only official place to get Plua 2. Marcio issued an analogous "cross-compiler" for Plua 2 called, analogously, plua2c, using 5.0.3's luac as the base. However, Plua was freeware but (Lua 5 is MIT-licensed) not open source, and the plua2c binaries — which, unlike Plua 1.0 and 1.1, were distributed separately — were only available for Windows and x86 Linux.

During the Plua 2.0 betas in 2006, I privately asked Marcio if I could build a PowerPC Mac OS X-compatible version of plua2c. He agreed to this with the condition that the source be kept private (I suspect, but do not know, that he had some interest in making it a commercial product or having a commercial support option). I agreed and over the next couple years ended up issuing four binary-only releases of the Mac OS X plua2c which I hosted on Floodgap. However, after Plua 2.0 left beta around 2008, although Marcio indicated he had interest in starting on a 2.1 based on Lua 5.1, I don't know if he ever actually did; the introduction of webOS in 2009 and the lack of interest in Palm OS Cobalt or further Garnet devices essentially ended classic Palm OS's market relevance in any case. I lost contact with Marcio and never received further replies from him regarding Plua or plua2c.

Still, I kept using it for various minor projects even if I didn't regularly keep a Palm in my pocket anymore. I eventually gave up on Port-A-Goph and started on a Plua 2 rewrite (the screenshot above), and turned my Zire 72 into a Plua-powered Hue light controller:

All of this development was done on my 32-bit Power Macs, using 32-bit builds, which was all that was ever supported. In the meantime, the Plua Yahoo! Group disappeared along with the rest of everything in the Yahoo! Groups genocide and along with it the only definitive source of Plua and plua2c. Although a few people have the PRCs, no one seems to have kept the cross-compiler, and other things like documentation and examples similarly evaporated.

When I recently decided to continue work using my Raptor Talos II, which is a 64-bit POWER9, I decided I would dust off the source code of plua2c still sitting in my G5 and develop on the new machine. plua2c compiled and appeared to function but ended up generating defective executables that weren't compatible with the Plua runtime (Plua2RT). They were dramatically bloated in size and caused the runtime to emit a low-level VM error.

Recall that plua2c is descended from luac, which more or less just dumps the Lua data structures in place. Lua's documentation says that "[t]he binary files created by luac are portable to all architectures with the same word size." To this end, luac 5.0.3 actually emits sizeof(int), sizeof(size_t) and sizeof(Instruction) (i.e., the typedef quantity for the size of individual bytecode instructions) into the bytecode header which should make an amphibious loader capable of selecting different bit widths, but Plua2RT doesn't swing both ways, at least not in that respect. To make the Plua VM happy, I had to force all of these to be 32 bits in size and change the emitter to only emit 4-byte int and size_t quantities.

This partially fixed the size, but it was still abnormally enlarged, suggesting 8-byte quantities were still being injected into the file somewhere else. After some detective work I found it was actually coming from plua2c's PDB header struct, so I hardcoded the correctly sized types in its typedefs, and the length matched up and the Plua VM could now execute the generated PRC. plua2c was now ported to 64-bit OpenPOWER.

You'll notice I said the length matched what my Power Macs emitted, but not the file itself. Besides a timestamp, the Lua bytecode is emitted using the native system's endianness, and the dump also has an endianness flag to indicate what that was. Interestingly, this is one situation in which the Plua VM does swing both ways: although the native endianness of the 68K Palm OS is big, and Plua was never ARM-native (which for Palm OS 5 is little, and PACE handles the endianness switch as part of thunking), it transparently converts the values just fine, just like Lua would. The docs even say, "binary files created on a 32-bit platform (such as Intel) can be read without change in another 32-bit platform (such as Sparc [sic]), even if the byte order ('endianness') is different." In fact, it has to, because Marcio's builds of plua2c were for little-endian 32-bit x86. Only my PowerPC Mac OS X builds actually emitted big-endian data, since that was the native endianness there. My POWER9 system runs Fedora in little-endian, so the endianness didn't match the Power Macs, but that was no problem for Plua.

I intend to honour my gentleman's agreement with Marcio about not disclosing the source code. Even if he's unable or unwilling to discuss changing the arrangement, a deal is a deal and I would want this to be a sign to anyone else who would share code with me for porting purposes that I keep my promises even a decade and a half after the fact. Still, I think Plua is a great way for retrocomputing enthusiasts to get back into Palm development. Yes, there are tools like OnboardC which compile on the Palm as well, and some but not all of the C cross-compiler infrastructure works on modern 64-bit systems, but Plua is a lot more straightforward for beginners and has tons of built-in functionality that would require external libraries or a lot of additional code with other development systems. The use of a separate runtime is a little obnoxious but hardly a dealbreaker for me personally.

So, in the spirit of our original arrangement to issue PowerPC Mac OS X binaries, I have compiled plua2c for modern 64-bit platforms, at least the ones I have a compiler or cross-compiler for. Besides the 32-bit PowerPC OS X version, which I still offer, and the 64-bit OpenPOWER ppc64le Linux binary I personally use, I also compiled it for Intel macOS 10.14+ with clang and 64-bit Intel Windows with a cross-compiling MinGW gcc. I'm willing to consider other platforms if I can easily set up compilation without a lot of additional work or disk space.

But these aren't much good without Plua itself, so I've additionally started hosting the Palm OS package on Floodgap with the runtime, onboard IDE and online help, along with Marcio's documentation and license terms. I also had a complete copy of the Plua 2 examples, so I've provided those, like the animated fishtank you saw on the introductory image. And, because building them from scratch needs the PILot Resource Compiler, I also made a minor 64-bit fix to its bitmap handling for modern systems too (it's GPL, so for that you get the full source code). Fortunately, pilot-link is still readily available for most modern platforms to sync your binaries over to the device.

It is my hope that one of two things will happen: Marcio will get in touch and bless an open-source release, or, with this tool, someone(tm) can work on a clean-room implementation of the runtime and maybe fix a few of the issues like memory usage and custom screen sizes. Sadly, that someone(tm) can't be me, because I've obviously seen the source code and know at least some of how it's implemented and that makes me a tainted implementor. But combined with the source for Lua 5.0.3 — and none of what I've divulged here can't be inferred from it — folks should be able to tease apart how the VM is constructed and how calls get to the OS, because except for the Palm-specific bits the VM core is still regular old Lua. Which is why, by the way, Plua was so great and is worth resurrecting. Maybe we could even get later releases of Lua 5 working. Who knows?

Even if we don't get either of those outcomes, at least now folks interested in Palm OS have another solid homebrew development option available once again. I got a lot of wear out of Plua and Marcio's hard work is why. I don't think he ever made a cent off it, but even with its minor warts it's still my favourite way to program the classic Palm OS. Now you can enjoy it too.

The binaries, documentation and examples are at Floodgap. As for the gopher client you saw? Well, that's a future post. But when I push that out, you can compile it yourself.

Unplanned Floodgap downtime

Floodgap is down due to an upstream circuit cut; all Floodgap services including web, gopher and E-mail are affected. The telco is on it, but I have no ETA for repair. If the downtime will be prolonged, I may host some services temporarily on a VPS. I apologize for the inconvenience.