Wednesday, February 24, 2021

So long, Fry's

I can't say I'm surprised to hear that Fry's Electronics is closing their doors. Back in the distant pre-pandemic year of 2019, I went to a Fry's in Fremont to pick up some replacement parts since it was on the way. The store was dead. Dead, dead, dead. I counted more staff than customers and I figured that store was ripe for closure; now that everyone is ordering everything online, and particularly their tech-savvy customer base, it was inevitable that COVID-19 would kill them too. The stores were too big and had too much stuff that people only needed intermittently.

But I do have very fond memories of Fry's. I grew up in San Diego, and they took over the Incredible Universe location off Murphy Canyon Road when Tandy got rid of the brand in the 1990s. I bought a lot of classic Mac peripherals and upgrades there, many of which I still have, including hard disks, processor cards and 3D accelerators. If I needed a SCSI cable or a modular jack or some wacky board, they probably had it. I also spent a lot of time in their CD section; for the last few years they were blowing them out at $5 a pop, which was great, since I'm apparently the last person in America to actually buy discs. They also had 3D Blu-rays and were one of the few stores to keep carrying them in any quantity even after the major film studios largely stopped producing them domestically. And of course who can forget the long junk food aisle of temptation we all had to walk along to check out?

My other favourite Fry's include Roseville, CA, with the locomotive sculpture bashing through the front (probably my favourite store display). Here's a couple from there in 2018:

Also North Sacramento, CA, ended up there a lot when I was still regularly going to Sacramento on official business; Sunnyvale, mostly because of pleasant memories going to Halted Electronics down the street which also closed, of course; and Las Vegas, with the big slot machine. I also remember the Fry's in Anaheim with the space shuttle and my wife's new favourite book:
but all I have of the San Diego store was this blurry picture I took of their lock screen showing everlasting disdain for the competition,
plus Fountain Valley, San Marcos, Woodland Hills (all Southern California) and Fremont.

I don't think any of these are likely to remain retail stores. In the future post-pandemic days, this sort of boutique electronic sales will entirely be ruled by Pacific Rim dropshippers sending directly from Guangzhou, and it'll be the same stuff for less. But I'll miss the experience, though. Kids in a candy store never had it so good.

Sunday, February 21, 2021

MacLynx beta 3 -- really!

Yes, I'm back to further hijinx with the port of Lynx 2.7.1 to the classic Mac OS. I have great fondness for it because it was the first browser I ran on the first Mac I personally owned (a IIsi) and I'm delighted to be dusting it off. No, it's not just a monkeypatch like "beta 2" (in scarequotes) was: this is a real rebuild off the real original source code once I did some housecleaning and refactoring. Here it is, running in A/UX:
Or, here's some actual proof it's different. Even with Crypto Ancienne providing bolt-on TLS 1.2 capability, you couldn't view Hacker News with MacLynx previously because it was sent as text/html;charset=utf-8. Now you can (admittedly System 7 doesn't understand what UTF-8 is, but let's just handwave that away for now). There's no easy way to monkeypatch in a fix like that because it's new logic and new strings, not just changing old ones, so only a code change will fix it.
And, symmetrically, here's lobste.rs, though the reason you couldn't view it before was a bug in Cryanc that was fixed in current 1.5:
That's not the only change, either. Besides accepting UTF-8 (or at least not refusing), it also has a proper fix for beta 1's inappropriate Content-Encoding header, has a "live" scrolling area (click the top or bottom halves of the scroll area to advance or go back a page), and reduces the event loop's constant redrawing of the screen. There are no changes to the rendering core in this version.

Under the hood, although the alpha 5 source code (the only version that survives) claimed to have been built with CodeWarrior Pro 2, the projects are actually from an earlier IDE, and it isn't clear which version of GUSI or CWGUSI (GUSI being a shim library to map "conventional" Berkeley sockets and file handling onto MacTCP and MacOS) was used. In addition, even though CWGUSI 1.8.0 is included with CW Pro 2, MacLynx won't, er, link with it (it can't find stdout or _Stdout, depending on where the symbol occurs). I don't blame the GUSI authors for this; that's Metrowerks' fault because the Metrowerks Standard Library (MSL) made stdout a preprocessor define, not an actual compiler symbol, meaning the pre-built version of 1.8.0 included on the CD could not have been built with CW Pro 2 in the first place.

That said, because reproducibility with classic Mac OS homebrew is a problem (the cool kids now use Retro68 but this doesn't help much with old codebases), I'm sticking with CWGUSI 1.8.0 because it's already on the disk and for MACINTOSH software there's a veritable GARDEN worth of places to find CodeWarrior Pro 2. Rather than go to the inconvenience of rebuilding it again, this ugly hackery in the main MacLynx source file made the linker error go away and doesn't affect the SIOUX debugging window.

#undef stdout
FILE *stdout;

I also purged and cleaned up the filepaths in the project after converting it to the "new" IDE. The only other piece you'll need to build is Internet Config Programmer's Kit 1.4, which you can get from Info-Mac. Everything else is nice and relative to compiler and project so it's portable again:

There are now also separate prefix files for debug and release builds, too, so you can cut out debugging code completely from a release build. Right now this only excludes the SIOUX debug window from the release version, but I'll be looking for more targets to cut the fat in future versions.

That's not to say there weren't other significant problems with the toolchain upgrade, even after the application built successfully. As another symptom that CWGUSI 1.8.0 was mismatched, various errnos were wrong because GUSI's set originally encroached on other errnos used by the MSL and used a different set of values to compensate. That was no longer the case with CW Pro 2 and thus manifested as failure to open any network connection at all (because errno didn't correspond to any of the expected signals you would get on a non-blocking socket), and required forging new processor defines that matched up with the expected values.

With that fixed, HTML pages came up all smushed together, with no links. Gopher menus appeared fine, but anything that generated HTML from a Gopher menu (like a CSO search) also didn't work, as well as Lynx's internal interfaces for downloads and image helpers. Some flailing around in the CodeWarrior debugger traced the problem down to the binary search algorithm that finds SGML tags: it wasn't matching any, even though the tags were there and correct. The problem turned out to be Lynx's internal strcasecomp (not strcasecmp), which seemed to barf on the MSL's implementation of TO_LOWER() and was returning unexpected values. I rewrote it in an embarrassingly obvious form to the compiler and finally got sane results.

With the browser basically working again, I turned my attention to something which always used to annoy me in the day: if the pointer is over the MacLynx screen, it flickers. The issue here turned out to be how Olivier had originally merged the Mac event loop and Lynx's main loop. Lynx waits patiently for a key, but you have to keep spinning the Mac event loop to service the GUI and allow other apps to run. Thus, the Mac event loop, even if the user is doing nothing, goes through all the work to update the status line and screen as if the user had done something, and the Mac curses implementation is inefficient enough that this ends up requiring an entire screen refresh every. single. tick. of the main loop. That's why the pointer flickers, and why keystrokes lag.

To reduce the need for keystrokes, I expanded the mouse support (which right now is pretty much limited to selecting links and manipulating the pulldown menu) to make the previously unused scroll area live. You can now page back and forth in a document by clicking in the top or bottom halves, and then click on links, reducing waiting for the main loop to catch up with keying around from link to link. But the pointer still obnoxiously flickered, so I worked on several ways of reducing the screen update frequency. The Mac curses is rather dumb and I wasn't sure how effective making it do partial updates would be (an exploration for another day), but it seemed like I could short-circuit all that work if a key wasn't being pressed. This turned out not to be the case. Lynx desperately assumes you pressed something, and trying to skip whole portions of its event loop when you didn't would maddeningly cause it to drop or delay other keystrokes. Eventually I added a simple global hack to Mac curses that just turns refreshes on or off and I tried to batch them as much as possible, which besides improving responsiveness also cut out a lot of CPU load too. It might have been possible to extricate the two loops with a little thought using the Thread Manager and spinning both loops separately, but I didn't want to introduce a new dependency (I like it being able to run on very minimal systems, and Thread Manager wasn't a standard part of Mac OS until System 7.5), and I didn't want to add too much complexity by telescoping yet another event loop inside Lynx's wait-for-a-key routine, at least not this time around.

Now, after everything that's been added and updated, there is one thing that's been removed: the PowerPC native build. That might seem strange coming from a vehement pro-Power ISA bigot like me (typing this post on a Raptor Talos II POWER9 workstation, even), but the reason is simply because I don't want to deal with multiple versions of CodeWarrior on my MDD G4, so I'm building this on my Quadra 800 instead and only a 68K target is available. Doing so, however, also allows me to test it on the same machine in System 7.1, System 7.6, MacOS 8.1 and A/UX 3.1, and keeps any additional toolchain changes I may need to make localized. I may resurrect the PPC build in the future but Classilla with styles off is more functional than MacLynx PPC, and of course this 68K build will run just fine on a Power Mac anyway.

What's next? I thought about porting a later Lynx but later releases may incorporate changes I might not want, and I have no idea if they would even build. More to the point, I haven't even been able to enumerate all the other changes Olivier already made to get it working, all of which would probably have to be adapted or dragged forward. I think improving its handling of <div> tags, especially with respect to positioning, as well as tables and HTML entities will get us a large part of the way to parity without introducing other potential incompatibilities. There are unused menu resources in MacLynx; those should be hooked up. I did initial work on a dialogue box for entering URLs instead of using Lynx's prompt, which is much faster than typing into the curses-driven prompt and lets you use cut and paste and the mouse, but that isn't in this version yet. And, because Lynx does know how many pages long a document is, it should be possible to actually have a real scroll bar which works by sending events to Lynx instead of the up/down mouse shortcuts (for that matter, we also need a mouse hotspot for backing up -- which is actually in this release, but I don't know if I'll keep it in its present location).

Finally, there's the whole topic of TLS 1.2. MacLynx can only access TLS sites by using Crypto Ancienne as a proxy (or anything else that offers an HTTP-HTTPS proxy, but I haven't seen others). It might be possible to build this into Lynx, but also keep in mind that most 68K Macs aren't fast enough for modern servers. My Quadra 800 is clock-chipped to 40MHz and is able to keep up by running the proxy self-hosted in A/UX or MachTen, but in my experience systems start running into trouble below that speed: my 36MHz Solbourne S3000, a SPARC, times out on a couple sites, while my 25MHz '030 IIci with no cache card can sometimes take up to 20 seconds to do a transaction and failed on most of the sites I tried. With those numbers it's painfully obvious a 8MHz 68000 wouldn't have a chance. For these, it makes more sense to run Crypto Ancienne on some other system and have the Mac talk to it because only the fastest 68040s are swift enough to do so themselves. However, current versions of Lynx don't allow offloading TLS to another system using an HTTPS URL to an HTTP proxy anymore, another reason to stick with 2.7.1.

Anyway, visit the MacLynx page for updated builds and the source code, and post your comments. Watch this blog for beta 4 when I feel up to it.

Sunday, February 14, 2021

Make the BeBox great again: TLS 1.2, inetd and more for PowerPC BeOS R5

Many nerds are at least historically aware of the BeOS, which died an untimely death two decades ago this year (when parent Be, Inc. sold out to Palm in 2001 and self-liquidated). Established in 1993 by former Apple exec Jean-Louis Gassée, Be's new OS was meant as a media-savvy alternative to MacOS and Windows, but with POSIX compatibility (largely), a command-line shell option and pervasive cheap multithreading, which is probably its most notable technical feature. It survives in recreated spirit on modern PCs, if not a direct descendent, as the perennially-beta Haiku.

A few nerds, however, will recall that BeOS didn't originally run on x86. In fact, its original architecture was one almost nobody remembers, the AT&T Hobbit, a strange stack-oriented CPU specialized for running C programs. The Hobbit had few takers due to its cost and various technical issues (Apple eventually rejected it for the Newton, leading to the rise of ARM), and when AT&T decided to kill the project in 1993 it nearly killed Be as well, who were using it for their dual-processor prototype wonderbox. After all, the best way to show off your all-singing, all-dancing, all-threading new operating system is with extra CPUs to power it.

Be regrouped around the PowerPC 603, which led to some unique technical issues of its own because the 603 has only three cache coherence states (MEI), making it notionally insufficient for multiprocessing. (This was carried over to the G3 as well, which is really just an evolved 603; the 604 offered a fourth state, and the G4 the full five MERSI states.) With little choice to get a product out the door, Be had to get around this problem with extra hardware to forcibly keep the processor caches synchronized. Be ended up making around 2,000 of the striking blue-and-beige PowerPC BeBoxes, deliberately targetted at technical users, over half of them in the slower dual 66MHz version and later a 133MHz version in the minority. Touches like the zooming LED load meters on the front, built-in MIDI and the customizable Geek Port made them beloved machines by their few owners: author Neil Stephenson, famous for Snow Crash, wrote the essay In The Beginning Was The Command Line with his own BeBox in mind. Pointedly, he declares in the essay that "[w]hat holds Be back in this country is that the smart people are afraid to look like suckers."

Naturally, there's a BeBox here too at Floodgap, a dual 133MHz model with 288MB of RAM running BeOS R5, the last release for PowerPC. And with a little hacking to get around its non-POSIXisms, it now has its own port of Crypto Ancienne with TLS 1.2. The screenshot is what's on the monitor (just press Print Screen anytime and a Targa file is dumped).

The Power Macintosh 7300 under the monitor isn't running BeOS, though it could (not sure if I'd need to remove its 800MHz G4 upgrade card, but it's basically compatible). Aside from PowerBooks (maybe the 3400 could be tricked into booting), PowerPC BeOS would run on pretty much any PCI beige Mac with a 603 or 604 CPU, including the clones. It even boots on systems with aftermarket G3 upgrades. It wouldn't run on an actual beige G3, however, and it wouldn't work on any New World Mac that came after.

And that's the reason why PowerPC BeOS withered after R5: Apple wouldn't provide technical documentation on future models, and Be didn't want to make the company dependent on reverse engineering them. By 1997 the BeBox, only ever a niche product for a niche OS, was discontinued. While Power Computing and other vendors still offered BeOS with their Power Mac clones, the Mac clones were themselves dying out and Be proceeded full speed ahead on an x86-compatible BeOS, releasing the dual-architecture R3 in 1998. PowerPC users became quickly neglected: BeOS never released a "try before you buy" personal edition of BeOS for the Power Macs, and unlike the situation with NeXTSTEP where fat binaries for all architectures were the rule for most software, the majority of developers simply wrote for x86 alone. There was never another browser for PowerPC BeOS other than Be's own NetPositive (while x86 had Opera and Mozilla), which is why I didn't show any BeOS browsers magically empowered by Cryanc in the screenshot, and when BeOS R5.1d0 "Dano" was leaked after Be's demise featuring the improved BeOS Networking Environment (BONE), there was no PowerPC release. At the time LowEndMac observed, "If you feel like Macs are treated like second class citizens, wait until you switch to BeOS — you might soon get the feeling of a fourth class citizen."

Nowadays I'd beg to be a fourth-class citizen. All of the old ftp.be.com archives appear to be gone, along with most of their games and freeware ports. A few packages developed by third parties survive in their original locations, and a few more in the Wayback Machine. There was a egcs port to PowerPC BeOS, but it seems to have evapourated completely, leaving BeIDE and Metrowerks C/C++ as your only development choice. I don't have many software packages but what little I do have for PowerPC BeOS I put on the Floodgap gopher server.

And no Intel crap. Twenty years later x86 has Haiku, which on 32-bit can run all your old x86 R5 apps and new ones besides, so x86 BeOS doesn't need our help. Instead, let's make the BeBox (and PowerPC BeOS generally) great again. And, hey, any of the Hobbit BeBoxes still out there too, being personally aware of a couple. (Especially if anyone wants to send me theirs.)

In future posts I want to talk about some of the other things I've been doing on this BeBox, including patching the SheepShaver Power Mac emulator (fun with page table entries) and writing a gopher client in BeIDE. But today, let's talk about porting Crypto Ancienne to BeOS, writing the only currently existing inetd-like environment for PowerPC BeOS, and why I say R5 is only mostly POSIX compliant.

Crypto Ancienne's core crypto library, ultimately derived from TLSe and libtomcrypt, is written in pre-C99. In fact, version 1.5, the current release, not only adds support for BeOS but also Tru64, IRIX 6.5 and SunOS 4, plus contributed builds for 68K NeXTSTEP, Professional MachTen, Haiku and Solaris 9 along with its previous support for Mac OS X (PowerPC and Intel), AIX, A/UX, Power MachTen, PA-RISC NeXTSTEP and of course Linux and the contemporary BSDs. While gcc 2.x is the most common compiler on these platforms, we also added support for MIPSPro on IRIX, Compaq C on Tru64 and Metrowerks C on BeOS. The core is generally the easiest portion to compile once you find the way the OS likes types and prototypes specified, and Metrowerks C had a good reputation for standards compliance, so other than adding a hack to get function-local variable allocations under 32K (!) that much was uneventful.

The tricky part turned out to be carl, the Crypto Ancienne Resource Loader, the Cryanc demo application and a desperate pun. BeOS has some unusual aspects to its POSIX support, all of which were rectified in Haiku, which built with the default code pretty much unmodified. The needed hacks boil down to the fact that, like the Windows API, standard input, standard output and standard error aren't "normal" filehandles. Let's say you want to check if there's input on an arbitrary file descriptor. There are no less than three non-interchangeable ways in BeOS:

  • If it's a socket, you can use select() like normal right-thinking people. There is no poll(), but overall this works like you think it should. This is also true for Winsock.
  • If it's a file or pipe, however, you can't. Instead, while this isn't well documented, you can make it non-blocking (something like (void)fcntl(fileno(stdin), F_SETFL, O_NONBLOCK);), and then busywait on the descriptor (return (read(fileno(stdin), &throwaway_char, 0) >= 0); will tell you if input is present). This is somewhat like PeekNamedPipe() in Win32, except that BeOS seems to lack any bespoke function for this purpose, and both require a similar combination of timeouts and alternating calls if you're waiting on a network socket and standard input.
  • But, if it's a TTY, it all goes out the window because there's an even more poorly documented ioctl you have to use instead (ioctl(fd, 'ichr', &numcharswaiting)). Haiku even preserves this ioctl for compatibility, though it is obviously discouraged. The non-blocking read() trick might also work but I ended up having to do a combination of both approaches, and even that doesn't work quite right.

For carl's loop where it transmits data from standard input and receives data from the socket, that had to be modified to check a utility function (stdin_pending()) and time-limit select() so that it could go back and forth between the two descriptors. This is ugly but it works, and the successful result is what you see in the screenshot (I grepped some lines from the HTML from lobste.rs as proof-of-doesn't-suck).

On the Crypto Ancienne web browser demo we showed that those computers could self-host their own carl in proxy mode so that they were their own "crypto proxies," assuming a suitable level of web browser support (or coercibility). NetPositive, your only choice on PPC BeOS, resolutely insists on using its own state-of-the-art 40-bit encryption over SSL; I'll see about hacking that later. Still, carl doesn't listen on sockets itself and relies on inetd or inetd-like environments such as xinetd (hi, Rob!) and Jef Poskanzer's micro_inetd, my personal favourite mini-inetd. We demonstrated running it as a proxy with micro_inetd on pretty much every other one of the OSes Cryanc supports, so it would be nice for BeOS to do the same.

Well, it won't come as a surprise to you that BeOS R5 works with none of these. Back in the day, it was even argued it might not be possible to implement inetd at all because sockets aren't shared across fork() (typically, for most inetd-like environments, they fork(), connect the socket to the standard filehandles and launch the dependent program, but this approach is unpossible in BeOS for that reason). Furthermore, you might think that net_server, the team (i.e., process) responsible for sockets in BeOS, would implement something of the sort and you would be wrong; the telnetd and ftpd in R5 are implemented differently. BONE does have a classic inetd but only because it fixes this problem as part of the other significant underlying changes in Dano, none of which were made available for PowerPC.

So this post also introduces inetb (kneeslaps and guffaws), less a port than a heavily multithreaded reimplementation of micro_inetd. Near as I can determine, this is the only inetd-like system that can run on a pre-BONE system. How can we do this if we can't pass the socket to the process we fork() to? Easy: don't pass the socket with fork()! Download it from Be-Power, or follow along with this gist:

  • We start up inetb with the port number and the dependent program. Let's use ./inetb 8765 awk '/quit/{exit}{print $1+1}' as a nice interactive example: this takes input, quits if it's quit, and otherwise tries to coerce it to a number and add one to it.
  • We listen on the port, initalize our array of iothreadstates (a struct we use to track sockets in flight), set up signal handlers for SIGCHLD and SIGPIPE, and go into an accept() loop. So far, so standard.
  • When we get a connection, we assign a new iothreadstate and then use an implementation of popen2() to fork() the dependent process but using pipes, not the socket.
  • Now for the BeOS magic. With the dependent process now running and its standard filehandles connected to pipes, we then start two threads, one to read from the process and write to the socket, and another to read from the socket and write to the process. (I have intentionally not implemented standard error: it's convenient to see it for debugging in the terminal you're running inetb from. Exercise left for the reader, but it would be a third set of pipes and a third thread.) The main thread goes back to its accept() loop to take more requests.

In the normal case, let's say that we quit this miniature awk session properly with the quit command. How do the threads react?

  • awk terminates, sending a SIGCHLD to the main thread and triggering the signal handler.
  • The signal handler reaps the process and based on the PID finds its iothreadstate. It then launches a cleanup thread for that iothreadstate, and goes back to the accept() loop to take more requests.
  • The cleanup thread now has to make the threads quit cleanly, since killing them leaves a mess in net_server (killing teams cleans up resources, but not individual threads within a team). It does this by sending a message to both the read-from-process and write-to-process threads. Any message will make them quit.
  • For the read-from-process thread, this is sufficient to interrupt its blocking read(). It sees there is a message, and exits gracefully.
  • For the write-to-process thread, this is a little more complicated. Even though the socket read should be blocking, in practice signals regularly interrupt it, so we use a select() on the socket to ensure we really do have data to read. There appears to be a bug in BeOS, however, where sending a message sometimes doesn't interrupt select(). We get around this problem by having the select() timeout every 10ms so it can look in its queue, which is less elegant, but better than a tight loop. Anyway, it too sees there is a message, and exits.
  • After waiting for both threads to exit, the cleanup thread flushes the socket and closes everything, returns the now spent iothreadstate to the pool and exits itself. Meanwhile, the main thread has already gone on to service other requests. Ain't multithreading great?

What happens if the user just disconnects?

  • As in standard POSIX, the write-to-process thread sees that the socket is ready but there is no data. Assuming a signal hadn't arrived, this is treated as a disconnect. It kills the dependent process (this is an entire team, so it's safe) and quits.
  • awk has just been killed, so a SIGCHLD goes to the main thread, triggering the signal handler.
  • The signal handler reaps the process, finds the iothreadstate, and starts the cleanup thread as it returns to the accept() loop.
  • The cleanup thread takes down the read thread as well by sending it a message, flushes the socket, closes everything and terminates. Meanwhile, the main thread has already gone on to service other requests. Another stupendous day in Cheap Thread Land!

It's BeOS' ability to effortlessly spawn huge numbers of thread even on constrained systems (even in 1996 a dual 133MHz wasn't stonking fast, and certainly not the 66MHz version) that makes this arrangement work effectively. Want to handle something asynchronously instead of busywaiting? Make a thread! The thread can block (usually)! Perfect for UI or network! Even the cleanup is asynchronous in inetb so that as little happens on the main thread as possible. The kernel handles all this messing around for you as long as you play by the rules.

BeOS isn't perfect, though, as that last sentence will attest. During my testing of inetb I unsettled net_server a lot. You can restart networking from its preference window, but it seemed bad that I had to do this as often as I did. In fact, as an unrelated note, I was able to pretty much wreck the machine every time if I accidentally started CDBurner. I don't have a burner and you'd think it would handle that circumstance, but it doesn't. The machine goes haywire if I'm lucky; it locks up if I'm not. I eventually had to remove it from the Applications menu. More generally, the notion of uids and gids is a veneer and you're pretty much doing everything as the superuser. That means wrong moves hurt.

But don't forget that early Mac OS X had its own weird problems during its earliest versions. BeOS, at least superficially, gives you that similar experience of a POSIX-alike underpinning with better multitasking and memory management, and it was definitely lighter on system resources than early OS X was, too. What NeXT had was Steve Jobs and a longer history with Apple than Jean-Louis Gassée, and while it is variously said that Be's demanded purchase price is what turned Apple away from buying them, I've always thought it was just a cover story for the real deal to get an original Apple founder back. And that worked out handsomely for Apple. But I think BeOS could have served as the next Mac OS at least as well.

Our next BeOS entry will talk about SheepShaver, which you can think of as "Classic" for BeOS. It even runs PowerPC code natively for surprisingly useable performance. But it started crashing incessantly after I upgraded the RAM in my BeBox. Can we fix that? Of course! Find out how next time!

Friday, February 5, 2021

The quest for Wolfenstein 3D music on the Apple IIgs

Many people are unaware there is a port of Wolfenstein 3D to the Apple IIgs (my GS is currently still in working order). It's actually a surprisingly good one, derived in part from the Macintosh port with a custom soundtrack, with a strange history of its own. It plays well enough on my 8MHz TransWarp GS accelerator with the viewport about half size, but you really need a ripping soundtrack to mow down Nazis, and while I could revel in their screams the game never once played a note of music.

Naturally, because coronavirus, I decided to spend a weekend rectifying this. And yes, it's the latest 1.1 version, yes, the checkbox for Music was checked in the Wolf3D preferences (Command-P), and yes, other sound effects worked fine.

I like to think I have a modestly tricked out system (hard disk, GS/OS 6.0.1, accelerator, ROM 03 with a Woz Special Edition topcase) but one thing it was a little weak on was RAM. A ROM 03 IIgs has 1.125MB on board (1MB, plus 128K); I had a fully populated 1MB Apple RAM expansion card in, so I had 2.125MB total. Officially the game doesn't play on anything less than 4MB, but it clearly ran, so I figured RAM was the immediate problem.

There are many choices for RAM upgrades for the IIgs but one with a long history (albeit differing opinions) is the GGLabs RAMGS/8. This gave me 8320K (8MB plus 128K on board), the price wasn't exorbitant and I could nab one easily on eBay. Compared to the other cards in my system, it's rather diminutive (and doesn't need the strut prop the long Apple RAM card did).

Its sole installation direction is a silkscreened arrow saying "FRONT." This is a little nerve-wracking because the slot has no keys and the card could go in it either way, and the installation appears backwards. But that's how it's supposed to fit (the controller board for the InnerSpace hard disk is behind it):

With that, I started up GS/OS, admired the nice ample addressing space in Get Info in the Finder, and ran the memory test provided with AppleWorks GS. After many minutes I got a clean bill of health.

However, I still got no music out of Wolfenstein 3D. Trashing the prefs file made no difference. So what else was I doing wrong?

The classic Mac OS divides components up into control panels and extensions primarily but GS/OS deals in control panels and tools. On a ROM 03 system, about half of these tools are built into the ROM and the rest reside on disk. I had a vanilla GS/OS install with all Tools through 034, but music support needs Tool035, which is not installed with standard GS/OS.

After some digging I found the Tool as a standalone file on an unrelated disk image, but this disk image was in the newer-fangled .2mg format, which I can't directly turn into a floppy with DiskCopy or DiskDup on my Power Mac 7300. (Yes, you can use things like ADTPro to write the disk on the IIgs itself, but this Mac has a suitable floppy drive, and it seems like I should be able to use it rather than messing around with serial cables.) CiderPress will convert these, however, and while the GUI is Windows-only you can build some pieces of it on Linux. None of them would directly do what I wanted, which is to turn it into a DiskCopy image, but the source code gets you most of the way there. Edit Convert.cpp and find the line switch(18). This is the destination format. Since I knew this was an 800K image, I changed the 18 to 14, which specifies a DiskCopy 4.2 image, and used the badly-named iconv (not to be confused with libiconv) to generate the image. On the Power Mac 7300 I changed the type and creator codes in ResEdit to dImg and dCpy respectively so that DiskDup would accept it. Bang, a 3.5" floppy with the needed tool.

Triumphantly I returned to the IIgs, inserted the disk, copied the Tool over, restarted GS/OS, started Wolfenstein 3D, and indeed heard some very impressive music out of the wavetable synthesizer. Success! Unfortunately this is where I think I pushed my luck: the 8MHz TransWarp GS just can't keep up with music and the rendering. In fact, while it tries to load the game data you can intermittently hear the music grind to a halt (it even crashed out to the monitor once, presumably because the data didn't arrive quick enough), and you have to have the viewport at postage stamp size to make it playable. I'm glad I have 8MB of RAM and it works very well in everything else, but after all that I decided to turn music back off so I could actually play the game (but I kept the Tool, in case other games require it).

So, with the RAM situation licked, I guess now I'm in the market for a faster IIgs accelerator. In today's world, a man's gotta kill Nazis with a soundtrack, you know.

Friday, January 22, 2021

Things I've learned about A/UX

I've been working more frequently with A/UX as actually a user rather than merely a collector/tourist to improve support in Crypto Ancienne (by the way, git tip has fixed the known remaining issues with A/UX and it now passes my internal test suite). Here are a couple things that don't appear in the manual or in the otherwise comprehensive A/UX FAQ.

If you're puzzled why you can't Telnet into your A/UX machine, nfs0 needs to be set to wait and net9 needs to be set to respawn in /etc/inittab, or incoming connections like Telnet and FTP don't (or, depending on what inetd you're using, connections may just sit there and inetd fails to spawn the daemon, sometimes for as long as a half an hour). This means you need to be running /etc/portmap as well as /etc/inetd; you can't run just inetd. You should probably also upgrade to jagubox inetd. You might be able to get around this by not using portmap services in /etc/servers but I haven't needed to try that.

If you are sitting at the "Welcome to A/UX" dialogue box (i.e., you aren't logged into the machine and you have autologin disabled), you have to select Special, Restart to properly unmount the file systems. Selecting Special, Shut Down bizarrely leaves them dirty (forcing a long and unnecessary fsck on the next boot), and running shutdown from a root console doesn't consistently work right either. So now I have it rigged to not autoboot from the Mac boot partition, I select Restart from A/UX when I'm done, and then when the machine comes back up in the Mac boot partition, the A/UX filesystem is clean and I just shut down the Mac partition without continuing through the boot. The downside is I have to press Cmd-B manually to start the boot when I do want to be in A/UX.

This machine runs A/UX with my custom partitioning, which I document in more detail elsewhere.

I do have to say that on my clock-chipped Quadra 800 (to 36MHz), A/UX is a real pleasure. If they had ported it to PowerPC natively I bet it could have really been something spectacular even though it was sort of a dog's breakfast under the hood (but in that respect no worse than the classic MacOS).

Thursday, December 24, 2020

Unboxing the best gift of 1983: the Commodore SX-64

Happy holidays and to those of you that celebrate it a very merry Christmas. As a holiday-appropriate entry, let's unbox ... a Commodore SX-64!

The Commodore SX-64 has the distinction of being the first portable colour computer. Originally part of an entire family of portable Commodore 64 systems, it was supposed to be the midrange model between the black-and-white SX-100 and the dual drive DX-64 as announced at Winter CES 1983, but only the SX-64 was ultimately released by May of that year. (There was even an SX-500, based on the Amiga 500, but it never got past the prototype stage.) Portable in this case is used advisedly, as it weighs about 23 pounds, but it has a 5" monitor which isn't terrible, a built-in 1541 5.25" floppy drive and a detachable keyboard all in one tank-like enclosure that can be lifted around by the handle (which doubles as a stand). Power usage is too much to run on an external power source any smaller than your typical car battery but in the age of systems like the Osbourne 1 such a machine wasn't implausible as a luggable. It did not sell particularly well but the sheer number of SX-64s that survive and see regular use today (I actually own three others) attests to their residual popularity. I acquired this SX-64 as NOS still in its original box and packaging a few months ago and decided to get it out of storage as a fun little exploration.

Getting it out, we experience our first Christmas emotion: anticipation. This box was only opened by the previous owner to check the contents, though it got flattened a bit by sitting stacked in the storage unit.

Obviously Commodore didn't lose a lot of sleep making the box pretty (not like the regular C64 or other packages); it's plain jane cardboard in the same look as their printer boxes and certain other sub-sexy accessories. The indication "PHILADELPHIA" on the label suggests this was stocked at their West Chester, PA headquarters. Although it's likely some stores did sell the SX-64 at retail, at least as a kid in Southern California I didn't see any retail sales at the usual suspects of the day, including where we bought our 64 and 128 and peripherals (Target, Sears, Toys R Us, etc.). We will verify the serial number shortly.

And now the second Christmas emotion — delight — as we see the SX-64 inside. On top is (in protective foam wrapping) the manual, and (in protective plastic) the "saddlebag" accessory that is very precious to SX-64 owners that is often separated from used units. I have put the manual aside. Let's get the saddlebag out of the plastic.

The "commodore" and logo are silk-screened onto the fake fuzzy exterior, which on my other units is worn or faded, but not so here on this absolutely pristine one. Inside the saddlebag are two boxes of the type 5.25" floppies used to come in. One contains a totally ordinary grey IEC power cord but the other contains a beautiful new DB-25 keyboard connector cable. (A straight-thru DB-25, suitably machined to fit, will do if your SX-64 doesn't have one.)

Serial number checks out. This unit has a manufacture date of September 1983 and was "made in Japan." SX-64s have the full complement of ports except for cassette; the SX-64 Kernal ROM instead defaults to device 8 (the internal disk drive), but has a bug that overruns the input buffer when using the SHIFT-RUN/STOP shortcut to load from disk. The Kernal also uses different, higher-contrast colours. You can connect an SX-64 to an external monitor, but do it when the power is off or you can kill the SID (I've actually done that once, to my great chagrin).

The other major glitch with the SX-64 was the top-loading expansion port. Its ribbon cable to the mainboard was so inadequately shielded that devices like REUs would not work reliably, forcing Commodore to issue a service pack to correct the problem.

Each side of the main unit's Styrofoam packing has "SX-64" and a Commodore logo stamped on it.

Portrait with the foam sheet protection still on the handle.

Foam protects the screen.

... well, sort of. Degrading as polyethylene foam of this age often does, the foam had also left a little "extra" but it was easily wiped off.

With the disk drive and monitor control door open. The 1541 disk drive still had the original head protector, which is a cardboard insert to prevent the disk heads from clattering themselves to death during transport. Please, if you're shipping a 5.25" drive, put a head protector in. A throwaway floppy disk will do nicely.

And finally, all plugged in and ready to rock, we now experience the last and most bittersweet Christmas emotion: disappointment. This unit does not power on. After some cursory checks of the fuse, it appears to be the power supply, which while more reliable than the obnoxious C64 potted epoxy brick is still subject to the capricious ravages of time. This unit will thus become a spare.

(The unboxing experience isn't exactly contemporary Apple, but I think I still would have been excited.)

Sunday, December 13, 2020

Stereoscopic computing: anaglyph sprites on the Commodore 64

This article is part of a series -- you can read other entries

In our first 3D article we talked about the various types of stereoscopy available on computers. Modern systems can both generate a 1080p image and/or (with most video cards) a high-refresh-rate image, and most 3D displays or 3D-capable TVs are 1080p, so depending on whether you have an active or passive display system you can either use fast refresh (like my 120Hz DLP projector, which delivers a full 60Hz to each eye with active glasses) or an interlaced image and polarization (like my Vizio 3D TV and Mitsubishi Diamondcrysta monitor). This generates a high-quality, (usually) flicker-free high definition 3D picture.

However, classic computers invariably don't have either of those options, so we must resort to less satisfactory approaches. While some systems implemented a spinning shutter wheel as an active 3D display option, many older systems lack sufficient refresh rates to be sufficiently smooth and very old machines can't update the screen fast enough between eye views anyway. (That gives me a headache just thinking about it.) For most situations the typical choice will be either anaglyph, i.e., red-cyan glasses, or exploiting the Pulfrich effect. The latter, though not general purpose due to how the effect is generated, can be sufficiently convincing with the right application and we'll look at that in a later article. A third option is possible with modern displays but it, too, will be the subject of a later post. Today we'll try to get a primitive anaglyph effect working on the Commodore 64, and we'll do it with the classic and widely available red-cyan glasses you can get on Amazon or from specialty shops like Berezin (no affiliation, just a satisfied customer).

The basic concept with anaglyph is that the coloured glasses filter certain wavelengths of light, delivering different views to each eye. Since the red filter is over your left eye, your left eye only gets red (primarily), so the image that should be delivered to the left eye is tinted red. Likewise, as the cyan filter is over your right eye, the cyan filter should optimally admit only what is part of the right eye image. In practice, as anyone who's looked at anaglyph images knows, the strategy is imperfect: most full colour images will have some bleed-through and while colour selection and processing can reduce differences in brightness between the eyes, some amount of ghosting and retinal rivalry between the two sides is inevitable. Still, anaglyph 3D images require no special hardware other than the glasses and some well-constructed images can be very compelling.

To make objects stick out, the left (red) channel is separated further and further to the left from the right (cyan) channel; to make them recede, the left channel is separated further and further to the right. When the channels overlie exactly, they are seen at a neutral distance from the viewer as appropriate to the image's composition. The mnemonic is thus the three R's: red right recedes.

Unfortunately, conventional colour anaglyphs are difficult on a system like the C64 because there is only one fixed, limited palette and the shades available may not correspond well with the lens colours. You may be able to make the displayed colours more appropriate for your glasses by messing with the display settings or colour balance, but this naturally has other side effects. Additionally, there is no alpha channel, so overlaying objects (which is necessary to deliver two views in one image) just obscures what's behind them. Usually you would use a proportional shade of purple to deliver an appropriate level to each eye but the C64 has but one shade of purple, and you would have to manually figure out when to use it.

A way around the latter problem is to either dither or (as a special case of dithering) interlace. This reduces resolution but eliminates having to do costly alpha calculations and screen updates. One way of doing a 3D anaglyph display on a C64 is alternating red/black and blue/black lines in multicolour mode, as the red (VIC-II colour 2) and blue (VIC-II colour 6) shades are the closest shades to most red-cyan glasses. This gives you effectively a 160x100 monochrome image. For greater dynamic range you could also consider red/pink/black and blue/light blue/black on alternating lines, using black as the common background colour, and some of the CPU-assisted modes like FLI and Hires FLI can do even better. But this requires substantial precomputation to generate the image and thus is only generally useful for static images. How might we do this for a dynamic display?

While computers like the Amiga have playfields for overlaid elements, the VIC-II chip in the C64 really only has one option: sprites. Sprites do effectively have an alpha channel, but it is a single bit, so there is no translucency. Thus, if we want a dynamic 3D anaglyph display on the C64, a straightforward means is to interlace a blue sprite and a red sprite, yielding a composite 3D plane that can move in the Z-axis by changing their relationship to each other. And that's what we'll do here.

The illusion of depth is enhanced not only by the shift between the left and right channels, but also the size of the object, so we will need a routine to scale a sprite. For simplicity we'll just use a solid square block, which we can generate on the fly. Sprites on the C64 are 24x21, each row three bytes in length, up to 63 bytes in size. We will write a quick little utility routine that will turn a number into a string of bits, and then copy that to the same number of alternating lines, clearing the rest of the sprite so we can grow and shrink it at will.

The assembler source for this quick interlaced sprite scaler is on Github and can be cross-assembled with xa. We'll put the sprite at 832 ($0340) in the cassette buffer, which appears as SPRITE in the text. Here's some highlights.

        jsr $aefd
        jsr $ad9e
        jsr $b7f7
As a convenience for playing around in BASIC, these calls accept a comma after the SYS statement followed by an arbitrary expression, and then convert the result to a 16-bit integer and store it in $14 and $15. We only care about the low byte, so $14 is the entirety of the value. We clear the top row of the sprite, and if the parameter is zero, just copy that to the rest of the sprite (clears it). Otherwise, let's make enough one bits in the top row of the sprite by setting carry and rotating it through:
        ; turn x into x 1-bits
lup1    sec
        ror SPRITE
        ror SPRITE+1
        ror SPRITE+2
        dex
        bne lup1
We then duplicate it on alternating rows (unless it's 1x1 or 2x2).
        sbc #0 ; ror cleared carry, so -1
        lsr
        beq clrs ; no copies

        tay
        clc
lup2    lda SPRITE
        sta SPRITE+6,x
        lda SPRITE+1
        sta SPRITE+7,x
        lda SPRITE+2
        sta SPRITE+8,x
        txa
        adc #6
        tax
        dey
        bne lup2
And then we clear the rest of the sprite. We assemble this to 49152 and load it, then set some parameters. After you load the binary into VICE from wherever you assembled it to (and remember to type NEW after loading), you can cut and paste these POKEs into VICE.

poke53281,0:poke53280,0:poke646,12:poke53287,6:poke53288,2
poke2040,13:poke2041,13:poke53271,3:poke53277,3
poke53248,150:poke53249,100:poke53250,150:poke53251,102:poke53264,0
sys49152,10:poke53269,3

You'll get this display.

We have set the sprite colours, made them double size for ease of use, and positioned them so that they are interlaced. If you put on anaglyph glasses at this point, it's just a block at neutral distance. Let's write a little BASIC code to move them around as a proof of concept. You can cut and paste this into VICE as well:

10 rem midpoint at x=10
20 forx=0to20:poke53250,160-x:poke53248,140+x:poke53251,110-x:poke53249,108-x
30 sys49152,x:for y=0to50:next:next
40 forx=20to0step-1:poke53250,160-x:poke53248,140+x:poke53251,110-x
50 poke53249,108-x:sys49152,x:for y=0to50:next:next
60 goto 20

With glasses on, you will see the block swing from receding into the distance and protruding into your view by sliding and scaling.

There are two things to note with this primitive example. The first is that the steps end up separating the red and blue components quite a bit; combined with the afterimage from the bleedthrough, we end up seeing two blocks at their furthest extent. We can solve that problem by separating the sprites at a slower rate (say, half) than the scaling rate. This limits how far the composite plane can be moved in the Z-axis, but there are usually other optical limits to this generally, so we're not losing as much as one would think.

The second thing to note is it's kind of jerky because it's not updating the sprite registers fast enough (the delay loop is just there so you can see each step of the animation; the lack of smoothness is because of the computations and POKEs necessary on each "frame"). We'll solve this problem by rewriting the whole thing in assembly language. For style points we'll add a background (a crosshatch) as a neutral plane, and flip the sprite priority bits for the composite plane as it moves forward and back so that it also has proper occlusion.

The assembler source for the "complete" demo is also on Github (and also cross-assembled with xa), but notable parts are discussed below. We will write it with a small BASIC loader so we can just LOAD and RUN it.

        .word $0801
        * = $0801

        ; 64738 sys2061

        .word $080b
        .word $fce2
        .byte $9e, $32, $30, $36, $31
        .byte $00, $00, $00
The motion routine is more or less a direct translation of the BASIC proof of concept except we will separate the sprites by only half of the scaled size for less of a "double vision" effect, so we change the constants to match. Here VALUE is still $14, which we're still using merely as a holding location even though we are no longer servicing BASIC directly, and HVALUE is a free zero space location for the temporary result of the math.
mlup    lda VALUE
        clc
        lsr
        sta HVALUE

        lda #150
        sec
        sbc HVALUE
        sta 53250
        lda #110
        sbc VALUE
        sta 53251

        lda #108
        sbc VALUE
        sta 53249
        clc
        lda HVALUE
        adc #140
        sta 53248
At the end we wait a couple jiffies so that the animation is actually visible, check for RUN/STOP, and if not pressed cycle the position in VALUE back and forth. MODE is $15, since we don't use it for anything else either here, and is initialized to zero when we start.
        ; wait two jiffies per frame
        lda #0
        sta $a2
waitt   lda $a2
        cmp #2
        bcc waitt

        ; check run/stop
        lda 203
        cmp #63
        bne cycle
        lda #0
        sta 53269
        lda #147
        jmp $ffd2

cycle   lda MODE
        bne decr

incr    inc VALUE
        lda VALUE
        cmp #21
        beq *+5
        jmp mlup
        sta MODE
        ; fall thru
        
decr    dec VALUE
        lda VALUE
        cmp #$ff
        beq *+5
        jmp mlup
        lda #0
        sta VALUE
        sta MODE
        jmp mlup
Here is an animated GIF of the result you can view with anaglyph glasses, though the result in an emulator or on a real C64 is smoother.

As you can see, the composite sprite recedes and protrudes appropriately, and depth cueing is helped by flipping the priority bits as it goes past the "zero" point (the crosshatch) where VALUE, in this coordinate system, is defined as 10.

While an anaglyph composite sprite approach clearly has drawbacks, it's still in my opinion the best means for independent motion of planar objects in the Z-axis and gives the most flexibility for "true" 3D on classic machines of this era. But the Pulfrich effect doesn't have its colour limitations and can be useful in certain specific situations, so we'll look at that in the next article.