Make the BeBox great again: TLS 1.2, inetd and more for PowerPC BeOS R5

Many nerds are at least historically aware of the BeOS, which died an untimely death two decades ago this year (when parent Be, Inc. sold out to Palm in 2001 and self-liquidated). Established in 1993 by former Apple exec Jean-Louis Gassée, Be's new OS was meant as a media-savvy alternative to MacOS and Windows, but with POSIX compatibility (largely), a command-line shell option and pervasive cheap multithreading, which is probably its most notable technical feature. It survives in recreated spirit on modern PCs, if not a direct descendent, as the perennially-beta Haiku.

A few nerds, however, will recall that BeOS didn't originally run on x86. In fact, its original architecture was one almost nobody remembers, the AT&T Hobbit, a strange stack-oriented CPU specialized for running C programs. The Hobbit had few takers due to its cost and various technical issues (Apple eventually rejected it for the Newton, leading to the rise of ARM), and when AT&T decided to kill the project in 1993 it nearly killed Be as well, who were using it for their dual-processor prototype wonderbox. After all, the best way to show off your all-singing, all-dancing, all-threading new operating system is with extra CPUs to power it.

Be regrouped around the PowerPC 603, which led to some unique technical issues of its own because the 603 has only three cache coherence states (MEI), making it notionally insufficient for multiprocessing. (This was carried over to the G3 as well, which is really just an evolved 603; the 604 offered a fourth state, and the G4 the full five MERSI states.) With little choice to get a product out the door, Be had to get around this problem with extra hardware to forcibly keep the processor caches synchronized. Be ended up making around 2,000 of the striking blue-and-beige PowerPC BeBoxes, deliberately targetted at technical users, over half of them in the slower dual 66MHz version and later a 133MHz version in the minority. Touches like the zooming LED load meters on the front, built-in MIDI and the customizable Geek Port made them beloved machines by their few owners: author Neil Stephenson, famous for Snow Crash, wrote the essay In The Beginning Was The Command Line with his own BeBox in mind. Pointedly, he declares in the essay that "[w]hat holds Be back in this country is that the smart people are afraid to look like suckers."

Naturally, there's a BeBox here too at Floodgap, a dual 133MHz model with 288MB of RAM running BeOS R5, the last release for PowerPC. And with a little hacking to get around its non-POSIXisms, it now has its own port of Crypto Ancienne with TLS 1.2. The screenshot is what's on the monitor (just press Print Screen anytime and a Targa file is dumped).

The Power Macintosh 7300 under the monitor isn't running BeOS, though it could (not sure if I'd need to remove its 800MHz G4 upgrade card, but it's basically compatible). Aside from PowerBooks (maybe the 3400 could be tricked into booting), PowerPC BeOS would run on pretty much any PCI beige Mac with a 603 or 604 CPU, including the clones. It even boots on systems with aftermarket G3 upgrades. It wouldn't run on an actual beige G3, however, and it wouldn't work on any New World Mac that came after.

And that's the reason why PowerPC BeOS withered after R5: Apple wouldn't provide technical documentation on future models, and Be didn't want to make the company dependent on reverse engineering them. By 1997 the BeBox, only ever a niche product for a niche OS, was discontinued. While Power Computing and other vendors still offered BeOS with their Power Mac clones, the Mac clones were themselves dying out and Be proceeded full speed ahead on an x86-compatible BeOS, releasing the dual-architecture R3 in 1998. PowerPC users became quickly neglected: BeOS never released a "try before you buy" personal edition of BeOS for the Power Macs, and unlike the situation with NeXTSTEP where fat binaries for all architectures were the rule for most software, the majority of developers simply wrote for x86 alone. There was never another browser for PowerPC BeOS other than Be's own NetPositive (while x86 had Opera and Mozilla), which is why I didn't show any BeOS browsers magically empowered by Cryanc in the screenshot, and when BeOS R5.1d0 "Dano" was leaked after Be's demise featuring the improved BeOS Networking Environment (BONE), there was no PowerPC release. At the time LowEndMac observed, "If you feel like Macs are treated like second class citizens, wait until you switch to BeOS — you might soon get the feeling of a fourth class citizen."

Nowadays I'd beg to be a fourth-class citizen. All of the old ftp.be.com archives appear to be gone, along with most of their games and freeware ports. A few packages developed by third parties survive in their original locations, and a few more in the Wayback Machine. There was a egcs port to PowerPC BeOS, but it seems to have evapourated completely, leaving BeIDE and Metrowerks C/C++ as your only development choice. I don't have many software packages but what little I do have for PowerPC BeOS I put on the Floodgap gopher server.

And no Intel crap. Twenty years later x86 has Haiku, which on 32-bit can run all your old x86 R5 apps and new ones besides, so x86 BeOS doesn't need our help. Instead, let's make the BeBox (and PowerPC BeOS generally) great again. And, hey, any of the Hobbit BeBoxes still out there too, being personally aware of a couple. (Especially if anyone wants to send me theirs.)

In future posts I want to talk about some of the other things I've been doing on this BeBox, including patching the SheepShaver Power Mac emulator (fun with page table entries) and writing a gopher client in BeIDE. But today, let's talk about porting Crypto Ancienne to BeOS, writing the only currently existing inetd-like environment for PowerPC BeOS, and why I say R5 is only mostly POSIX compliant.

Crypto Ancienne's core crypto library, ultimately derived from TLSe and libtomcrypt, is written in pre-C99. In fact, version 1.5, the current release, not only adds support for BeOS but also Tru64, IRIX 6.5 and SunOS 4, plus contributed builds for 68K NeXTSTEP, Professional MachTen, Haiku and Solaris 9 along with its previous support for Mac OS X (PowerPC and Intel), AIX, A/UX, Power MachTen, PA-RISC NeXTSTEP and of course Linux and the contemporary BSDs. While gcc 2.x is the most common compiler on these platforms, we also added support for MIPSPro on IRIX, Compaq C on Tru64 and Metrowerks C on BeOS. The core is generally the easiest portion to compile once you find the way the OS likes types and prototypes specified, and Metrowerks C had a good reputation for standards compliance, so other than adding a hack to get function-local variable allocations under 32K (!) that much was uneventful.

The tricky part turned out to be carl, the Crypto Ancienne Resource Loader, the Cryanc demo application and a desperate pun. BeOS has some unusual aspects to its POSIX support, all of which were rectified in Haiku, which built with the default code pretty much unmodified. The needed hacks boil down to the fact that, like the Windows API, standard input, standard output and standard error aren't "normal" filehandles. Let's say you want to check if there's input on an arbitrary file descriptor. There are no less than three non-interchangeable ways in BeOS:

• If it's a socket, you can use select() like normal right-thinking people. There is no poll(), but overall this works like you think it should. This is also true for Winsock.
• If it's a file or pipe, however, you can't. Instead, while this isn't well documented, you can make it non-blocking (something like (void)fcntl(fileno(stdin), F_SETFL, O_NONBLOCK);), and then busywait on the descriptor (return (read(fileno(stdin), &throwaway_char, 0) >= 0); will tell you if input is present). This is somewhat like PeekNamedPipe() in Win32, except that BeOS seems to lack any bespoke function for this purpose, and both require a similar combination of timeouts and alternating calls if you're waiting on a network socket and standard input.
• But, if it's a TTY, it all goes out the window because there's an even more poorly documented ioctl you have to use instead (ioctl(fd, 'ichr', &numcharswaiting)). Haiku even preserves this ioctl for compatibility, though it is obviously discouraged. The non-blocking read() trick might also work but I ended up having to do a combination of both approaches, and even that doesn't work quite right.

For carl's loop where it transmits data from standard input and receives data from the socket, that had to be modified to check a utility function (stdin_pending()) and time-limit select() so that it could go back and forth between the two descriptors. This is ugly but it works, and the successful result is what you see in the screenshot (I grepped some lines from the HTML from lobste.rs as proof-of-doesn't-suck).

On the Crypto Ancienne web browser demo we showed that those computers could self-host their own carl in proxy mode so that they were their own "crypto proxies," assuming a suitable level of web browser support (or coercibility). NetPositive, your only choice on PPC BeOS, resolutely insists on using its own state-of-the-art 40-bit encryption over SSL; I'll see about hacking that later. Still, carl doesn't listen on sockets itself and relies on inetd or inetd-like environments such as xinetd (hi, Rob!) and Jef Poskanzer's micro_inetd, my personal favourite mini-inetd. We demonstrated running it as a proxy with micro_inetd on pretty much every other one of the OSes Cryanc supports, so it would be nice for BeOS to do the same.

Well, it won't come as a surprise to you that BeOS R5 works with none of these. Back in the day, it was even argued it might not be possible to implement inetd at all because sockets aren't shared across fork() (typically, for most inetd-like environments, they fork(), connect the socket to the standard filehandles and launch the dependent program, but this approach is unpossible in BeOS for that reason). Furthermore, you might think that net_server, the team (i.e., process) responsible for sockets in BeOS, would implement something of the sort and you would be wrong; the telnetd and ftpd in R5 are implemented differently. BONE does have a classic inetd but only because it fixes this problem as part of the other significant underlying changes in Dano, none of which were made available for PowerPC.

So this post also introduces inetb (kneeslaps and guffaws), less a port than a heavily multithreaded reimplementation of micro_inetd. Near as I can determine, this is the only inetd-like system that can run on a pre-BONE system. How can we do this if we can't pass the socket to the process we fork() to? Easy: don't pass the socket with fork()! Download it from Be-Power, or follow along with this gist:

• We start up inetb with the port number and the dependent program. Let's use ./inetb 8765 awk '/quit/{exit}{print $1+1}' as a nice interactive example: this takes input, quits if it's quit, and otherwise tries to coerce it to a number and add one to it.
• We listen on the port, initalize our array of iothreadstates (a struct we use to track sockets in flight), set up signal handlers for SIGCHLD and SIGPIPE, and go into an accept() loop. So far, so standard.
• When we get a connection, we assign a new iothreadstate and then use an implementation of popen2() to fork() the dependent process but using pipes, not the socket.
• Now for the BeOS magic. With the dependent process now running and its standard filehandles connected to pipes, we then start two threads, one to read from the process and write to the socket, and another to read from the socket and write to the process. (I have intentionally not implemented standard error: it's convenient to see it for debugging in the terminal you're running inetb from. Exercise left for the reader, but it would be a third set of pipes and a third thread.) The main thread goes back to its accept() loop to take more requests.

In the normal case, let's say that we quit this miniature awk session properly with the quit command. How do the threads react?

• awk terminates, sending a SIGCHLD to the main thread and triggering the signal handler.
• The signal handler reaps the process and based on the PID finds its iothreadstate. It then launches a cleanup thread for that iothreadstate, and goes back to the accept() loop to take more requests.
• The cleanup thread now has to make the threads quit cleanly, since killing them leaves a mess in net_server (killing teams cleans up resources, but not individual threads within a team). It does this by sending a message to both the read-from-process and write-to-process threads. Any message will make them quit.
• For the read-from-process thread, this is sufficient to interrupt its blocking read(). It sees there is a message, and exits gracefully.
• For the write-to-process thread, this is a little more complicated. Even though the socket read should be blocking, in practice signals regularly interrupt it, so we use a select() on the socket to ensure we really do have data to read. There appears to be a bug in BeOS, however, where sending a message sometimes doesn't interrupt select(). We get around this problem by having the select() timeout every 10ms so it can look in its queue, which is less elegant, but better than a tight loop. Anyway, it too sees there is a message, and exits.
• After waiting for both threads to exit, the cleanup thread flushes the socket and closes everything, returns the now spent iothreadstate to the pool and exits itself. Meanwhile, the main thread has already gone on to service other requests. Ain't multithreading great?

What happens if the user just disconnects?

• As in standard POSIX, the write-to-process thread sees that the socket is ready but there is no data. Assuming a signal hadn't arrived, this is treated as a disconnect. It kills the dependent process (this is an entire team, so it's safe) and quits.
• awk has just been killed, so a SIGCHLD goes to the main thread, triggering the signal handler.
• The signal handler reaps the process, finds the iothreadstate, and starts the cleanup thread as it returns to the accept() loop.
• The cleanup thread takes down the read thread as well by sending it a message, flushes the socket, closes everything and terminates. Meanwhile, the main thread has already gone on to service other requests. Another stupendous day in Cheap Thread Land!

It's BeOS' ability to effortlessly spawn huge numbers of thread even on constrained systems (even in 1996 a dual 133MHz wasn't stonking fast, and certainly not the 66MHz version) that makes this arrangement work effectively. Want to handle something asynchronously instead of busywaiting? Make a thread! The thread can block (usually)! Perfect for UI or network! Even the cleanup is asynchronous in inetb so that as little happens on the main thread as possible. The kernel handles all this messing around for you as long as you play by the rules.

BeOS isn't perfect, though, as that last sentence will attest. During my testing of inetb I unsettled net_server a lot. You can restart networking from its preference window, but it seemed bad that I had to do this as often as I did. In fact, as an unrelated note, I was able to pretty much wreck the machine every time if I accidentally started CDBurner. I don't have a burner and you'd think it would handle that circumstance, but it doesn't. The machine goes haywire if I'm lucky; it locks up if I'm not. I eventually had to remove it from the Applications menu. More generally, the notion of uids and gids is a veneer and you're pretty much doing everything as the superuser. That means wrong moves hurt.

But don't forget that early Mac OS X had its own weird problems during its earliest versions. BeOS, at least superficially, gives you that similar experience of a POSIX-alike underpinning with better multitasking and memory management, and it was definitely lighter on system resources than early OS X was, too. What NeXT had was Steve Jobs and a longer history with Apple than Jean-Louis Gassée, and while it is variously said that Be's demanded purchase price is what turned Apple away from buying them, I've always thought it was just a cover story for the real deal to get an original Apple founder back. And that worked out handsomely for Apple. But I think BeOS could have served as the next Mac OS at least as well.

Our next BeOS entry will talk about SheepShaver, which you can think of as "Classic" for BeOS. It even runs PowerPC code natively for surprisingly useable performance. But it started crashing incessantly after I upgraded the RAM in my BeBox. Can we fix that? Of course! Find out how next time!