MacLynx beta 6: back to the Power Mac

Time for another MacLynx save point in its slow, measured evolution to become your best choice within the remarkably narrow niche of "classic MacOS text browsers." Refer to prior articles for more of the history, but MacLynx is a throwback port of the venerable Lynx 2.7.1 to the classic Mac OS last updated in 1997 which I picked up again in 2020. Rather than try to replicate its patches against a more current Lynx which may not even build, I've been improving its interface and Mac integration along with the browser core, incorporating later code and patching the old stuff.

The biggest change in beta 6 is bringing it back to the Power Macintosh with a native PowerPC build, shown here running on my 1.8GHz Power Mac G4 MDD. This is built with a much later version of CodeWarrior (Pro 7.1), the same release used for Classilla and generating better optimized code than the older fat binary, and was an opportunity to simultaneously wring out various incompatibilities. Before the 68000 users howl, the 68K build is still supported!

However, beta 6 is not a fat binary — the two builds are intentionally separate. One reason is so I can use a later CodeWarrior for better code that didn't have to support 68K, but the main one is to consider different code on Power Macs which may be expensive or infeasible on 68K Macs. The primary use case for this — which may occur as soon as the next beta — is adding a built-in vendored copy of Crypto Ancienne for onboard TLS without a proxy. On all but upper-tier 68040s, setting up the TLS connection takes longer than many servers will wait, but even the lowliest Performa 6100 with a barrel-bottom 60MHz 601 can do so reasonably quickly.

The port did not go altogether smoothly. While Olivier Gutknecht's original fat binary worked fine on Power Macs, it took quite a while to get all the pieces reassembled on a later CodeWarrior with a later version of GUSI, the Mac POSIX glue layer which is a critical component (the Power Mac version uses 2.2.3, the 68K version uses 1.8.0). Some functions had changed and others were missing and had to be rewritten with later alternatives. One particularly obnoxious glitch was due to a conflict between the later GUSI's time.h and Apple Universal Interfaces' Time.h (remember, HFS+ is case-insensitive) which could not be solved by changing the search order in the project due to other conflicting headers. The simplest solution was to copy Time.h into the project and name it something else!

Even after that, though, basic Mac GUI operations like popping open the URL dialogue would cause it to crash. Can you figure out why? Here's a hint:

Pencils down. If you're an old Mac programmer, you've immediately spotted the conveniently highlighted cause in the screenshot, but if you're not, here's the answer: when providing function pointers to Toolbox routines, you have to tell the operating system whether it's going to run PowerPC or 68K code. The normal state, as it were, of the Power Macintosh nanokernel is to be monitoring the on-board 68K emulator running 68K code, even in New World Macs, and even in later versions of the operating system where most of the Mac OS was finally PowerPC-native (watch for the message "into the 68K fire" when running a Power Mac ROM with logging on). Don't confuse this with the normal state of your application: your application itself was almost certainly fully native. However, a certain amount of the Toolbox and the Mac OS retained 68K code, even in the days of Classic under Mac OS X, and your PowerPC application would invariably hit one of these routines eventually.

The component responsible for switching between ISAs is the Mixed Mode Manager, which is tightly integrated with the 68K emulator and bridges the two architectures' different calling conventions, marshalling their parameters (PowerPC in registers, 68K on the stack) and managing return addresses. I'm serious when I say the normal state is to run 68K code: 68K code is necessarily the first-class citizen in Mac OS, even in PowerPC-only versions, because to run 68K apps seamlessly they must be able to call any 68K routine directly. All the traps that 68K apps use must also look like 68K code to them — and PowerPC apps often use those traps, too, because they're fundamental to the operating system. 68K apps can and do call code fragments in either ISA using the Code Fragment Manager (and PowerPC apps are obliged to), but the system must still be able to run non-CFM apps that are unaware of its existence.

To jump to native execution thus requires an additional step. Say a 68K app running in emulation calls a function in the Toolbox which used to be 68K, but is now PowerPC. On a 68K MacOS, this is just 68K code. In later versions, this is replaced by a routine descriptor with a special trap meaningful only to the 68K emulator. This descriptor contains the destination calling convention and a pointer to the PowerPC function's transition vector, which has both the starting address of the code fragment and the initial value for the TOC environment register. The MMM converts the parameters to a PowerOpen ABI call according to the specified convention and moves the return address into the PowerPC link register, and upon conclusion converts the result back and unwinds the stack. The same basic idea works for 68K code calling a PowerPC routine. Unfortunately, we forgot to make a descriptor for this and other routines the Toolbox modal dialogue routine expected to call, so the nanokernel remains in 68K mode trying to execute them and makes a big mess. (It's really hard to debug it when this happens, too; the backtrace is usually totally thrashed.)

Because we have access to the source code and know what we're calling (it's our own code), we can simply use an Apple-provided macro to create a static descriptor on the stack as needed. This is a(n in)famous Universal Procedure Pointer, bane of later System 7 programmers everywhere. If you're not sure what you're calling or you're calling a library you don't control, you're reduced to writing these descriptors manually and/or often putting them on the heap. We don't have to do either of those things here, and for other things Apple's Universal Headers generally bridge the differences — meaning mostly the same code just works on both our 68K CodeWarrior and our much later PowerPC CodeWarrior.

Do note that the PowerPC version is already different in one respect: it has a 4MB preferred allocation instead of the 68K's 2MB. No Power Mac ever shipped with less than 8MB of RAM and many Power Macs even have that or a similar amount soldered to the logic board. It also may have problems with 7.1.2, so if you're running a 601 Power Mac that you don't want to upgrade to 7.5 or later, you may need to run the 68K version which still supports 7.0 and 7.1. Otherwise the two versions of beta 6 should act exactly the same. If you notice differences in behaviour, or you get crashes with certain operations, post in the comments.

A couple UI changes in this release. I mentioned the last time that my idea with MacLynx is to surround the text core with the Mac interface. Lynx keys should still work and it should still act like Lynx, but once you move to a GUI task you should stay in the GUI until that task is completed. In beta 5, I added support for the Standard File package so you get a requester instead of entering a filename, but once you do this you still need to manually select "Save to disk" inside Lynx. That changes in beta 6:

The download modal now offers you a direct "Save to disk" option (which, under the hood, "pushes all the keys" for you). You can still use Lynx keys in the modal just as before, but now that your hand is on the mouse, you can finish everything with mouse clicks. Speaking of, beta 6 also corrects a bug where you couldn't overwrite a file with a new one you were saving, even though it would ask you if you wanted to. I also found and fixed a problem that manifested in the PowerPC build (but would affect the 68K build also) when translating slashed paths into MacOS colon paths, turning double slashes into :: which in MacOS is treated as the parent folder.

Resizing, scrolling and repainting are also improved. The position of the thumb in MacLynx's scrollbar is now implemented using a more complex but yet more dynamic algorithm which should also more properly respond to resize events. A similar change fixes scroll wheels with USB Overdrive. When MacLynx's default window opens, a scrollbar control is artificially added to it. USB Overdrive implements its scrollwheel support by finding the current window's scrollbar, if any, and emulating clicks on its up and down (or left and right) buttons as the wheel is moved. This works fine in MacLynx, at least initially. When the window is resized, however, USB Overdrive seems to lose track of the scrollbar, which causes its scrollwheel functionality to stop working. The solution was to destroy and rebuild the scrollbar after the window takes its new dimensions, like what happens on start up when the window first opens. This little song and dance may also fix other scrollwheel extensions.

Always keep in mind that the scrollbar is actually used as a means to send commands to Lynx to change its window on the document; it isn't scrolling, say, a pre-rendered GWorld. This causes the screen to be redrawn quite frequently, and big window sizes tend to chug. You can also outright crash the browser with large window widths: this is difficult to do on a 68K Mac with on-board video where the maximum screen size isn't that large, but on my 1920x1080 G4 I can do so reliably.

For this reason, I arbitrarily decided to enforce a maximum 132 column size because otherwise MacLynx crashes with a null pointer if the screen is too wide, likely an intrinsic problem with MacCurses trying to handle such a large screen matrix. This is not as big a hardship as it might seem because scrolling becomes slower and slower the bigger the window gets — after all, it's not really "scrolling" anything, it's redrawing practically the entire screen every time! You can just drag the window out to whatever width and it will snap back to the maximum.

Conversely, beta 6 also requires a minimum of 80 columns' width. Lynx prompts can look bad at smaller widths and it's not really designed for screen sizes smaller than that, so now we don't let it. Even a 512x384 Mac can still manage an 80-column window with the standard Monaco nine-point font. In a like fashion you can drag the window back in smaller than 80 columns and it will snap back to the minimum.

When resizing horizontally, MacLynx now forces a reload if the width changes and the current document was fetched with an idempotent HTTP method (i.e., GET), since MacLynx (quite reasonably) doesn't cache the HTML; rather, it caches its rendered version, which is now the wrong width. If the method is not idempotent, you get to reload the page yourself. If you simply resize vertically and the horizontal width doesn't change (or it keeps snapped to the minimum or maximum), no reload is required.

A number of bugs are also fixed in the new UTF-8 support added to beta 5. Among them is a character set overhaul to prevent Lynx from internally disagreeing with itself about whether high-bit characters should be interpreted as native MacRoman, or part of a UTF-8 sequence which gets converted to MacRoman. Because of the need to support more UTF-8, including HTML entities and other things which may generate it, the other character sets available in MacLynx other than ISO Latin 1, 7-bit Approximations and the Macintosh 8-bit default are now officially deprecated and will likely be removed in future betas. In fact, I'm considering going further and eliminating other character set support except for Macintosh 8-bit, effectively making the character set option in lynx.cfg a no-op. However, if you are intentionally using another character set and this will break you, please feel free to plead your use case to me and I will consider it.

Another bug fixed was an infinite loop that could trigger during UTF-8 conversion of certain text strings. These sorts of bugs are also a big pain to puzzle out because all you can do from CodeWarrior is force a trap with an NMI, leaving the debugger's view of the program counter likely near but probably not at the scene of the foul. Eventually I single-stepped from a point near the actual bug and was able to see what was happening, and it turned out to be a very stupid bug on my part, and that's all I'm going to say about that.

Cookies were also significantly overhauled (please note that this is in no way a reflection on mainline Lynx, which handles them in a much more modern fashion): as proof, here we are browsing Hacker News and posting a comment. After teaching MacLynx what the SameSite and HttpOnly (irrelevant on Lynx but supported for completeness) attributes are, the next problem was that any cookie with an expiration value — which nowadays is nearly any login cookie — wouldn't stick. The problem turned out to be the difference in how the classic MacOS handles time values. In 32-bit Un*xy things, including Mac OS X, time_t is a signed 32-bit integer with an epoch starting on Thursday, January 1, 1970. In the classic MacOS, time_t is an unsigned 32-bit integer with an epoch starting on Friday, January 1, 1904. (This is also true for timestamps in HFS+ filesystems, even in Mac OS X and modern macOS, but not APFS.) Lynx has a utility function that can convert a ASCII date string into a seconds-past-the-epoch count, but in case you haven't guessed, this function defaults to the Unix epoch. In fact, the version previously in MacLynx only supports the Unix epoch. That means when converted into seconds after the epoch, the cookie expiration value would always appear to be in the past compared to the MacOS time value which, being based on a much earlier epoch, will always be much larger — and thus MacLynx would conclude the cookie was actually expired and politely clear it. I reimplemented this function based on the MacOS epoch, and now login cookies actually let you log in! Unfortunately other cookies like trackers can be set too, and this is why we can't have nice things. Sorry. At least they don't persist between runs of the browser.

Even then, though, there's still some additional time fudging because time(NULL) on my Quadra 800 running 8.1 and time(NULL) on my G4 MDD running 9.2.2, despite their clocks being synchronized to the same NTP source down to the second, yielded substantially different values. Both of these calls should go to the operating system and use the standard Mac epoch, and not through GUSI, so GUSI can't be why. For the time being I use a second fudge factor if we get an outlandish result before giving up. I'm still trying to figure out why this is necessary.

You may have noticed the images in some of these screenshots. They appear courtesy of MacLynx's picture viewer functionality, which will download images and hand them off to the viewer of your choice (on the G4 I use QuickTime 6's Picture Viewer, using the signature ogle). This didn't work for PNG images before because it was using the wrong internal MIME type, which is now fixed. (Ignore the MIME types in the debug window because that's actually a problem I noticed with my Internet Config settings, not MacLynx. Fortunately Picture Viewer will content-sniff, so it figures it out.) Finally, there is also miscellaneous better status code and redirect handling (again not a problem with mainline Lynx, just our older fork here), which makes login and browsing sites more streamlined, and you can finally press Shift-Tab to cycle backwards through forms and links.

If you want to build MacLynx from source, building beta 6 is largely the same on 68K with the same compiler and prerequisites except that builds are now segregated to their own folders and you will need to put a copy of lynx.cfg in with them (the StuffIt source archive does not have aliases predone for you). For the PowerPC version, you'll need the same set up but substituting CodeWarrior Pro 7.1, and, like CWGUSI, GUSI 2.2.3 should be in the same folder or volume that contains the MacLynx source tree. There are debug and optimized builds for each architecture. Pre-built binaries and source are available from the main MacLynx page. MacLynx, like Lynx, is released under the GNU General Public License v2.