Saturday, July 2, 2022

A brief dive into Power Mac INITs and NVRAM scripts, or, teaching Mac OS 9 new device tricks

Although I'd much rather use a real Power Mac, and of emulators I tend to use my own bespoke hopped-up fork of SheepShaver for the POWER9 CPU with my daily driver, QEMU is still important for Mac OS 9 emulation because it handles the full system rather than the quasi-paravirtualization approach of SheepShaver. Indeed, certain classes of application can only run in that context.

However, because QEMU is a much lower-level emulator, that means that things like mice are also emulated, and that tends to chug a bit even with QEMU's JIT (currently KVM-PR, the virtualization system for Power ISA, does not work properly with Mac OS 9 in QEMU for reasons that have not yet been determined). If you use the default mouse support, the mouse is entirely maintained by the operating system and the polling frequency is just slow enough to be frustrating, and you have to grab and ungrab it all the time. The normal solution is to use an absolute pointing device like the QEMU tablet and solve all these problems at once, but the classic Mac OS doesn't support it and the existing Wacom driver doesn't work. (SheepSforza, on the other hand, hooks the system mouse and system mouse pointer into the guest OS, so it's much more responsive and transparent if I do say so myself.)

To support this and other QEMU virtual devices requires updating the guest Mac. And, happily, there are now two approaches at least for doing so for the QEMU tablet, which make an interesting comparison on how we could get other devices supported on the classic Mac OS.

The original solution was kanjitalk755's extension (yes, the same dude who updated SheepShaver, and upon whose updates SheepSforza is based). This would be considered the conventional approach and is almost a canonically simple INIT, so it also makes a good pedagogical example.

On startup, the extension's main() is run. This is actually 68K code, coming from a resource called (unimaginatively) INIT. This block of code knows nothing about USB; its entire reason for existence is to display the extension icon and create a Universal Procedure Pointer for a single PowerPC function in this file. UPPs are needed to tell the operating system the type of code it will be running (remember that the "normal" state of the classic Mac OS, even on Power Macs, is to run 68K code), and since this code is 68K, we need to tell the nanokernel it will be running native code instead.

You'll notice that it must call main() in that second file to start up the USB driver and naturally it does, but the function name main doesn't appear anywhere in this call, so how does it know what function to ask for? In simple terms, it finds the right function because there's nothing else to find. It gets this pointer by using the Mac OS Code Fragment Manager (all PowerPC code, even in resources, is managed by CFM) on a second resource 'PPC ' — in typical Mac applications PowerPC code is stored in the data fork, but PowerPC code can run perfectly well from a resource too. Here it is in ResEdit.

There is only one such resource, and if you look through the entire second file, only one function is not declared static: main(). So when we call GetMemFragment, even though we just pass a single Pascal string \p, we're guaranteed to get a pointer to the only non-static function there because there's nothing else for the CFM to return. The initializer then declares it to be a PowerPC function and calls it.

Now in native code, we locate any class driver for USB HID peripherals and find the device dispatch table as documented by Apple (PDF). Assuming the guest Mac is configured to use an ADB keyboard and mouse (the default if the emulated VIA is the CUDA) and no other interfering HIDs, this pointer will be to the class driver attached to the QEMU virtual tablet, which is USB. Using the dispatch table, the extension then gets and stores the report descriptor and installs a report handler. Control then returns to the INIT resource, which displays the icon and returns control to the operating system.

With the driver thus installed, as the QEMU emulated tablet sends reports our handler locates the logical cursor device using the Cursor Device Manager, moves its pointer and sets its buttons, and makes sure it calls any other previous report handlers. Rather than looking for the QEMU tablet's specific vendor and product code, it simply distinguishes the reports it wants by checking the length of the report.

This is all very Mac-like and clean, and a straightforward example of how one might write a simple Mac OS USB HID driver generally. About all it's missing is code to uninstall the handler, which it could do by calling pHIDRemoveReportHandler in the HIDDeviceDispatchTable struct; it could also look for hot plugging (which, theoretically, could happen within QEMU) or specific vendor/product IDs by using USBInstallDeviceNotification with a callback instead. That said, it has some disadvantages, two minor and one major.

The first minor disadvantage is that you'll need a 68K compiler to build it from scratch and later versions of CodeWarrior don't have one. Theoretically you could use SC from the Macintosh Programmer's Workshop to build the 68K portion, but you'll need to do some modification for the missing A4Stuff.h, and then you'd need to come up with the MPW incantation to link it all together with the PowerPC code resource because there's no 68K linker anymore in CW either. That said, you could simply build the PowerPC resource on its own and then use ResEdit as your "ghetto linker" to replace the 'PPC ' resource in the pre-built extension. Naturally this isn't particularly elegant nor a completely reproducible build, but at least this way you can still make changes to it or use it as the scaffolding for your own extension (change the icon too in ResEdit while you're at it).

The second minor disadvantage is because this driver is not particularly picky, it can conflict with other USB human interface devices that may be connected, which is why you can't run QEMU with the VIA set to PMU as that configuration provides USB keyboards and mice (ergo CUDA, which provides an ADB mouse and keyboard that won't interfere). That means it won't work as is on a real Power Mac with a USB keyboard and mouse either, only one with ADB devices. Again, it should be possible to fix this by refactoring around USBInstallDeviceNotification with a callback and the right vendor and product IDs, or using an approach that we'll discuss momentarily.

The major disadvantage is what happens when you must boot with extensions off. Since QEMU in tablet mode doesn't provide a mouse by default, that means you would essentially be limited to the keyboard without reconfiguration. I don't think I need to tell any regular reader here that the classic Mac OS is a lot harder to work with when it's limited to a keyboard.

That brings us to a rather more unusual solution, Elliot Nunn's take. Nunn's version is a radically different approach: it installs a fake ROM resource containing a USB driver specific to the QEMU tablet, and to ensure it is available on startup, is actually embedded in an NVRAM script that Open Firmware runs. You provide this as the NVRAMRC to QEMU with a lon-gass command line argument (which is partially in Base64 and compressed with LZSS to be under a 2K Open Firmware limit bug).

There are several distinct parts to this version. Recall that Open Firmware's "native" tongue is Forth. Stored in NVRAM is a small Forth script that decodes and decompresses a payload (marked with a ~ character). The script is executed by Open Firmware on startup, causing this payload to be installed as an Open Firmware property and run in the early boot process before any of Mac OS 9's built-in USB class drivers. It finds the ROM driver resource USBHIDMouseModule referenced by a known location, sets the low-memory global RomMapInsert at 0xb9e to true, and patches into the 68K system trap 0xA06A HSetState (PDF, part of the Memory Manager). Every time this trap is called, if its patch routine (written in 68K assembly) sees that the handle argument matches that resource, it substitutes its own handle.

The handle points to this driver, which is primarily PowerPC, but has a 68K component for a particular reason. Unlike the driver above which simply worms into an existing HID class driver, this driver is a USB driver all its own (it has to be, it's replacing the one in ROM). It knows to look for the specific vendor and product, but it makes sure it patches the device class to "vendor specific" so that any later driver loaded from disk won't get priority over the built-in driver. This patcher is written in 68K assembly purely to save space and called with (you guessed it) a UPP during initialization. With this new driver installed USB events are handled by a more formal state machine, within which a USB interrupt read event handles the report and moves the cursor. The cursor movement method here is also more interesting: instead of using the Cursor Device Manager, it sets low globals to the new position of the mouse pointer and directly calls the vertical blank manager routine CrsrVBLTask through its vector at JCrsrTask to redraw the pointer right then and there. This is how the mouse pointer was moved on systems that lacked the Cursor Device Manager but all classic-capable Power Macs still support this method for compatibility.

But building it is the really fun part. MPW supports build scripts, which are analogous to things like Makefiles. The master build script starts off by calling the build script to build the USB driver. This creates a single ndrv component in a conventional manner, compiling it with the PowerPC compiler MrC and linking it with PPCLink. The next step is the early loader, which needs to know its own length, because the USB driver is concatenated to it. It builds itself with a dry run dummy value first and then compiles in its own length on the second run, appending the driver to itself.

Obviously MPW doesn't know how to do the compression and Base 64 encoding, so the next step is to build the LZSS compressor (note the use of either the SC or C compilers to make the tool, in case one or the other bugs out). The compressor isn't a classic Mac application; it just has a regular main() and uses stdio.h like anything you'd run from a command line anywhere else, and such programs can be run directly within MPW as tools. So we'll link it as an MPW tool and immediately call it to compress the combined loader-driver. The the Base 64 encoder-argument generator is compiled in the same way, which trims comments and extraneous whitespace out of the Forth script and substitutes in the encoded compressed binary at the right location. It then emits this final "object" over standard output to the MPW worksheet as the command line options to pass to QEMU.

It too has some drawbacks. First, that's a lot to pass on the command line, although modern systems won't have a problem with it. Second, while the trap used to swap in the new driver is a documented and perfectly valid 68K trap, this trap does get called for other things, so there ends up being a small amount of additional overhead. Additionally and analogously to the original driver, this driver does not cleanly handle other USB HIDs, although its severe OF-imposed space contraints also mean there's little room to handle anything else anyway. (You would do so in the state machine code; exercise left for the reader. You could marry such a driver to the conventional extension approach used in the previous iteration for a best-of-both-worlds hybrid.) On the other hand, this more low-level approach means the driver is always available, even if extensions are disabled.

Much of the "institutional memory" on creating classic Mac OS system extensions and components is fading from Google and other sites, and Apple has taken down a great deal of it as well (if it posted it at all). Trying to document what's out there and preserve it for retrocomputing posterity gives us a chance to support new and better environments and devices either on real machines or in emulation. The classic Mac OS was a strange OS internally, but a real joy to actually use, and better system integration makes it that much more accessible to curious future generations.

Sunday, June 26, 2022

Overbite Palm 0.2

Overbite Palm 0.2, the Gopher client for classic Palm devices, is now available (see it running on real hardware). This update includes Shawn Mulligan's Sony Clié key patch (thanks!) plus more aggressive memory management and menu compression, which should not only increase the number and length of documents that 68K Palms with their pathetic dynamic heaps can view but also cut down on the fatal Plua-generated "out of memory" errors that our memory heuristics failed to anticipate. Additionally, all transfers are hard-capped to 32K raw data since this appears to be the empiric limit for the native scrolling text gadget. It was tested on my Palm Centro and AlphaSmart dana, as well as my emulated m515. You can get it from Github (plus Plua source, natch) and I will be making a place for it on the Floodgap gopher server as soon as I get a round tuit.

Saturday, June 11, 2022

prior-art-dept.: ProleText, encoding HTML before Markdown (and a modern reimplementation)

Steven P. Spackman allegedly once observed that "flat text is just never what you want." Which, I guess, is true: half the historical advances in computing have come from figuring out ways to tart up plain text, whether embedding control codes or out-of-band styling or in-band markup. However, with the exception of out-of-band styling (I always liked the Macintosh text file formats that kept the text in the data fork and the styling in a resource), you still needed to parse the file or at best you'd get blocks of text separated by gobbledygook. Enhanced text formats like Markdown were thus designed to make cognitive sense to human eyes without further parsing — but also encoding sufficient metadata to facilitate improved ways of rendering the document.

Markdown circa 2004 has displaced most of the others today, but it explicitly never claimed to be the first such human-readable format; indeed, AsciiDoc predates it by about two years, reStructuredText a year before that and MakeDoc about a year before that. For that matter, some of the concepts popularized in Markdown might not have existed at all were it not for earlier ancestors like 2002's Textile.

But a forgotten rich text language predates most of these, with the interesting property in that much of the markup is encoded using trailing whitespace, almost a fusion of in-band and out-of-band styling systems. If the whitespace is munged, it's largely just a text document (like those particular Mac files if you pass along only the data fork); but if it passes through intact, an intelligent converter can use attributes encoded in the whitespace to style it into HTML. That system is ProleText.

Brad Templeton was one of the early names in microcomputers, starting out as Personal Software's employee #1 while still a teenager. Peter R. Jennings, who wrote Microchess for the Commodore KIM-1, founded the company with Dan Fylstra to publish it. In 1979 they made a deal with outside company Software Arts to publish their new program for the Apple II called VisiCalc, which is now known as the first electronic spreadsheet package.

Templeton was experienced with the Commodore PET and one of his initial jobs was the VisiCalc port to that platform. Having done the job, Personal Software, which in 1982 changed its name to VisiCorp, subsequently assigned him to work on a graphics companion package for a new and secret upcoming machine (that we now know as the IBM PC) while he was an undergraduate at the University of Waterloo in Canada. He developed the software in C remotely over a Tymnet X.25 packet-switched link to an East Coast minicomputer that also happened to have Usenet newsgroup access via uucp and Arpanet. He was hooked, and successfully lobbied to get the University a Usenet feed with generous connectivity paid for by Digital Equipment Corporation. It was the first international link to Usenet outside of the United States.

Templeton left VisiCorp in 1983 and founded a new software company, but didn't forget the experience. His online activities started to take up substantial portions of his time, particularly his work as moderator for rec.humor.funny, and he began looking at ways to potentially earn from it without arousing the ire of NSFNet, who still administered the nascent Internet in those days and banned commercial use of the backbone.

In 1988 The Source was dying out partially because of the expense of its content contracts, but CompuServe was flourishing, not least because it was cheaper and had lower costs. Part of those lower costs came from outsourcing a large part of its news and content acquisition to a company called Comtex, which started in 1980 electronically publishing scientific research and expanded into financial and general mass-market news. Templeton contracted with them and others as well, converting their feeds to articles that could be consumed over Usenet but within a private newsgroup feed for which he planned to sell subscriptions. He reasoned, and NSFNet agreed, that academic institutions could get his feed over the Internet because they would use it for academic purposes; thus, as long as he got academic institutions up first, they could then propagate his premium newsgroups to commercial subscribers using uucp and other non-Internet means where NSFNet had no involvement (and he would give them and other such feeders a discount as incentive). Eventually he was able to cut out the middleman to contract with content providers like United Press International directly and added other syndicated content. The new ClariNet delivered its first set of articles to Stanford University in 1989, which in turn propagated throughout Silicon Valley, and Brad tells the rest of that story himself. (I met Brad at a Vintage Computer Festival one year, where, instead of merely infrared-beaming his contact to my Palm m505, he sent me a "You Have Been Hacked" card which also served as his contact. Very funny.)

ProleText (circa 1995-6 or thereabouts) came about as a means to have a way to visually enhance ClariNet's premium articles, but in a fashion that wouldn't look any different — or at least much different — to text consumers. Articles were authored in ProleText on the ClariNet side with the intent that a sufficiently advanced newsreader could do the processing on the client side or with a CGI script on a webserver (a translator written in C was provided). If the articles remained unprocessed by a text newsreader, however, the text was still legible. The encoded formatting could even survive cutting and pasting in a text editor to at least some degree. In fact, that's the case for the inline ProleText in this very article.

Whitespace formatting works by constraining most lines in the document to 60 characters and then using a sequence of spaces delimited by tabs to emit "tuples" (for example, [SPACE][SPACE][TAB][SPACE][SPACE][TAB][TAB] translates as a 2,2,0 tuple, which is understood as a header marker). The tuples tag the line with a single format type, or possibly a continuation from a preceding tuple. Special tags occupy their own lines and some act as containers. It is able to encode hyperlinks, headings, basic paragraphs and breaks, horizontal rules, images, lists of varying types and pre-formatted text. An in-band system of escapes using various metacharacters (#, * and _) allowed text decoration with boldface and italics, as well as inline links and images.

There is no great publicly available corpus of ProleText, so for didactic and personal gratification purposes I did two things. First, I reimplemented a ProleText to HTML translator in Perl using Brad's specification and probing his original C version of same, called inform (not to be confused with the interactive fiction package). My version corrects some edge cases with how it processed inline substitutions — see below — and I think it does a better job with more standards-compliant HTML (admittedly when this was written there was much less concern about it). On the security site it's also less likely to get suckered into emitting arbitrary characters in bad places by malicious input, and by being written a memory-safe(r) language, it is much less prone to general misbehaviour as well.

Second, because of the extreme paucity of ProleText in the field and the relative difficulty with handwriting it in a text editor, I also implemented an HTML to ProleText translator, similar to things like Turndown. I'm not aware of any such module for ProleText ever existing publicly (Brad himself says "[a]n HTML to Proletext translator is needed") nor for any other source format. You can feed the output of one to the other and see how they interact. We'll do some examples below.

The two tools are on Github as the "PeroleText" (heheh) toolset. They are tested with Perl 5.8 or higher, and do not depend on any external modules (the HTML-to-ProleText converter in particular includes its own miniature HTML parser, because I'm one of those people who will reinvent the wheel given the slightest opportunity). They are coded so that you can either chmod +x them and run them directly, or require them into a script, or rename them to a .pm and use them; the magic unless (caller) lets the script determine its mode of operation based on the context (see example symlinks). Both tools are stream-oriented on both input and output, so you can send files or pipe output of any length to them. If you want to use them in your own code, I'll explain briefly the class functions at the end, though there are only three and they are largely the same for both modules.

Let's start by seeing how a ProleText file is tagged. This example is the very same one Brad himself provides (the Clinton references are very 1990s), and is in the Github repo as example.txt.

% chmod +x pt2html
% ./pt2html -debug example.txt
220001> 
    80>                                 A big title
     1>                               With more for H1
    37> 
     8>   Point One
     3>                 This is the definition for Point One
     8>   Point Two
     3>                 This is an even bigger definition for point two
     2> 
     2>         WASHINGTON (AP) -- Greek Prime Minister Andreas Papandreou will
[...]

What pt2html in debug mode does is merely emit the contents of the file, but display any tuples it finds on individual lines. Blank lines can have tuples, too (a completely blank line is, reasonably, seen as a paragraph break), and in fact certain tuples only have meaning or function in a different way when they're attached to a blank line instead of one with text. The (2,2,0,0,0,1) tuple says this is a header for a ProleText document, and the ProleText version in use is 0.0.1 (which is the only known extant version). When a decoder sees this, it shifts into decoding mode and starts translating any tuples it finds on subsequent lines until it gets a trailer (2,3,0). A document might shift in and out of ProleText multiple times. Lines that are not within a header-trailer pair are considered unformatted and emitted as plain text; similarly, if the line has no tuple at all, it is also treated as unformatted plain text.

Tuples don't necessarily translate into specific HTML tags, even though many do, and some translate into multiples. For example, tuple 8,0 provides both a title and an <H1> heading. The first line is used as both the title and the first part of the heading, and then subsequent lines are incorporated in the heading. In HTML as generated by pt2html, it looks like this (hard wrapped for legibility; actual output is two lines):

% ./pt2html example.txt
<!DOCTYPE html><HTML><HEAD><TITLE>                              A bi
g title</TITLE></HEAD><BODY><H1>                            A big ti
tle
                              With more for H1</H1>

The tuple 1 on the next line indicates it is a continuation of tuple 8,0, so it becomes the second line of the block.

ProleText tuples are not true containers, even though some modal line tags can act like containers. Moreover, no single line can have multiple tuples attached to it (ProleText is nearly completely line-oriented). Instead, to facilitate things like inline links and boldface and other kinds of text decoration, ProleText provides a system of inline substitutions which operate within blocks. Continuing the debug output:

     2>         Clinton invited Papandreou to Washington last fall and the dates
     1> have now been #<www.clari.net/foo#> *arranged* #:, said White House Press
     1> Secretary Dee Dee Myers.
      > 
     9>                         news:clari.news.briefs
     1>                    The newsgroup clari.news.briefs

There are two ways to make a link, both demonstrated here. The second way using tuple 9 takes the first line as the URL and the second line as the text of the anchor (if a single line block, the URL is used for both the link and text). However, the first method used (in the middle of a tuple 2, a regular paragraph) uses hash characters and greater/less-than symbols to set off the URL, and a hash character followed by a colon as the terminator. This maps directly to <A HREF...> and </A>:

<P>        Clinton invited Papandreou to Washington last fall and th
e dates have now been <A HREF="http://www.clari.net/foo"> <STRONG>ar
ranged</STRONG> </A>, said White House Press
Secretary Dee Dee Myers.</P>
<P>

<A HREF="news:clari.news.briefs">                  The newsgroup cla
ri.news.briefs</A><

Also note the boldface with asterisks, same as many later formats (you can thus guess that italics use underscores). Even though these are not preformatted lines such as you would use <PRE> with, the spacing and line delimiters are passed through and become whitespace to HTML. As such the whitespace around the link text "arranged" is faithfully maintained, but neither Brad's C implementation nor my Perl implementation requires spacing to recognize inline sequences. Interestingly, boldface and italics are always automatically cancelled at the end of a line.

Inline sequences also permit inserting images (there's a conventional tuple for images too). Another unusual touch is that even though the link lacked the http:// portion, the spec specifically requires anything starting with www. should have the protocol added to it. This saves precious line width, though I note with some amusement that even though the spec strongly urges a max of 60 character lines, this example file doesn't adhere to that advice.

No system of generating ProleText from other types of documents seems to have publicly survived, so let's turn our attention to the second tool I wrote, html2pt. This turns arbitrary HTML, even something you might pipe to it from curl, into as good a reproduction in ProleText as it can automatically generate.

Let's compare it with the more famous Turndown, which is the same type of process for Markdown (alternatively, compare with the output from something like Html2Markdown). The test vector used below is provided in the Github repo as test.html. Turndown, using that HTML as input, generates this output:

Turndown Demo
=============

This demonstrates [turndown](https://github.com/mixmark-io/turndown) – an HTML to Markdown converter in JavaScript.

Usage
-----

    var turndownService = new TurndownService()
    console.log(
      turndownService.turndown('<h1>Hello world</h1>')
    )

* * *

It aims to be [CommonMark](http://commonmark.org/) compliant, and includes options to style the output. These options include:

*   headingStyle (setext or atx)
*   horizontalRule (\*, -, or \_)
*   bullet (\*, -, or +)
*   codeBlockStyle (indented or fenced)
*   fence (\` or ~)
*   emDelimiter (\_ or \*)
*   strongDelimiter (\*\* or \_\_)
*   linkStyle (inlined or referenced)
*   linkReferenceStyle (full, collapsed, or shortcut)

The ProleText equivalent, or at least the best match I can make the script generate, isn't too different superficially (the tuples are present in this output; try drag-selecting the text to see them):

% ./html2pt test.html
  	  				 

Turndown Demo  	


This demonstrates  
#<https://github.com/mixmark-io/turndown#>turndown#: – 
an HTML to Markdown converter in JavaScript. 

Usage   	

  	   	
var turndownService = new TurndownService()
console.log(
  turndownService.turndown('<h1>Hello world</h1>')
)
  	  				 

------------------------------------------------------------	 

It aims to be #<http://commonmark.org/#>CommonMark#:  
compliant, and includes options to style the output. These 
options include: 
   	 
    * headingStyle (setext or atx)   
    * horizontalRule (#*, -, or #_)   
    * bullet (#*, -, or +)   
    * codeBlockStyle (indented or fenced)   
    * fence (` or ~)   
    * emDelimiter (#_ or #*)   
    * strongDelimiter (#*#* or #_#_)   
    * linkStyle (inlined or referenced)   
    * linkReferenceStyle (full, collapsed, or shortcut)   

 
  

Unlike the example file, html2pt tries aggressively to keep everything to 60 characters as preferred in the specification. If we pipe that to ./pt2html -debug, we can see the tuples explicitly, or pipe it to ./pt2html to generate something very similar to the original HTML, including the preformatted plain text section in the middle and the unordered list. The literal asterisks and underscores in the list use different escapes than the backlashes in Markdown, though another part of the spec is that inline lists using asterisk bullets become list items, as you would expect. The same thing is true for ordered lists and dictionaries.

Its aggressiveness about line length extends to links. If a link is too long to inline in the text, html2pt will try to use tuple 9 and just eat the long line there rather than have a long line mess up text flow. The downside of doing it this way is that tuple 9 is a block of its own, so it tends to introduce an epenthetic paragraph break immediately after even in those situations where it can recover what the last tuple block was. If a line breaks in the middle of a section of boldface and/or italics, a new set of inline substitutions is automatically emitted on the next line to continue it as seamlessly as possible.

Relative links are emitted as she is spoke and thus will work fine if the document is rendered back to HTML, though jumps to fragment anchors in the text aren't currently possible (while tuple 4,2 lets you emit a series of autogenerated anchors, there's no way to generate an exact anchor the document specifies).

Let's feed it something a little bigger; I picked this old entry of mine because it has some preformatted blocks as well. Blogger puts a lot of crap into the page which doesn't translate through and causes a lot of blank lines, so html2pt has output filtering to try to cut down on the spew. Tuples are invisibly present in the text below.

% curl http://oldvcr.blogspot.com/2022/04/tonight-were-gonna-log-on-like-its-1979.html | ./html2pt
 				 
Old Vintage Computing Research: Tonight we're gonna log on       	
like it's 1979 (Telenet, Dialcom and The Source) 


#<http://oldvcr.blogspot.com/#> Old Vintage Computing  	
Research #: 


REWIND and PLAY  



Sunday, April 10, 2022   	

#:  

Tonight we're gonna log on like it's 1979 (Telenet,    	
Dialcom and The Source) 


https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_vWjs5J6UFSm28d70dufz1R59zD4SURtPn7em4NHdZv4JASS9mTvhFy1YXTBpendiyQ_0iPGSgo5uyTwH3SpVtoNV6--ySkAQBu2pomDfuWlC-KtdzLy2EWJ3Wdk3topIHlbXQnrGAgb5ZtQMgjjBrp-F5zCZxwAvnKzDVYtfietfEXtss4iHzGmV/s4080/IMG_20220403_210236.jpg         
[Image] 

Teletypes may have killed a lot of forests by emitting  
every line to hard copy instead of a screen, but there's 
something to be said for the permanence of paper, 
especially when people hang onto it for some reason. While 
getting duff units to build a functional 
/2022/02/refurb-weekend-texas-instruments-silent.html         
Silent 700 Model 765 ASR teletype 

Obviously pages with lots of text like this translate relatively well, but ProleText was designed for that use case specifically, so that shouldn't be too surprising. Notice the spurious #: end of an anchor, which came from a <a name="..."> it couldn't translate. After dithering over whether I should suppress them there, I decided to leave it alone since it makes a nice visual section header (pull requests welcome if you disagree). As you go through the document, you'll notice it jump in and out of preformatted blocks by emitting trailer and header tuples, which is absolutely acceptable behaviour per the spec.

While most of the documented tuples are implemented in pt2html (note that html2pt does not generate the full spectrum, for a variety of reasons), there are a few which I decided not to. Nevertheless, some are unusual features that have little peer in other systems, so they're worth mentioning. Perhaps the most unusual is the macro facility, where a document encoded with a future version of ProleText can "polyfill" an old decoder by providing substitution hints on tuples it may not support. These hints can even nest up to 10 levels. However, there's no future version of ProleText to demonstrate with it, so the facility presently has no functional use (if you pass an unknown tuple to pt2html, currently it will simply echo the line and flag the offending value). Related to this I've also not implemented the behaviour for handling undefined line tags: again, there's nothing to test it with, and coming up with some and causing html2pt to generate them frankly defeats the purpose of historical reconstruction.

A tuple I've outright refused to implement is tuple 4, or raw HTML. That's like it sounds: the HTML is emitted to the client. I think raw HTML escapes are asking for trouble — I don't like it in Markdown either — and so using tuple 4 will generate a warning and be treated like an ordinary paragraph. Although line tag 3,4 is also called RAW, this version wraps its blocks in <XMP> tags instead of emitting it straight. Theoretically this should escape any HTML but the tag is quite antiquated and may not be handled well in modern browsers, so I map them to <PRE> and substitute any problematic characters.

I've also not followed the spec exactly as written for #<...#> and #{...#}, which implement inline substitutions for links and images respectively. In the spec the first half of these pairs emits a partial HTML tag <TAG... and the second half closes the tag with >, which smells ripe for abuse by a cleverly malicious document that might insert metacharacters in the middle. Instead, an entire discrete set of one or the other must appear on one line and not have quotation marks or greater/less than signs to be substituted between them; if such hankypanky is detected, or the substitutions are unbalanced, they are emasculated by escaping the offending sequence. I haven't done the same for any unbalanced #: because multi-line links are allowed, and #: generates a full tag </A> anyway, which doesn't seem to hurt anything if there wasn't a link specified before it.

In any event, the HTML-to-ProleText converter should be considered a work in progress that worked acceptably well only for the corpus I ran through it during development. It also needs to have more HTML entities added to its converter. Something for a future lazy weekend.

The code and these examples are on Github. Since Brad merely copyrighted the existing C converter and didn't put it under any particular license, I've put these under the Perl Artistic License 2.0. If you want to play with ProleText in your own Perl script instead of just running them on the command line, you can either require or use them directly into your own code (no modification required). The ProleText-to-HTML converter object is called PeroleText and the HTML-to-ProleText converter object is called PeroleHTML. They are line-oriented; you feed lines of text to them until you're out of data. Both have just three methods, differing only in the arguments to the constructor:

 my $p = new PeroleText([$debug]);
 print $p->proline($string);
 print $p->done;
 
 my $p = new PeroleHTML([$debug,][$img]);
 print $p->proline($string);
 print $p->done;

The $debug argument (0=false, 1=true) indicates whether to run the converter in debug mode. For ProleText to HTML, this emits the tuples. For HTML to ProleText, this emits the view of the HTML parser. The second argument, only with PeroleHTML, says whether to always replace images with their ALT text (if true, <IMG> tags are used as is). Any unspecified argument to the constructor is treated as false.

Then, for each line of data, call the method proline (this is a specific pun I refuse to explain but Canadian Commodore users should get — hint: Jim Butterfield and Steve Punter), which will return a string you can print or store. This string may not include all of the elements you passed it, particularly in the HTML converter, so when you're out of data call the method done to return anything left over in the object's buffer. The object instance is not reusable when the conversion is complete; destroy it or let it go out of scope after calling done, and make a new instance if you need to do a new operation. Although there are other "secret" instance methods available for these objects, they should be considered neither public, stable nor supported. If you want a one-liner example, try this (in the repo directory):

% perl -I. -MPeroleHTML -e 'my $p=new PeroleHTML();while(<>){print $p->proline($_)} print $p->done;' < test.html

The remit and spectrum of supported HTML being as limited as it was in the mid-1990s, ProleText was no doubt a concept a bit too ahead of its time. Combine that with the lack of an ecosystem, no publicly available generators, a more difficult means of editing and its origination as part of a proprietary commercial service, and it's not too surprising it didn't catch on outside of ClariNet. (For that matter, it's not clear how much, if it all, it remained in use even at ClariNet after the company was sold in 1997 to Individual Inc.) Plus, the multiple and not quite overlapping alternatives for some constructions and an eclectic, non-orthogonal and incomplete (even for 1996, the last date on the C translator files) selection of tags make it questionably suited for the expanding Web despite attempts at future-proofing. Templeton, while asserting it "worked well for delivering ClariNet's news," himself acknowledged its flaws and proposed a later standard called Out of Band HTML — which ended up never being implemented at all. Any similarity to later text markup systems thus seems frankly coincidental and almost certainly a case of convergent evolution.

Nevertheless, ProleText remains an indisputable example of (forgotten) prior art, and demonstrates clearly that even in the mid-1990s there existed a system tolerant of manipulation that does nearly everything modern Markdown does, and did it even before many of Markdown's explicitly acknowledged influences. Its unique design choices have their own unusual consequences, but it seems to have achieved the purpose it was created for even if it survived no further.

Monday, May 30, 2022

So long, home T1 line; hello, hacking the T1 router

Floodgap has had a T1 line since I moved into this house in 2011. I'm one of those weirdos who runs my own hardware and prefers to avoid co-lo so that I can access things whenever I want despite the additional logistical complexity, and also acts as useful immunity against acquiring other expensive hobbies. The area was boonies-ish when I moved here (big house, cheap price, bottom of the market after the housing crash), arguably still is, and for at least several years there was no DSL due to its distance from the central office (fiber? hahahaha). Cable was around, but the only cable provider at the time refused to pull a run or even quote me a price to do so despite sending contractors on three separate occasions to scope out the site. After several weeks of downtime culminating in me making a formal complaint to the Public Utilities Commission, they agreed to stop messing around and released me from the contract. Of course, by then I was down for nearly a month.

T1 lines have never been cheap, though back in the day they were prized because at 1.536 Mbit/s each way they were comparatively high capacity. At my first job out of college in 1997 the university had a T1 connected to an AdTran CSU/DSU, adding more T1 lines on and bonding them for additional bandwidth until they upgraded to optical fiber. A friend of mine in the very late 1990s had his own residential T1 (this was when consumer DSL was uncommon and 56K dialup was still frequent) that his employer paid the bill for, reportedly close to a cool grand a month in those days; he would never been able to afford it otherwise. On a cost basis alone (and certainly dollars per megabit) a T1 would have been far from my first choice, but I needed a reliable server-grade connection and I couldn't find any other alternatives at the time, so if I wanted to get my hardware back online from the house I was going to have to pay up and get one. Rather than use the actual telephone company I went with an overlay vendor and was quoted $295 a month on an annual contract for a 16-address netblock plus $199 installation. Now going into my fourth week of downtime, I signed immediately. They called the telco who came out the next week, installed the smartjack (we'll talk about what this is), took over both telephone wire pairs to the house and wired it into the pedestal — conveniently, the local telco pedestal is literally in my backyard. Good thing I'd already moved the house alarm modem to a wireless connection since I could no longer have a landline now. I then ran lines from the smartjack into the server room (thanks to the telephone guys I used to work next to when I was consulting I already had good experience with a punchdown tool), the vendor came out the week after that with the T1 router, and finally Floodgap was back up.

The original idea was to use the T1 until something less expensive came along, but the T1 just plain worked and was always highest priority on service calls, so inertia and inflation eventually turned the $295 a month into $399 and a 12-month contract into 48. Still, it was a tariffed line with a service-level agreement, I had plenty of addresses and my personal bandwidth requirements have always been modest — I don't cloud, I rarely stream and YouTube is worse than television — so I ended up just using the T1 as my personal ISP at the same time and avoiding a second bill. This worked out fine for awhile except, of course, for love. My wife needed her Netflix and her iCloud, and by then the previously intransigent cable provider had been bought by someone else (fiber? hahahaha) who didn't know any of the previous history; they came out and finally pulled an RG-6 cable run five years after the fact, and switched us on. I moved the Wi-Fi to the new cable net and her bandwidth needs were thus met in the manner to which she was accustomed. We joke about the his'n'hers networks: I still had my lab and servers on the T1, and everything else including her devices was on the cable.

Well, it was good we did that because I mentioned in January this year that the vendor (which had changed owners twice over the years) was abruptly getting out of the residential T1 business and I had a month before it was switched off. I may well have been their last customer in the region. So I'd like to publicly thank John who reached out and offered a no-strings VPN arrangement — which I'm routing over the cable — to keep Floodgap online while we consider our housing options in a market as bad as it was good when I first bought the place. We turned the VPN on and the vendor turned the T1 off. They never asked for the router back and the smartjack still sits in the back of the house.

Now it's Memorial Day in the United States and I suppose I'll have to do something about that now superfluous wiring run sometime soon. Before I do, though, let's document the T1 for a generation who may have never seen one ... and figure out something fun to do with the router they left behind other than, you know, routing stuff.

Like every computing story, this one begins with a box.

And that box can go by various names, but most commonly is known as a network interface device, or NID. It is the meeting point where my wiring meets the telco's.

Before we look at that wiring, let's also discuss in general terms how analogue telephone service is delivered. The wiring to your house is called the local loop, and consists of one or two pairs, each pair containing a pair of wires referred to as "tip" and "ring." This name came from the old operator plugboards where the tip wire was connected to the tip of the connector and the ring to the conductive section of the "sandwich" on its shaft (like a quarter-inch headphone plug). These lines carry voltage, with a potential difference of about 48 volts or more between the two, and when the phone line is not engaged these lines float; they are not grounded (except perhaps briefly for signaling purposes) to avoid picking up alternating current hum.

For plain old telephone service (POTS) and consumer asymmetric DSL (ADSL, the most common residental data connection over copper wire), a single tip/ring pair is sufficient to serve as a "telephone line." Many houses have two pairs and thus two lines available, though the second isn't always hooked up at the exchange.

T1 works over the same sort of wiring. The T-carrier system was developed in 1962 by Bell Labs as a means for digitally transmitting multiplexed phone connections, but because T-carriers are digital systems they can transmit any digital data, not just telephone calls. The T1 is the lowest capacity of the T-carriers, transmitting at 1.544 Mbit/s using a carrier signal called Digital Signal 1 (DS-1), though after removing the framing the effective throughput is 1.536 Mbit/s. This is enough for 24 simultaneous 8-bit mono phone calls sampled at 8kHz (or a very slow night of Netflix) and a call center might get a T1 purely for telephone connectivity. It requires four wires, so two pairs, which fortunately would have been possible with my house without pulling more wire runs to the pedestal. If I wanted to bond multiple T1 connections together, as many places did, I would have needed extra two-pair sets for each additional T1. The European E1 and Japanese J1 lines are similar; later additions to the T-carrier spec enabled higher capacity lines such as the T3.

In modern installations over copper wire like this one, however, T1 lines are usually provisioned using an early type of DSL called high-bit-rate DSL, or HDSL. Baseline HDSL also requires two pairs, so from a wiring perspective it has the same physical plant requirements as traditional DS-1, but it can transmit over much longer distances without repeaters and correspondingly uses higher voltages. (Later DSL technologies like symmetric DSL [SDSL] cut the requirement down to a single pair, but you couldn't also use it as a phone line; ADSL, on the other hand, can co-exist with voice frequencies on the same pair.) Despite the name HDSL is not, in fact, faster than later DSL technologies. HDSL was specifically developed to replace legacy T1 runs and is not rate-adaptive like ADSL or SDSL; that is, although you only get 1.536 Mbit/s, you always get 1.536 Mbit/s, and you always get all of it up or down. The subsequent standard HDSL2 only needed a single pair to function, but the installation here is actually HDSL4, which still uses the second pair for enhanced robustness.

Let's open the box up. You'll notice the panel directly in front of us is divided into two sides, one with jacks and accessible wiring and the other tightly closed. This is the demarcation point or DMARC, not to be confused with the later E-mail authentication scheme. The telephone company has exclusive access to the closed left side, notionally so you don't get zapped (the telco guy installing the box said the differential was about 197 volts, "and it bites a bit"), but also so you don't screw around with their work. Everything on the left is their problem. Everything on the right is yours.

The wires coming into the closed section are the two pairs from the pedestal routed through my utility closet directly behind the NID. Those get fed through the black cable between the door and the panel to the smartjack, and the smartjack then sends signals back to the (in this case) top-most RJ-45 connector. This is now considered my wiring. The cable then feeds back into the utility closet on the right side; note the blue "ring" on it because we're going to find its other end in a minute. In this NID, only the top-most jack is active; the others would be for additional T1 lines if they were present.

The smartjack lives in the door, taking HDSL signals from the telephone pairs and emitting "conventional" T1 signals on the other end. This one is an AdTran H4TU-R, an HDSL4 remote-end transceiver, festooned with its bank of status lights that are now dark since the line is down. T1 lines always bidirectionally transmit at a fixed rate even if they're just zeroes, which is useful, because the line status is thus effectively under constant monitoring. If there is an issue in the line towards the telco side, then no signals or proper framing will be received from the telco by my smartjack and it will enter "red alarm," sending a "yellow alarm" (alternating signal) back to the telco if it can and a "blue alarm" (the so-called Alarm Indication Signal, all ones) down to my equipment such that the signal is recognized as obviously abnormal yet framing is preserved and clock recovery is possible. The other lights will then tell me what the likely issue is, almost always shenanigans at the central office causing total loss of HDSL reception (one day they forgot to plug me back in after an upgrade). If there is a break in the line towards the customer side, however, such as my T1 router releasing the magic smoke and failing to transmit to the smartjack, my smartjack will send a yellow alarm to me and a blue alarm to the telco.

To test where the problem might be, the telco can attempt to remotely put the smartjack into "loopback" and reflect any signals it sends back to it. In loopback, if the alarm state halts then the loss of signal must be on my side of the DMARC (and if it were on the telco side they might not be able to send the loopback sequence anyway). The smartjack is the property of the telephone company, so if it breaks, part of the tariff agreement is they come out and fix or replace it; the RS-232 port lets the tech plug in a serial cable and query the smartjack's status on field calls.

The smartjack board itself. The card is on a hinge, so it just comes out of the door. Notice the large number of fuses for voltage protection.
On the right, next to the ground clamp, you can see the pairs coming from the telco pedestal through the conduit under the yard, up to the red splice connectors to the wires feeding the telco side of the DMARC (my actual house wiring isn't even involved here; the phone jacks inside were dead as soon as they were disconnected). To the left of the splice connectors is the other end of that blue "ringed" RJ-45 cable and the hand-punched jack which marks the beginning of my home wiring. T1 cabling has separate shielding for each pair to avoid crosstalk on long runs like the 25-ish feet from here to the server room, but if you don't have T1-rated cable you can just separate each pair into a separate Cat. 5 run, which accomplishes the same thing. That's what I did here, wiring the jack to ensure continuity on each side, and using outdoor UV-resistant cables for durability. The red tape flag tells me which pair is which.
I then widened the hole where the power conduit to the outdoor lighting exited and put a little flex tubing there to thread the two runs out of. The green wire is the grounding wire the cable wiring is using, which is separate from the grounding wire in the phone lines' grounding clamp.
The other side of the cable run in the server room terminated at the T1 router. Old T1 installations had a separate box called the CSU/DSU (channel service unit/data service unit) which was basically the modem. It took the T1 signal from the telco and emitted V.35 over big fat connectors with long pins looking like honeycombs that most routers of the time would accept directly. V.35 is very low level, effectively OSI layer 1, and what the actual data looked like varied depending on the setup (usually this was frame relay, sometimes PPP, rarely SLIP). This situation is analogous to old Ethernet implementations where the MAU and PHY were separate devices, connected by AUI.

Eventually this became similarly inconvenient and specialized T1 routers emerged that took one or more RJ-45 T1 connections and presented a WAN Ethernet interface to devices on the other end, consolidating the CSU/DSU and the router into a single unit. This particular router is a Samsung Ubigate iBG-1000 (specifically the iBG-1000A), the littlest of the Ubigate line circa 2008. Although the Ubigate is absolutely capable of frame relay and many other such protocols, my connection to the vendor (via the telco central office) was over PPP.

Despite being the family runt the iBG-1000 is hardly Best Buy consumer crap. It can bond up to four T1 lines for an aggregate of 6.144 Mbit/s, which at the time of installation was still respectable bandwidth, and was fully configurable over serial (two ports, one as a console and one auxiliary, which could be connected to a modem) and Ethernet. The Ubigate line as a whole, however, is long obsolete; Samsung hasn't sold them in years and last time a vendor tech was out at the house he mentioned the company had moved to AdTran T1 routers, though he liked them less. The vendor never asked for the equipment back and I never asked to send it to them because I suspect they don't really want it anyway.

So here's another computing story that also starts with a box. Let's open it up.

First off we immediately see an SD card and a 512MB SO-DIMM of DDR2 SDRAM at the bottom. There are several large chips here; let's try identifying them. The Lattice LCMXO1200C chip in the top middle is a generic 1200 LUT FPGA. The PMC chip above it is a COMET™ QUAD PM4354, a four-channel all-singing, all-dancing T1/J1/E1 transceiver and framer; it even independently handles alarm status without having to involve the main CPU.

The big one at the top right was harder to identify. This is a Marvell GT-96124 and it seems like it's doing a lot. I couldn't find a datasheet for that specific IC, but the GT tag is a hint it was a design from Galileo Technology, an Israeli fabless semiconductor design firm Marvell bought in 2000, and after a little digging I was able to find a datasheet for the likely related GT-96132. Does this bus diagram look familiar?

There's our four T1 ports (via the quad T1 transceiver) and our two Ethernet ports, plus two serial ports and PCI (not apparently used here) and SDRAM controllers. This is obviously an all-in-one chipset to support the CPU and drive other onboard peripherals.

And where is the CPU? Why, that's the Freescale chip underneath it, an MPC8347EA (here labeled MPC8347EVVAGDB, indicating a 400MHz part [AG] with crypto acceleration [E] on a 266MHz bus [D] mounted via a lead-free tape ball-grid array package [VV]). The 8347EA is part of the PowerQUICC II Pro family introduced in 2004 intended as systems-on-a-chip for networking applications. This chip is a 32-bit PowerPC SoC using the e300 core, more or less a hopped-up PowerPC 603e, with its own two-port Ethernet and SDRAM controllers and a single UART (it also has on-board support for two USB ports, PCI, I2C and SPI).

Given the layout of the board my guess is the CPU is actually handling the SD card (via SPI), the Ethernet ports and the SDRAM itself, and probably the console port as well so that there's direct access to its bootloader. The Marvell chip likely services the auxiliary port and maintains the HDLC interface to the T1 transceiver.

If you've read my blogs for any length of time you knew my eyes lit up the minute I realized this was a PowerPC system. Now I had to make it all mine.

The box the vendor gave me is a complete kit and I got the router new with the protective stickywrap still on it. Conveniently this included the necessary RJ-45 to DE-9 (DB-9) serial cable, labelled IBG-CCOM10 (EQ39-00004A); if you don't have or can't find this cable, the pinout is in the manual. You'll need the serial cable to get into the bootloader, which we'll have to do to get control over it (stay tuned). Connect this cable to your serial port and set your terminal program to 9600bps, 8N1. Oddly, although the cable has hardware flow control signals, I had to turn hardware flow control off in minicom to get it to work properly.
We've got tunes and we've got signal. Main screen port turn on. The first stage is the bootloader.

Copyright (c) 2008-2010 Samsung Electronics, Co.

Samsung Ubigate iBG1000 (Freescale MPC8347EA, 512M memory)
BootROM
  Version: 1.4.0
  NORMAL Boot, Compiled Jan  4 2011, 17:17:55 by build
  Gold boot update baseline ver: 1.0.1
Boot flash(Normal boot ver: 1.4.0, Gold boot ver: 1.4.0)
Mainboard CPLD ver: 7.0


Press any key to stop auto-boot...
 0
auto-booting...

[BOOTM] Loading file(/cf0/factory_default)...

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<  
!!Warning!!
 Factory default 'Boot file name' was set 'factory_default'.
 System will load a default SNOS(the latest version) from Compact Flash.
 For the proper operation, user should change exact 'Boot file name'.
Compact Flash Device: CF0, Filename: /cf0/iBG1000_Advanced_1.5.0.3.Z
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 


[BOOTM] Loading package...
 Loading [100]
[BOOTM] System image loading done
 Loading [100]
[BOOTM] Bootrom image loading done
[BOOTM] Checksum validation is checked.[OK]
[BOOTM] Package validation pass
[BOOTM] Bootrom checksum validation is checked.[OK]
[BOOTM] Checking boot image for auto update...
[BOOTM] Done : No need to update Normal boot image
[BOOTM] System image validation pass
[BOOTM] System image is decompressing
[BOOTM] System image loading on the start address

"Loading on the one"

Despite the fact the storage is really an SD card, other Ubigate hardware actually had a Compact Flash card on board and the nomenclature apparently carried over.

The manual doesn't define what SNOS is, but an educated guess would say it's Samsung Network Operating System. That loads now.

==================================================
             SAMSUNG UBIGATE SNOS
==================================================
[ATA] Starting ATA driver(S) 2.0
This System is optimized for Evil Empire T1 Vendor site.
! Set to console terminal baud rate(9600)

SAMSUNG ELECTRONICS CO., LTD.
Suwon-si, Gyeonggi-do 442-600, Korea
Copyright (c) 2008-2010
Samsung SNOS Software, iBG1000 Software

Freescale MPC8347EA processor, 512 MB memory
-> 
[INIT] Performing Level-1 Init. SUCCESS
[STDDBG] tDbgTask (0x1ffbb548) is spawned ...
[CPUMON] tCpUMon (0x1ffb8c28)  is spawned ...
[SYSLOG] tSyslogTask (0x1ffb4b98) is spawned ...
[EVMGR] tEvMgr (0x1ffa8638) is spawned ...
[INIT] Performing Level-2 Init. SUCCESS
[COMNBUF]Buffer Mgmt: Allocating default # of buffers
[COMNBUF]totalBuf = 0x03bc0      tBufs = 0x1280  xtraBufs 0x0
[COMNBUF]Calculated Total-Buffers = 15296
[COMNBUF]SYS_COMN_BUF_ALLOC_MAX= 29058
[COMNBUF]Allocating 1952 size 15296 packet buffers
[AAA] User initialization done
[HTTPD] Initialization done
[CFGMGR] attached the link-restoral timer from Config-Mgr task. 
[L2Task] Started diff-Delay-timer in context of L2-task 
[TELCO] Call spawnAllTasks.
[TELCO] Threshold task spawned. 
[TELCO] Statistics task spawned. 
[TELCO] Log task spawned. 
[TELCO] Evlogtask msgQ = 0x1805a6e0
[TELCO] Test task spawned. 
[TELCO] Update task spawned. 
[TELCO] SpawnAllTasks done.
[TELCO] Card up message sent
[TELCO] Telco & TE1 Driver Init Done
[SECURITY] Security Function Initialization started...
[SECURITY] Security Function Initialization completed...
[INIT] Performing Level-3 Init. SUCCESS
[TSEC0] pTbdBase = 0x3422000
[TSEC0] pRbdBase = 0x3422400
[TSEC1] pTbdBase = 0x3422800
[TSEC1] pRbdBase = 0x3422c00
[SNMP] AGENT INIT ENDED.
[WIF] Init Done successfully!
[INIT] Forwarding Engine Init. SUCCESS
[QOS] QoS Initialization completed
[PBR] Local PBR Init is done successfully
[VoiceDrvs] Skip open FXS/FXO device driver config file 
[INIT] Performing Level-4 Init. SUCCESS
[ISDN]: LAYER1 Register ... status(0)
[ISDN]: LAYER2 Register ... status(0)
[ISDN]: LAYER3 Register ... status(0)
[ISDN]: LAYER4 Register ... status(0)
[VQM] Initialization completed

[LICENSE] No license key is present.
[SNMP] COMMON-SUB-AGENT : MIB REGISTER ENDED 

[VoiceInit] Voice Initialization Start...
[VoiceInit] NO VoIP DSP Card Found !!!
[TASKMON] taskMon (0x1fbacc00) is spawned ... Start task monitoring
[INIT] Performing Level-5 Init. SUCCESS
[VoiceInit] Voice memory partition 0xfac72f8 - 0x12dc72f8
[VoiceInit] Voice memory partition created, size=51M, id=0xfac72a0
[VoiceInit] Voice task configuration memory initialized
[VoiceInit] Voice task shared memory initialized
[VoiceInit] Loose Timer Task initialized
[VoiceInit] Provisioning Data Handler Task initialized
[VoiceInit] Resource Monitor and Maintenance Data Handler Task initialized
[VoiceInit] Call Statics Data Handler Task initialized
[VoiceInit] SIP Entity SBC Server Task initialized
[VoiceInit] Snmp Voice Subagent Agent initialized
configuring from file '/cf0/system.cfg' ... 
 Loading system.cfg file [100%]     


Ubigate iBG1000 Software, Advanced SNOS (Basic + Security/Voice)
Version: 1.5.0 ("/home1/build/release/ucrel_1.5.0.3/src")
Complied Apr 25 2011, 11:31:19 by build

#-----------------------------------------------------------------------
# SAMSUNG ELECTRONICS CO., LTD. Login
#-----------------------------------------------------------------------

login: 

Interesting that the operating system supports ISDN and voice T1 as well (useless on the 1000, however, because there's no way to connect any telephone hardware to it).

The default login and password are samsung (both). Naturally that didn't work, because that would have been too easy. But we can see it's loading from a configuration file, and that configuration file is on the SD card (/cf0/system.cfg), so maybe we can bust in by messing around with that. I removed the tamper tape from the SD card, pulled it out — an off-the-shelf Transcend 2GB card — set the lock tab and put it in the Talos II's card reader.

Before going any further I imaged the card and put it away for safekeeping, and copied the image to a new 2GB card I got out of my parts drawer. Unsurprisingly it's just a FAT16 filesystem, with an unallocated partition most likely left for wear-leveling purposes. Most of the card is empty space:

% ls -lR
.:
total 17056
drwxr-xr-x. 2 spectre spectre    32768 Jan  6  2000  CDR
-rw-r--r--. 1 spectre spectre        8 Jan  6  2000  Certificates.dat
-rw-r--r--. 1 spectre spectre    51200 Jan  6  2000  command.log
-rw-r--r--. 1 spectre spectre 16926715 Sep 27  2017  iBG1000_Advanced_1.5.0.3.Z
-rw-r--r--. 1 spectre spectre       28 Jan  6  2000  Keys.dat
-rw-r--r--. 1 spectre spectre     3207 Jan  6  2000  oldsystem.cfg
drwxr-xr-x. 2 spectre spectre    32768 Jan  6  2000  profiles
drwxr-xr-x. 2 spectre spectre    32768 Jan  6  2000  pwp
-rw-r--r--. 1 spectre spectre      672 Jan  6  2000  shdsakey
-rw-r--r--. 1 spectre spectre      610 Jan  6  2000  shdsakey.pub
-rw-r--r--. 1 spectre spectre      887 Jan  6  2000  shrsakey
-rw-r--r--. 1 spectre spectre      230 Jan  6  2000  shrsakey.pub
-rw-r--r--. 1 spectre spectre     3216 Jan  6  2000  system.cfg
drwxr-xr-x. 2 spectre spectre    32768 Sep 28  2017 'System Volume Information'
-rw-r--r--. 1 spectre spectre     6816 Jan  6  2000  __userext__.dat
drwxr-xr-x. 2 spectre spectre    32768 Jan  6  2000  vqm

./CDR:
total 0
-rw-r--r--. 1 spectre spectre 0 Jan  6  2000 CDR_VQM.csv

./profiles:
total 0

./pwp:
total 0

'./System Volume Information':
total 32
-rw-r--r--. 1 spectre spectre 76 Sep 28  2017 IndexerVolumeGuid

./vqm:
total 0

The system configuration file is too long to print here, but suffice it to say that there was no user authentication information in it. Although the SSH1 server was enabled in that file and replacing the RSA and DSA public keys might get me in through there, I couldn't even determine exactly what IP it was listening on (the config file didn't say) and whether it could be accessed outward from the inside ports. I poked at the other files but didn't see any obvious usernames and passwords there either.

As a first attempt I deleted the configuration file and rebooted it that way to see if I was wrong. The machine made lots of complaints about that, but the login prompt remained impervious. Since we're almost certainly going to rework the configuration anyway I didn't bother restoring it.

The manual was also (probably intentionally) opaque about how to change a "forgotten password" and there was no obvious factory reset switch. The only other permanent storage was the on-board flash where the bootloader lived, so that was my next port of call. Press a key in the autoboot sequence and it will drop to the bootloader prompt.

Copyright (c) 2008-2010 Samsung Electronics, Co.

Samsung Ubigate iBG1000 (Freescale MPC8347EA, 512M memory)
BootROM
  Version: 1.4.0
  NORMAL Boot, Compiled Jan  4 2011, 17:17:55 by build
  Gold boot update baseline ver: 1.0.1
Boot flash(Normal boot ver: 1.4.0, Gold boot ver: 1.4.0)
Mainboard CPLD ver: 7.0


Press any key to stop auto-boot...
 3
[BOOT]:

No authentication required, so let's see what we've got for commands. There's occasional Engrish here; I cut and pasted this directly out of minicom.

[BOOT]: h
 ?                     - print this list
 @                     - boot (load and go)
 #                     - boot in DIAG. mode
 p                     - print boot params
 c                     - change boot params
 v                     - print boot logo with version
 o                     - show bootrom header information
 L                     - show file list in Flash 
 F                     - format the Flash 
 A                     - show simple PCI device scan result
 r                     - show current boot image(Golden/Normal)
 w                     - print status of watchdog timer
 D                     - cold reset
 u                     - change User ID and PW default set
 U                     - reset to factory 
 W                     - Set Watch Dog Time. 
 J                     - Change Vender Information.
 t                     - Run HDLC Diagnostic
 i                     - show the current console baudrate
 ! [value]             - set the console baudrate
 Checksum enable:      
  0  - check image checksum 
  1  - skip checking image checksum 
 Show header enable:   
  0  - disable showing image header contents 
  1  - enable  showing image header contents 
 Save bootrom image:   
  0  - autoupdate NORMAL boot area rom image by checking version 
  1  - save boot image on the NORMAL flash area 
  3  - No bootrom update 
 Boot image redundancy: 
  NORMAL boot image - Default boot image saved on the NORMAL boot area 
  GOLDEN boot image - Backup boot image saved on the GOLDEN boot area 
                      GOLDEN image area should not be corrupted 

Oh, I do like our chances with the u option (not U, just in case I wreck something else in the process). Let's try that.

[BOOT]: u


WARNING: All system user information will be lost!!!
1)All user account information will be reset.
2)All user lock table information will be lost.
3)The default user ID and password will be set.

Are you sure (Y or N)? y

User information is set to factory-reset.

Default User & Password can be availabe.

I love it when the default user and password are availabe. Let's bring up SNOS again and see if the default username and password are now valid.

[BOOT]: @
[BOOTM] Loading file(/cf0/factory_default)...

[...]

Ubigate iBG1000 Software, Advanced SNOS (Basic + Security/Voice)
Version: 1.5.0 ("/home1/build/release/ucrel_1.5.0.3/src")
Complied Apr 25 2011, 11:31:19 by build

 *Jan 01,2000,01:08:44 #PLATFORMn-informational: Boot up procedure completed
#-----------------------------------------------------------------------
# SAMSUNG ELECTRONICS CO., LTD. Login
#-----------------------------------------------------------------------

login: samsung
password: 

                            SAMSUNG ELECTRONICS CO., LTD. CLI
evil_empire_T1_router#  *Jan 01,2000,01:08:49 #AAAn-notification: samsung logged
?

      clear                      access clear commands
      configure                  configure from ( flash / network / terminal )
      debug                      accesses debug commands
      exit                       Exit from this session
      file                       access file commands
      password                   Change the user password 
      ping                       invoke ping
      reboot                     reboot the system
      save                       save configuration to ( local / network )
      show                       access show commands
      ssh                        OpenSSH SSH client (remote login)
      telnet                     open a telnet connection
      test                       access test commands
      trace                      trace route to destination address or host name

      <cr>                       
evil_empire_T1_router# show version

     Model: Ubigate iBG1000
      SNOS: Advanced SNOS (Basic + Security/Voice)
   Version: 1.5.0
  Bld date: Apr 25 2011, 11:31:19 (by build)
  Bld Path: "/home1/build/release/ucrel_1.5.0.3/src"
SNOS package boot version: 1.4.0 (golden boot base version: 1.0.1)
Flash normal boot version: 1.4.0 (golden boot version: 1.4.0)
Flash boot version check : No action required

Chassis info:

Slot/SubSlot  Card-Type  Status  FPGA-Rev  CPLD-Rev
---------------------------------------------------
   0/-        CBU         NORMAL    N/A      0x07  

In like Flynn in the MCP's business. Dig the Y2K-proof default date and time.

Now that we're at a command prompt, let's talk about SNOS. Every time I write something gently derogatory about any kind of tech there's always someone to pop up and say I'm wrong, so I look forward to hearing from the three people who have used SNOS and like it in the comments. For everyone else, if you've used Cisco IOS on Cisco router hardware, SNOS is sort of the shabbier bootleg grey copyright cousin, like those Mackey Moose cartoons we know get sold on DVD-Rs from truck liftgates: very similar, as exact as it can be without being a blatant ripoff, but with enough subtle and sometimes obvious differences to land squarely in the network engineer uncanny valley.

I should also note that vendors could customize SNOS with their own add-ins or remove modules that were irrelevant to their business line. This is what my vendor seems to have done; their name, censored so I don't get in trouble, even appears in the boot messages. It's very possible there is differing functionality between this build of SNOS and the stock version, so just keep this in mind if you're using these steps for the Ubigate you found at the electronics recycler.

I'll discuss more about the architecture of SNOS when we look at monkeypatching it. For now, let's get this up on the network so I don't need to keep it plugged into the serial port. SNOS, like IOS, is a modal CLI: certain sets of commands are only allowed in certain modules which you must explicitly enter and exit, and the kernel keeps track which module a user is presently in so that two people logged into the device will be (largely) prevented from issuing dueling commands. That should give you enough context to understand the next set of entries.

evil_empire_T1_router# configure terminal
evil_empire_T1_router/configure# hostname george
george/configure# keymap Delete delete-left-char
george/configure# system logging
george/configure/system/logging# no logging_on
george/configure/system/logging# exit
george/configure# interface ethernet 0/0
george/configure/interface/ethernet (0/0)# ip address X.X.X.X Y.Y.Y.Y
george/configure/interface/ethernet (0/0)# exit ethernet
george/configure# ip domain-lookup
george/configure# ip pname_server Z.Z.Z.Z
george/configure# ip domain_name floodgap.com
george/configure# sntp server ntp.floodgap.com
george/configure# telnet_server
george/configure# ftp_server
george/configure# ip http server
george/configure# exit
george# save local

WARNING :
Do not reboot during this process.
Saving current system configuration.
Please wait... (up to a minute)
2382 bytes written
Done.
george# 

I named it George after the sales exec who sold me the line back in 2011; I have no idea if he's even still with the company in its current form. Although the IP addresses are censored so you stay out of my network backyard, you little script kiddie, these commands should be relatively self-explanatory.

In detail, I first enter the configuration module with configure terminal (versus configure file or configure network), which says I'll enter configuration information manually and locks the configuration session to me so another logged in administrator can't change it simultaneously. I then set the hostname, set the delete key to be sane, turned off logging (or it gets really spammy with the T1 ports disconnected), set ethernet port 0 to the correct address (X) and netmask (Y), enabled the DNS client, set the primary name server, set the domain name, set the NTP server (yay! clock!), and then enabled the Telnet, FTP and Web servers. (I left off the SSH server since it's not much good to me in its current form being merely SSH1, and george is currently only connected to the internal non-routable network anyway. There's an HTTPS server also, but it too is almost certainly fatally deficient by modern cryptographic standards.)

The final exit from configuration mode commits the changes to the current working state; save local then writes a clean, new /cf0/system.cfg with those settings to the card for next bootup. Hello, george, welcome to Floodgap.

I rebooted it (a power cycle will do) to verify it all stuck and logged in over Telnet this time, which is much faster than 9600bps serial. Obnoxiously the only one of these settings that does not properly persist is the delete key setting; it's properly written to the configuration file, but doesn't seem to be honoured by the Telnet server as the default. Either type it manually when you log in or use something like expect on top of telnet to remap ^? to ^H.

What else can we get into? There aren't many commands otherwise because this is supposed to be a router, darn it, and no part of the visible filesystem obviously contains those commands (more to say about that later), so I couldn't see a way to upload executables and have SNOS run them. The SSH1 client, as expected, failed to connect to any of the servers locally since they're all SSH2. The file module lets you manipulate the SD card from within the CLI interface, ping, trace and telnet do what you'd expect, and the test module just lets you run tests on the T1 line which, of course, doesn't do me any good anymore.

Let's start with the FTP server. Interestingly, the FTP credentials are separately maintained.

george# show ftp
FTP Setting:
--------------------------------------
      FTP Server:     Enabled
        (File list format: iBG private)

Allowed FTP Client:
--------------------------------------
        Username:       admin
        Password:       admin

Much like the Telnet server, the FTP server is a decent, unadorned, standards-compliant FTP server. Files go to and from the SD card. You could easily use it as a drop-box in this configuration.

% ftp george
Connected to george (X.X.X.X).
220 FTP server ready.
Name (george:spectre): admin
331 Password required for admin.
Password:
230 User admin logged in.
Remote system type is iBG:.
ftp> quote HELP
214- The following commands are recognized (* =>'s unimplemented).
   USER EPRT STRU REST CWD SYST XMKD CDUP 
   PASS PASV MODE RNFR XCWD STAT RMD XCUP 
   QUIT LPSV RETR RNTO LIST HELP XRMD STOU 
   PORT EPSV STOR ABOR NLST NOOP PWD SIZE 
   LPRT TYPE APPE DELE SITE MKD XPWD MDTM 
214 Direct comments to ftp-bugs@george.
ftp> quote SITE HELP
214- The following SITE commands are recognized (* =>'s unimplemented).
   IDLE HELP 
214 Direct comments to ftp-bugs@george.
ftp> quote SYST
215 iBG: Version: 1.5.0.3
ftp> quit
221 Goodbye.

That's pretty much all I have to say about the FTP server and the Telnet server.

The web server, on the other hand, is a stranger beast. By default, you get this error page.

% curl -v http://george/
*   Trying X.X.X.X:80...
* Connected to george (X.X.X.X) port 80 (#0)
> GET / HTTP/1.1
> Host: george
> User-Agent: curl/Q.QQ.Q
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Server: SNOS-HTTP-SERVER
< Connection: close
< Content-Type: text/html
< 
<H1>SNOS-HTTP-SERVER Error Report:</H1><HR>
<H2>Server Error: 404 Not Found</H2><HR>
<H2>File not found</H2><HR>
<H2>/start.html</H2><HR>
<H3> SNOS-HTTP-SERVER </H3><HR>
* Closing connection 0
Fair enough: we know the FTP server works, so I wrote up my customary tiny "hello world" page and uploaded it to start.html on the CF SD card.

% curl http://george/
Mortimer, we're back!
Now this leads to an interesting question. Is everything accessible on the card? Let's try for the brass ring right now:

% curl http://george/system.cfg
<H1>SNOS-HTTP-SERVER Error Report:</H1><HR>
<H2>Server Error: 404 Not Found</H2><HR>
<H2>File not found</H2><HR>
<H2>/system.cfg</H2><HR>
<H3> SNOS-HTTP-SERVER </H3><HR>

That's a relief, but it also means it won't serve arbitrary content. The exact filenames it will serve must be in the firmware. Let's look at that now.

Going back to our list of files on the card, iBG1000_Advanced_1.5.0.3.Z is obviously the boot image. At nearly 17MB pretty much everything is in it. You would think that the .Z suffix on the end implies it was shrunk with compress, but both uncompress and gunzip object to the file and won't process it. On the other hand, running strings on it yields nothing but garbage, so it is almost certainly compressed with something. The total length of this file is 16,926,715 bytes; remember this for the offsets we'll be using.

Let's start off by seeing what binwalk makes of it. It immediately determines the firmware is gzip-compressed with a separate 52-byte header, which explains gunzip's objections:

% binwalk iBG1000_Advanced_1.5.0.3.Z 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
52            0x34            gzip compressed data, from Unix, last modified: 2011-04-25 02:32:24
16048242      0xF4E072        MySQL MISAM compressed data file Version 5
16307655      0xF8D5C7        Copyright string: "Copyright 1984-2006 Wind River Systems, Inc.8`"
16326587      0xF91FBB        VxWorks operating system version "5.5.1" , compiled: "Jan  4 2011, 17:17:58"
16327012      0xF92164        Zlib compressed data, default compression

It also found two interesting strings indicating that SNOS is built on VxWorks 5.5.1, which is not at all surprising; it's a very common kernel for real-time applications like this, especially on PowerPC.

However, it would still take some trial and error to determine any subcomponents in the file as well as the exact meaning of the header, and I'm quite sure there's not really an ISAM database in it (probably). Fortunately it turns out the bootloader itself can tell us what's there. While I'm at it I'm going to hard-set the boot filename so we know exactly what's getting loaded.

[BOOT]: c
Boot dev  [ftp,active-ftp,cf0]: cf0 
Boot file name            : factory_default iBG1000_Advanced_1.5.0.3.Z
Server name               : host 
Server IP address         : 192.168.1.10 
My IP address             : 192.168.1.11 
My subnet mask            : 
Gateway IP address        : 
User name                 : demo 
Password                  : 
Checksum enable     [0:Check  ,1:Skip  ]: 0 
Show header enable  [0:Disalbe,1:Enable]: 0 1
Save bootrom image  [0:AutoUpdate,1:NormalBTupd,3:NoUpd]: 0 
Golden boot autosync [0: disable, 1: enable]: 0 
Save Diag Flag  [0:Normal mode,1:Diag mode]: 0 
Rollback SNOS [/cf0/file,null]: 

Other than setting the filename, the major change I made here was to enable showing the header on the file as it loads, i.e., that 52 bytes at the beginning that binwalk identified. We'll now boot the image and see what it displays.

[BOOT]: @

[BOOTM] Loading file(/cf0/iBG1000_Advanced_1.5.0.3.Z)...

* File Header *
version      : 01.05.00.03
no.of images : 2
packaging    : Advanced
model        : 1000
size         : 16926715 (0x10247fb)
checksum     : 0xa8a6d921
hdr checksum : 0x897a394f

* Image Header [0] *
version      : 01.05.00.03
type         : System
size         : 16307591 (0xf8d587)
checksum     : 0x8f385188

* Image Header [1] *
version      : 01.04.00.00
type         : Booting Rom
size         : 619072 (0x97240)
checksum     : 0x1b81b7d6

[BOOTM] Loading package...

This indicates there are exactly three pieces to the file, namely the header, the image and the boot ROM (the on-board flash). Let's look at the first 52 bytes in a hex editor.

00000000  01 05 00 03 
          02 
          01 
          03 e8  
          01 02 47 fb 
          a8 a6 d9 21  |..........G....!|
00000010  89 7a 39 4f 

          01 05 00 03 
          02 52 00 00 
          00 f8 d5 87  |.z9O.....R......|
00000020  8f 38 51 88 

          01 04 00 00  
          01 00 00 00 
          00 09 72 40  |.8Q...........r@|
00000030  1b 81 b7 d6 

This is 32-bit PowerPC, so all values are big-endian in the fashion G-d Himself intended humanity to think. In order, we see the version, number of images, the packaging type (1), the value 1000 (for the model) as a 16-bit short, and as 32-bit ints the total file size and the checksum. The two other entries should be similarly self-explanatory except for their type codes (0x02520000 and 0x01000000), which look like otherwise opaque 32-bit ints also. 52 + 16307591 + 619072 = 16926715, meaning all bytes are accounted for in the header, so we now have enough information to break the file apart.

% cat >break.pl 
#!/usr/bin/perl


read(STDIN, $buf, 52); # header
read(STDIN, $buf, 16307591); # sysimg
open(X, ">sys.gz") && print(X $buf) && close(X);
read(STDIN, $buf, 619072); # bootrom
open(X, ">boot.rom") && print(X $buf) && close(X);
^D
% perl break.pl < iBG1000_Advanced_1.5.0.3.Z
% ls -l
total 33072
-rw-rw-r--. 1 spectre spectre   619072 May 27 21:51 boot.rom
-rw-rw-r--. 1 spectre spectre      226 May 27 21:51 break.pl
-rw-rw-r--. 1 spectre spectre 16926715 May 27 21:45 iBG1000_Advanced_1.5.0.3.Z
-rw-rw-r--. 1 spectre spectre 16307591 May 27 21:51 sys.gz

It's the SNOS image we're most interested in, since we have no need right now to futz around with the bootloader. binwalk says it's gzipped, so let's try that.

% gunzip -c < sys.gz > sys
% file sys
sys: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), statically linked, not stripped

Ta-daa. It's actually one huge single ELF executable, probably loaded into real memory even though the MPC8347EA does have an MMU. And strings yields a lot of interesting things we might talk about in a future entry, including the list of filenames that the webserver will serve. Note that .htm and .html are not treated synonymously:

/start.htm         
/start.html        
/login.htm         
/login.html
/errlgn.htm        
/errlogin.htm

And that's it. CGI? Hahahaha.

For our last stupid pet trick, let's monkeypatch the firmware. Being one big executable all commands are embedded within it, and it should be possible to compile in new, arbitrary functionality, link it into the monolith and then modify some victim code somewhere else to call it. For the moment I don't know enough about VxWorks to do that currently and I probably should find something more productive to do with the rest of the weekend, so as proof of pwnership we'll settle for modifying strings in place. SAMSUNG and SPECTRE are both seven letters long, so I loaded the blob into hexer and changed it everywhere I saw it in the file.

The next step is to figure out how the checksums work to stream this back in. Yes, you did indeed see that the bootloader can be told to ignore checksums, but it's always better to maintain safeguards when you're this close to the metal; I wouldn't want to brick it on the eve I finally got it open. Running the pieces through crc32 did not match the checksums in the header, but by dumb luck cksum did, once you convert its hashes to hexadecimal. You should be able to find these values in the header, proving they check out.

% cksum sys.gz
2402832776 16307591 sys.gz
% cksum boot.rom
461486038 619072 boot.rom
% perl -e 'printf("%08x %08x\n", 2402832776, 461486038)'
8f385188 1b81b7d6
% cat sys.gz boot.rom | cksum | perl -ae 'printf("%08x\n", $F[0])'
a8a6d921

How about the checksum for the header itself? This is a tougher call because the header has its own checksum embedded in it, a classic chicken-and-egg problem. Some schemes require a specific placeholder value to be placed where the checksum would go and you checksum the data like that. It may be possible to brute force the starting value since we know the desired result, but a good initial guess is always to put zeroes where the checksum would go:

% perl -e 'undef $/;$_=<STDIN>;1while(s/[\r\l\n\s]+//);print(pack("H*",$_))' > x.out
  01 05 00 03 02 01 03 e8  01 02 47 fb a8 a6 d9 21  
00 00 00 00 01 05 00 03  02 52 00 00 00 f8 d5 87  
  8f 38 51 88 01 04 00 00  01 00 00 00 00 09 72 40  
  1b 81 b7 d6 
% cksum x.out
2306488655 52 x.out
% perl -e 'printf("%08x\n",2306488655)'
897a394f

I'd rather be lucky than good as that is indeed the correct method. Now we can assemble the replacement firmware: compute the checksum and length of the new compressed SNOS image, compute the checksum of the compressed SNOS image concatenated with the boot ROM (we didn't change the boot ROM, so that has the same length and checksum), create the header checksum, create the header with the checksum, and finally concatenate everything together and upload it to the SD card. (Whew!)

% gzip -c < sys > sys.gz
% cksum sys.gz | perl -ae 'printf("%08x %08x %d\n", $F[0], $F[1], $F[1])'
8e6adee9 00f8d584 16307588
% perl -e 'printf("%08x\n", 16307588+52+619072)'
010247f8
% cat sys.gz boot.rom | cksum | perl -ae 'printf("%08x\n", $F[0])'
4e8012dc
% perl -e 'undef $/;$_=<STDIN>;1while(s/[\r\l\n\s]+//);print(pack("H*",$_))' > x.out
01 05 00 03
02
01
03 e8
010247f8
4e8012dc
00000000
01 05 00 03
02 52 00 00
00f8d584
8e6adee9
01 04 00 00
01 00 00 00
00 09 72 40
1b 81 b7 d6
% cksum x.out | perl -ae 'printf("%08x\n", $F[0])'
57f4f58e
% perl -e 'undef $/;$_=<STDIN>;1while(s/[\r\l\n\s]+//);print(pack("H*",$_))' > x.out
01 05 00 03
02
01
03 e8
010247f8
4e8012dc
57f4f58e
01 05 00 03
02 52 00 00
00f8d584
8e6adee9
01 04 00 00
01 00 00 00
00 09 72 40
1b 81 b7 d6
% ls -l x.out
-rw-rw-r--. 1 spectre spectre 52 May 27 22:21 x.out
% cat x.out sys.gz boot.rom > iBG1000_Advanced_1.5.0.3.Z

Rebooting!

Copyright (c) 2008-2010 Samsung Electronics, Co.

Samsung Ubigate iBG1000 (Freescale MPC8347EA, 512M memory)
BootROM
  Version: 1.4.0
  NORMAL Boot, Compiled Jan  4 2011, 17:17:55 by build
  Gold boot update baseline ver: 1.0.1
Boot flash(Normal boot ver: 1.4.0, Gold boot ver: 1.4.0)
Mainboard CPLD ver: 7.0

$
Press any key to stop auto-boot...
 0
auto-booting...

[BOOTM] Loading file(/cf0/iBG1000_Advanced_1.5.0.3.Z)...

* File Header *
version      : 01.05.00.03
no.of images : 2
packaging    : Advanced
model        : 1000
size         : 16926712 (0x10247f8)
checksum     : 0x4e8012dc
hdr checksum : 0x57f4f58e

* Image Header [0] *
version      : 01.05.00.03
type         : System
size         : 16307588 (0xf8d584)
checksum     : 0x8e6adee9

* Image Header [1] *
version      : 01.04.00.00
type         : Booting Rom
size         : 619072 (0x97240)
checksum     : 0x1b81b7d6

[BOOTM] Loading package...
 Loading [100]
[BOOTM] System image loading done
 Loading [100]
[BOOTM] Bootrom image loading done
[BOOTM] Package validation pass
[BOOTM] Bootrom checksum validation is checked.[OK]
[BOOTM] Checking boot image for auto update...
[BOOTM] Done : No need to update Normal boot image
[BOOTM] System image validation pass
[BOOTM] System image is decompressing
[BOOTM] System image loading on the start address

==================================================
             SPECTRE UBIGATE SNOS
==================================================

For all those in-place changes this was the only spot where I saw SPECTRE actually visibly appear, but this is good enough to prove that the change took, the checksums verified and the new binary was accepted. If I have another lazy weekend I might look at the symbols in that executable, write up some headers and try to link in a hello-world module without blowing anything up. Linux? NetBSD? Well, it does have an MMU, kinda-sorta mass storage and sufficient RAM, so it should be possible.

In the meantime, the least the vendor could do after 10 years of fees and a precipitous exit from the market is not ask for the hardware back now after I finally managed to get into it. I mean, c'mon, at least leave me my boxy toys for such sunny holiday weekends. To do otherwise would not be a great ending to these two nerdy stories.