Refurb weekend: Cobalt RaQ 2

In the post on our recently resurrected fork of Dreamcast Linux, I mentioned the NetBSD NFS server providing basically all of its persistent storage. A few days into the development work I started hearing a weird whine coming from the server room and sure enough the NFS server had a bad fan — in fact, the only fan cooling the entire 1U system. That means it's time ... for another Refurb Weekend!

As most readers of this blog will attest, we don't go for that commodity x86 crap around here: this is actually a MIPS box, a Cobalt RaQ 2, based on a 250MHz Quantum Effect Design RM5231 running NetBSD/cobalt. Its name is oulu.

Cobalt Networks were one of the early pioneers in network appliance hardware and produced some of the first turn-key webserver boxes you could buy, founded in 1996 as Cobalt Microserver. Cobalt boxes are immediately identifiable from their distinctive deep blue plastic bezels starting with the 1998 Cobalt Qube 2700. The Qube used a 150MHz QED RM5230; these CPUs are part of QED's R5000 family and we'll talk about their architecture a bit later. They came with 2.1GB hard disks with later larger options, 10Mbit Ethernet, 16MB of RAM standard with up to 256MB supported, and a "console" consisting of a backlit rear-mounted 2-line LCD and control buttons (on later machines, but not the original 2700, a serial port provided an actual console if you held down a button during startup). A fair number of typical configuration tasks such as setting its IP address could be done directly from the panel and the rest were intended to be done through its Perl-based web console. They were designed to run Linux from the ground up and shipped with Red Hat using a 2.0.x kernel.

However, the cube form factor, though cool, was not rack-friendly, and 10Mbit was no longer cutting edge. To serve the enterprise market Cobalt redesigned the Qube logic board with a flatter, more flexible layout, moved the LCD and button panel to the front, and installed everything in a more conventional 19" 1U form factor. This was the original Cobalt RaQ (later retconned as the RaQ 1) released later that year with the same specs, and later revisions of the Qube 2700 and the O.G. RaQ came with 10/100 Ethernet. In 1999, they upgraded both the Qube and the RaQ into the Qube 2 and RaQ 2 with the newer 250MHz QED RM5231. The Qube 2 also featured a PCI slot for expansion, and Cobalt additionally offered dual-NIC versions of the 10/100 Qube 2700 and RaQ 2 intended as caching or proxy servers; these were the CacheQube and CacheRaQ 2 respectively. Seagate OEMed the RaQ 2 as the NASRaQ, and Gateway the Qube 2 as the Gateway Micro Servers 100 and 200, with a 56K modem in the PCI slot and differing amounts of RAM and disk space.

My MIPS RaQ systems started with an original RaQ given to me by a friend (hi Bill!) who wanted to find it a good home. That system served well for a number of years but eventually burned out its power supply. At the time I was intending to repair it but there was a cheap RaQ 2 on eBay so I just grabbed that, put the old hard disk and RAM into it and kept going (and got a faster CPU in the process). Bill's RaQ is still in my repair bay, though, along with a CacheRaQ I also wound up with. All of these are the MIPS variants and not the later x86 nonsense, but we're getting ahead of ourselves.

The LEDs, from left to right, are Tx/Rx, Link, Col (for packet collision), 100M (when 100Mbit), Disk and Web. All of these are self-explanatory but the Web light is particularly interesting: it illuminates on traffic to the webserver, with specific support for it in the operating system package.

The control panel consists of a U/D/L/R cluster and Select and Exit buttons. The hole directly left of them has the password reset button behind it, which you hold in with a pin while turning the system on to enable the serial console (115200bps, 8N1).

Plate sticker on the underside. When was the last time you saw "Made in USA" on your server hardware?

Accordingly there are only a few ports on the back: the serial port, the 10/100 port and the power switch and connector (100-240 VAC, 1.0A, 40 watt max). The network lights minus the Web LED are replicated on the back along with an LED that illuminates when the operating system has halted (from the front you'd just see this on the LCD).

Let's pull the top sheet metal panel off. It's just two screws and the top lifts up and away.

There is almost as much dust in here as there is empty space to hold it. Besides the grit we only see a single logic board, a separate power supply, the front panel and the hard disk. At this point I got out the compressed air and blew out a lot of grime before we continued on.

Two ribbon cables connect to the logic board, one from the front panel routed under the hard disk, and one for the hard disk's IDE PATA cable. This was the original Quantum Fireball drive that came with Bill's RaQ.

The newly less dusty logic board itself. Nothing is heatsinked; this system doesn't pull a lot of power. The major components are the prominently marked QED CPU (with the silver label), the Galileo GT-64111 "all-in-one" MIPS system controller ASIC (containing the 32-bit PCI 2.1 bus controller [up to 66MHz], host-to-PCI bridge, DRAM controller, host-to-RAM bridge and device and DMA controllers, compatible with 32-bit NEC VR4300 [see our "MIPS ThinkPad" article], QED RM523x and IDT RV46x0 CPUs), a DEC "Tulip" 21143 Ethernet controller (here manufactured by Intel), and a VIA Technologies VT82C586A ASIC containing a PCI-to-ISA bridge and the single ATA/33 controller and real-time clock. The VT82C586A also has a keyboard controller and USB controller, but this functionality is not exposed on these machines.

The CPU itself. Quantum Effect Design was a fabless CPU design firm founded by three MIPS Computer Systems, Inc. developers with substantial support from Integrated Device Technology (IDT), who in return got a fast track to license their designs for manufacture. QED's initial goal was the development of a laptop CPU and to that end acquired a MIPS license, issuing their first chip design, the R4600 Orion, in 1993. IDT fabricated the R4600 at 650nm starting at 100MHz in 1994, where it and other licensees' chips were used in the SGI Indy and DeskStation Raptor II and Raptor Plus workstations — not to be confused with the modern Raptor POWER9 systems — with the DeskStation Raptors running a "limited version of Windows NT" (whatever that means). It had 16K L1 instruction and data caches, decent for the time, but was hurt by its lack of on-die L2 support and a weak non-pipelined FPU. Still, it was cheap enough and fast enough for lower-end applications. It was succeeded by the DSP-enhanced R4640 (used in the original WebTV) and R4650 (used in Namco System 23 games like Time Crisis II), and the R4700, a 500nm die-shrink clockbumped to 200MHz.

One of the strangest applications of the R4600 was the 1994 ShaBLAMM! NiTro-VLB, an R4600 on a VESA Local Bus card intended as an ARC-compliant Windows NT accelerator for 486 PCs. When activated it would take over as the main CPU, suppressing the 486, and used the PC's RAM as a DMA buffer for its own on-card "EDRAM" with interleaved SRAM serving as cache. It shipped with the 100MHz part, but apparently sold poorly, and as VLB was largely specific to the then-fading 80486 the intended 133MHz and 150MHz upgrades don't seem to have ever been made.

In 1996 QED developed the MIPS R5000 core, an in-order two-way superscalar design that could issue one integer instruction and one floating point instruction simultaneously, executing most integer instructions in a single clock cycle. To reduce die size and cost it did not branch-predict but rather used static branch hints provided by the compiler. It doubled the L1 caches to 32K each, had on-die L2 support up to 2MB, and at SGI's request featured a much stronger pipelined FPU. IDT fabbed it at 350nm along with NEC and NKK, and SGI offered the R5000 as an Indy upgrade at speeds ranging from 150MHz to 180MHz. It was later the low-end CPU option for the SGI O2, and derivatives were also used in many Atari and Midway arcade boards (like the original Mortal Kombat) as well as network hardware. The RM7000 CPU used in higher tiers of the SGI O2 was an R5000 with 256K L2 on-chip and support for L3. The Sony PlayStation 2 R5900 Emotion Engine was a rather later R5000 derivative with a non-standard FPU and vector extensions.

QED subsequently acquired a manufacturing license and sold bespoke R5000 derivatives of their own in 1997, the RM5230 and RM5260, produced by TSMC on a 350nm process. The RM5260 was a lower-end R5000 with 32K L1 caches, no L2, and a 64-bit 75MHz multiplexed address-data bus at up to 150MHz; the RM5230 was a further cut-down version with 16K L1 caches and a 32-bit 67MHz bus at up to 133MHz. Later MIPS WebTV systems used the RM5230 instead of the IDT R4640, while the Qube 2700 and O.G. RaQ used the second version of the RM5230 at 150MHz. The two chips were joined by the RM5270 later in 1997, which had a 100MHz 64-bit bus, restored on-chip L2 support, and ran up to 200MHz. It was pin-compatible with the R7000 and was intended as an alternative.

In 1998 QED upgraded the line to the RM5231, RM5261 and RM5271, produced at 250nm. Besides the die-shrink, the RM5231 now had 32K I/D caches like the others, bus rates were notched up to 125MHz, and the three chips now offered higher top clock speeds of 250MHz, 266MHz and 300MHz respectively. The 250MHz top-of-the-line RM5231 is the chip here and was the last MIPS CPU Cobalt used. It's no speed demon but it runs cool and fairly efficiently, appropriate for the high-end embedded markets it was targetted at.

QED also briefly produced a PowerPC design in 1996 for Motorola to manufacture using the R4600 pipeline design, the 500nm 160MHz PowerPC 603q, but it was intended for very low-end Apple systems (possibly a student-based home system and/or a Pippin-derived game console) which were never ultimately made. Initial samples were actually fabricated and ran at a promising 1.6W typical TDP, but the chip had no other interested takers and QED had no PowerPC license of their own. QED was bought by PMC-Sierra in 2000 after an expensive attempt to develop the RM9000 SOC; while completed at PMC, the chip was late to market and poorly received, and PMC-Sierra was itself bought by Microsemi in 2016.

The RaQ board has many pads for options, some of which can be populated by enterprising tinkerers (I'd have a greater risk of wrecking it if I tried, personally). A second IDE interface can be fitted so as not to run two disks on the same one, though the case only has room for two 3.5" drives. Those pads are in this orientation north and slightly east of the VIA controller. It might be more practical these days for SSDs.

The blank chip pads directly east of the VIA controller are for an optional LSI Logic NCR53C810-compatible SCSI controller. If this chip were fitted, then the easternmost set of pads would be connected to an external 50-pin SCSI port. Sadly, there does not appear to be anywhere a PCI port could be easily added like the Qube 2.

The other location with option pads is the network port area. If you look back at the overall logic board picture, you'll see unpopulated pads between the Galileo system controller and the Tulip 21143 Ethernet controller. Those pads are for a second Tulip to provide the second interface in the CacheRaQ, along with another PE-67540 choke and an Ethernet port both placed here, plus a different back cover to actually let you get at the port.

The fan was pretty choked with dust and dirt, and being its original fan probably meant its bearings were going as well. Cobalt systems were notorious for fan problems and it's impressive this one lasted as long as it did. We'll just replace it instead of cleaning it. I pulled it out and blew the fan aperture completely clear of obstructions. While the motherboard probably would be okay without much airflow, I'm not sure the same is true for any drive inside.

Although I was prepared to chop and swap the connectors, I found a replacement 30x30x6mm fan with a plug that looked like it could be wormed onto the motherboard pins, and with a little bit of gentle coercion it does.

Bolting it securely into the case so it doesn't vibrate. I also checked the voltage on the CR2032 battery while I was in there, but it's still a healthy 3.1 volts, so I left it alone. Let's get the top back on, wipe the dirt off the case and put it back in service.

The Cobalt firmware requires that the first partition on the first disk be an ext2 volume and looks there for /boot/vmlinux.gz. The firmware can only boot kernels 675K and smaller, so most modern OSes still supporting it put a small first-stage bootloader in that location to pull in the larger main kernel, like NetBSD is doing here. It can also be set to netboot from a DHCP server and an NFSv2 (not v3) server. The NetBSD Restore CD provides these pieces, or you can netboot manually with the serial console (requires hardware modification for the original Qube 2700). Similar steps exist for current Linux distros.

Starting up the main OS.

Back in service and sparkling shiny, with the new fan and newly cleaned fan path now moving a prodigious amount of air much more quietly.

Cobalt by then had developed an enthusiastic following with ISPs and resellers, but no MIPS chip in the pipeline could keep up with what customers were demanding, so the company switched to a 300MHz AMD K6-2 for the RaQ 3 and Qube 3. This generation was notable as the last of the Qubes and also for including support for scriptable ASP (Active Server Pages) using Chili!soft's ASP implementation, whom Cobalt would later buy out. The RaQ 4 ran at 450MHz and added PHP support, laying groundwork to move the web console to PHP from the old Perl-powered interface, and offered a RAID option. Although a Qube 4 was initially in development, it was ultimately cancelled.

In 2000 the RaQ XTR migrated CPUs again, this time to the Pentium III, with systems configured up to 1GHz. It offered four removable drives in the same 1U form factor but was a troublesome device with many bugs, and an initial recall required months to fix the hardware. Symantec's VelociRaptor firewall-VPN appliance was based on the 2-port XTR with an added 2-port network card (yielding four).

During this time Cobalt was acquired by Sun Microsystems in September 2000 for US$2.2 billion (US$3.61 billion in 2023 dollars), who intended it to be the linchpin of their lower-end server appliance line. Investors were surely delighted but Cobalt employees were upset with the new management and many left or were laid off. On top of that, Sun had bought the company to support their dot-com expansion right about when the dot-com bubble blew. Faced with rapidly slowing sales, the 1GHz/1.26GHz Pentium III-based RaQ 550 was the last RaQ system, the only one that supported Java and the only one that fully used the PHP-based user interface.

Sun terminated the product line and ended support for everything but the 550 and Qube 3 in 2003, but to their credit did open-source the operating system, though none of the various efforts (BlueQuartz, BlueOnyx, Rackstar, Strongbolt, probably others) are still maintained and none of them ever supported MIPS. Official support from Sun for the 550 and Qube 3 ended in 2007.