Another story from printer-land of 20 years ago: this time about a seemingly impossible bug.

While working on the LPS-20 PostScript software, a bug was filed that said roughly,

Print the attached file. The LPS-20 will jam. You'll have to open the printer to remove the scrunched up paper.

We were no strangers to jammed printers, but a particular file that could jam the printer? Yeah, right. It was crazy.

I printed the file. The printer jammed! I cleared the jam, printed the file again, it jammed again. I printed a different file, it printed fine. I printed a third file, it printed fine. Printed the bug report file again, the printer jammed, WTF!? How can a file reliably cause a printer to jam?

The mystery in this case was solved by the hardware team, because while we software guys were working on the software that fed PostScript files to the printer, the hardware guys were still getting the kinks out of the printer itself.

A laser printer has a drum on which the image is formed, and then used to transfer the image to the paper. In a 20 ppm printer like the LPS-20, the drum rotates once every three seconds. With a PostScript printer, a page could take an arbitrary amount of time to render. If the page takes less than three seconds to render, then the drum will rotate at full speed, with no pauses.

But if the PostScript interpreter takes longer than three seconds to finish a page, then the drum has to stop and wait for the image to be ready. Starting and stopping the drum and all the associated feed machinery is not trivial to get right.

In the case of the bug report file, the page took longer than three seconds, but only slightly longer, so not only did the drum have to stop, but it hadn't quite totally stopped before it was started again. That mechanical edge case is what made the printer jam. The file had pages that took just the right amount of time to hit a bad timing window in the drive train firmware, and the printer jammed every time. Once the hardware guys adjusted the firmware, the problem went away.

Lessons:

  • Just because a bug seems impossible doesn't mean it is.
  • Abstractions are everywhere, and they can be broken. As a software guy, I believed that getting the paper into the output tray was a solved problem.
tagged: , , » 30 reactions

Comments

[gravatar]
Doug 10:39 PM on 24 Nov 2008

As a QA guy I enjoyed this posting. Sometimes 'luck' is involved with running into the defect but it takes follow through on the part of the tester and developer to see it to resolution.

[gravatar]
Jon 12:21 AM on 25 Nov 2008

Oh yeah I see this kind of thing every day. Finding proof is nearly impossible though. Could you please provide a like to the example page that caused the jam?

[gravatar]
Vincent Manis 1:00 AM on 25 Nov 2008

I well recall the IBM 1403, a massive line-printer that was the workhorse of 1960s and 70s computer centers. This thing had a massive rotating chain that smashed down onto the paper in order to print (think of a typewriter wielded by the Terminator while on speed). Printing a + in the first position caused the printer to overprint; overprint enough lines (say around 20, with various combinations of AAAA and ZZZZ), and the printer would jam. The thing ran fast enough that any jam had fed enough paper into the printer to require some serious work to clear.

1403s had another peculiarity. There was a paper tape that told the printer where the top of page was. An eject command caused the printer to space the paper forward until the tape (moving at the same speed) indicated the top of the next page. If the tape broke, the paper would eject at high speed...generally most of a box of paper (2000 sheets) would eject before the operator noticed it.

I don't know when IBM discontinued the 1403, but a Google search just now revealed several companies that sell ribbon cartridges for them.

[gravatar]
interesting 1:28 AM on 25 Nov 2008

why does it take 3 seconds to interpret a postscript file

[gravatar]
JD 1:39 AM on 25 Nov 2008

>why does it take 3 seconds to interpret a postscript file

"Another story from printer-land of 20 years ago"
Things were not as fast then.

[gravatar]
Ruben 2:06 AM on 25 Nov 2008

This reminds me of a similar problem I once had with a Word document that crashed our departemental server. As you probably know, in windows networking the printer driver is running on the server and not on the client. Due to an obscure bug in the printer driver, every time somebody printed a certain Word Document, the server rebooted. It took 6 days before our IT experts realised that the combination of that Word document + printer driver were causing the issue. Everyone was thinking that the document was not printed because the server crashed, while it was the other way around. A group of 80 people in the IT darkness for 6 days, isn't that funny. By the way the Word document that caused the issue was a normal Word document that could be opened and saved in Ms Office without problems. The printer was an HP laserprinter with a Microsoft certified printer driver

[gravatar]
dec printer 2:19 AM on 25 Nov 2008

Like the experience with the IBM 1403, we had a DEC chain line printer when I was at a university back in the late 70's. The chain held all the slugs which whirred rapidly like a tank tread over the paper. When a slug with the desired character was over the correct space, an electro-mechanical hammer, one of 132 for each space on the line, would stamp it. Each hammer was actuated independently so multiple characters might be stamped at the same time and the chain would spin until all the characters on the line were stamped before line feeding. It was loud enough that it had to be inside a large refrigerator sized foam lined box. But if you printed a line with the same character order as the slugs on the chain though...omigod...it sounded like bowling balls raining on a used car lot and the printer would start skittering around the floor. We expected something amusing to happen but not for it to be shaking itself to death. Being back in the days when mainframe printers cost more than any university student dared to imagine, we did not repeat our little hardware experiment.

[gravatar]
Svein Halvor 5:54 AM on 25 Nov 2008

This reminds me of the case of the 500 mile email.

[gravatar]
Ned Batchelder 6:45 AM on 25 Nov 2008

For those that want a copy of the file: I don't have it, and it wouldn't do you any good anyway, it was particular to a bug in a specific model of printer, and the bug was fixed 20 years ago!

About why the file would take 3 seconds to execute: not only were things not as fast then as now, but PostScript is a Turing-complete programming language. You can construct a file to take as long as you like.

As I recall, the file was a TeX output file full of Type3 bitmap fonts. Perhaps that had something to do with the complexity.

[gravatar]
tenaciousJk 8:45 AM on 25 Nov 2008

@svein

The 500 mile email. A Classic!

[gravatar]
Steph 10:24 AM on 25 Nov 2008

That's a pretty amazing bug story! I have to admit, I would have been in the same boat. How can a particular file cause a printer jam. It's amazing how working in the abstract world of software so much causes us to forget that there's real hardware running our software underneath the seams. Great story.

[gravatar]
Jesse McNelis 10:58 AM on 25 Nov 2008

I heard a story about a program that would crash on wednesdays in september. Sounds crazy...but it was an off-by-one error in a buffer allocation to store the date in the format "Wednesday, 01 September 2005".
"wednesday" and "september" are the two longest day and month names.

[gravatar]
JD 11:03 AM on 25 Nov 2008

Good sir, I have looked for an email for you but failed, so I must reply here.

The emails you send in 'Email me future comments' appears to be broken. I am using Evolution as my email client that does not seem to like HTML mail, and you don't send a plain version - so I cannot easily read these emails.

[gravatar]
Patrick 11:26 AM on 25 Nov 2008

Ned, I think it is very possible that I wrote that bug report. I was a co-op in the Westford "TASTE" group in 1989/90. My responsibility was to help with development of the PostScript test suite, with the LPS-20 and LPS-40 (I think) being my key targets. I recall writing some long-duration-interpretation tests with delay loops and reporting bugs like this. I think we actually met briefly a few times. I worked with Shari?

I found your blog through a reference on Gruber's blog. Quite a blast from the past.

- Patrick

[gravatar]
leo 11:42 AM on 25 Nov 2008

ruben,

Your right but your wrong, its not windows networking, its called having a Print Server. The driver is downloaded to your machine from the print server, and when you print the document is converted into the output file that the print driver would create and sends it to the printer THROUGH the print server. SO you are right, but you said windows networking as if it has anything to do with your print driver or the process of printing. Microsoft Networking is just the general name that all the different components of networking fall under.

[gravatar]
Brian Costlow 11:50 AM on 25 Nov 2008

> About why the file would take 3 seconds

Oh, for files that only took three seconds.

I worked on desktop publishing software that printed to some of the early hi-res filmsetters using a PostScript RIP. It was normal to print a 32 page tabloid document and wait an hour or two for all of it to finish. Sometimes, a single complex page done in Adobe Illustrator, or its then competitor, FreeHand, would take an overnight shift to output. Worse, it might run for several hours then die when the RIP ran out of memory and/or temp disk space. Good times.

> PostScript is a Turing-complete programming language.

My first programming language was PostScript. I learned while working as a QA tester, by debugging and optimizing PS routines in some of our products. Long story on why I had to do that instead of just demonstrate the issue and send it back to engineering.

It warps you for life, sort of like if your cradle language was pig-latin.

Thanks for the trip down memory lane.

[gravatar]
David 12:47 PM on 25 Nov 2008

I'm really glad to read this article. My experience is that the more I working with software, the more I learn to doubt my own assumptions when encountering seemingly "non-deterministic" or "paranormal" behavior. A classic example is editing a file that drives dynamic web content but forgetting to restart Apache. In the default configuration, there are multiple processes, so if some have not launched yet, then you will get random results depending on whether or not your request is served by a process that has already launched.

[gravatar]
Johnjay60 1:16 PM on 25 Nov 2008

I know this is only marginally related, but I have a brand new out-of-the box HP 6000, with an HP 300 printer. I have to restart the print spooler in XP every time I print a document. I went to HP's official site: their 'fix' for the problem is to restart the print spooler every time you print a document. Some help desk you got there, HP.

But on a closer note - back at McDonnell-Douglas in late 70's a legend was that someone figured out the arm movement speeds on an old hard disk (IBM 3380?) and and set up a program to seek back and forth at high speed and start moving the disk drive unit across the floor, like a washing machine that is unbalanced. Never saw it happen.

[gravatar]
Galen 1:37 PM on 25 Nov 2008

I've seen and heard the chain printer phenomenon (severely clattering printer) described above with some old (even at that time) CDC chain printers that we had at Lockheed Martin in Sunnyvale back in the mid-1980's. Also something similar, though perhaps less severe-sounding, with band printers. And fun with rapid paper ejection when the "form control" tape (made of Mylar in these printers, I think) occasionally got damaged. Was there a common generic name for such tapes?

I wonder what would happen if you sent that same text over and over, with no paper feed, and each time shifting the text one character in the direction the band or chain was moving--until you'd made two or three complete cycles through the character set?

And on the dot-matrix graphic line printers like the Printronix P600 or the DEC-branded LG02, you could tell whenever the printer drew a horizontal line by the "bzzzt" that was a little louder than usual.

[gravatar]
DrJon 3:03 PM on 25 Nov 2008

This is a perfect real-world example of Doug Hofstaeder's machine-killing information from Godel Escher Bach.

[gravatar]
TJ 3:30 PM on 25 Nov 2008

Hollerith (punch) card readers (like line printers) were also 'fragile'. Punching too many holes in a card caused it to become too flexible; inserting one in a deck of cards (the machines sorted dozens a second) would reliably cause the machine to jam ... and destroying many of the cards in the process.

[gravatar]
Vincent Manis 5:05 PM on 25 Nov 2008

@johnjay60: i can well recall IBM 2311 disk drives (7.5MB, with the form factor of a washing machine), which did move while seeking. I used to operate a System/360 Model 40 (small-scale mainframe), with a bunch of these. When you ran the sort program, the drives started swaying in a distinctly eerie manner. The IBM 1130 small computer system had a disk drive with a noisy stepping motor for seeking. You could be down at the other end of the building and hear the distinctive noise that meant that the Fortran compiler had found no errors, and was beginning to write out the object file.

@galen: we called them `carriage control tapes'. I don't know if there was an official IBM name for them. My memory is hazy now, but I believe the early ones were indeed paper. Maybe later 1403 carriage control tapes were Mylar too.

[gravatar]
William 5:21 PM on 25 Nov 2008

Re: Long Render times in PS.

PS is a turing complete language (as noted above), and in those days ran on a 4Mhz 68000 -- hence, pretty slow. At one point, it was considered grand fun to write ray tracers in PostScript, which were programs that would render within the memory of the printer, and then print out the resultant image.

It took hours and hours of time to run programs that were under a page long. Irritated the hell out of the admins...

[gravatar]
Dave P 5:49 PM on 25 Nov 2008

I got that beat. Herewith is the hands-down weirdest printer bug I've ever seen.

I was working for QualityLogic (formerly Genoa), a company that produced, among other things, certification tests for page description languages. Without boring you with the details, these tests just ran through each command in a PDL and exercised it six ways from Sunday.

Anyway, I was working on an upgrade to an HP PCL-XL test, and I came to the command (I don't recall the name off the top of my head) that set the color mode for printing bitmaps. This command actually did TWO things: it set the color mode to either indexed or direct, and it created a palette in memory.

Now, if you had set the color mode to direct, the palette was pointless, the color information came from the bitmap. So I figured it would be a good test to try poking various colors into the palette with direct mode set, and verify that the palette colors had no effect on the printed image.

And sure enough, on our reference printer (some HP color Laser Jet model), the command worked exactly as expected.

Our test pages had several pre-defined cells per page for separate tests, so I created a few tests that poked various colors into the palette, and verified that none of them affected the output.

Then I noticed I had one cell left on the page, so just for grins, I used it to set a palette that was all grays (ie, the R, G, and B values were equal).

The image didn't print. All the other tests on the page (the ones with non-gray palettes) printed. Just not the grayscale test.

I thought I'd made some kind of mistake, so I tried it again, with a different set of grays.

The image didn't print.

I tried a whole series of grayscale palettes, and in every case, the image didn't print. But when I inserted just ONE non-gray color into the palette, the image printed just fine. If I altered a gray by so much as +/-1 in any channel, the image printed. If I mixed gray and non-gray tests on a page, all of the non-gray tests printed fine, none of the gray ones did.

Unfortunately, since what I was doing was essentially black-box testing, I never got a chance to see the source code for the printer, so I have no idea what caused this. If you told me to write a command that would *deliberately* do this, it would be trivial. But if you wanted a command that was supposedly properly-written, yet had this as a side-effect, I wouldn't even know where to start.

[gravatar]
Stephen 7:10 PM on 25 Nov 2008

Now -- imagine you are the poor sucker who, as the end user, "discovers" a bug like this. Who on earth is going to believe you? You could spend hours, nay DAYS on the phone with technical support until somebody has the compassion to elevate it and get it into the hands of the engineering team.

That is, IF they even bother to let it get that far.

Lord help us all.

[gravatar]
Peter 7:34 PM on 25 Nov 2008

IBM 1403 printers, LPS40's and 20's used them all. The 1403 when connected to a IBM 7044/1401 could not over print and treated the "+" in column one the same as a space, some users started always using "+", which caused some problems when an upgraded printer (CDC 512?) did overprint!

The oddest problem I saw on a LPS20, was a paper jam error on every page. It turned out the paper was A4, but the printer was set for A5, and complained that each page took too long to exit the printer!

Don't get me started on old disk drives!

Peter.

[gravatar]
Alan Frisbie 8:30 PM on 26 Nov 2008

About a year or two ago, I discovered a Postscript file that would cause
most HP B&W LaserJet printers to lock up, requiring a power-off cycle to
clear. It was a color operator that was not properly handled by the
printer firmware.

I reported the bug, but for some reason HP was reluctant to issue
fixes for all their Postscript printers. :-)

[gravatar]
sing yee 7:53 PM on 20 Feb 2009

One of the PC in my office is having this problem: each time sending a new word file to print will cause a paper jam but it's no problem with the old generated word file. There is no problem with .xls or .jpeg .... documents.

Reinstalled the Office didn't seem to solve the problem. I have no idea at all.

[gravatar]
Adrian Hutchinson 12:49 AM on 3 Nov 2012

Here we are in 2012 and I have an ms word document that can make an HP inkjet go "out of paper". Works on 3 different models I know of. Copying the text to a new document which prints will stop that document ever printing!
We have a spare printer now as we purchased a new 8600 pro as our other printers were apparently broken!
I will ave to spend some time with this document finding out how it does it.

[gravatar]
joel berman 3:22 PM on 16 Aug 2014

Oh you kids. Back in the day we had all kinds of bugs to destroy hardware. Reading the same memory location over and over could heat the cores enough to damage them on some machines. Printing a row of the same character on a printer would fire all the hammers. If you timed it right you could kill the printer power supply. Many bugs on setting programmable monitor setting to values that would burn out various components. Punching fully laced cards could both jam the heck out of the card punch and mess up the relays. On the other hand, if you were lazy and needed to move a printer or a disk drive, you could seek or print at the resonant frequencies and get those big suckers to slowly march along the floor.

Add a comment:

name
email
Ignore this:
not displayed and no spam.
Leave this empty:
www
not searched.
 
Name and either email or www are required.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.