Broken Pipe

I’m writing a simple utility called wires. It’s a clone of the GNU utility strings, but it’s written in Rust so I wanted the name to be more metallic. I had a working build now (though it doesn’t quite have all the flags and features of the original), and I thought everything was fine, but then I piped the output to head -10. And I immediately got this:

thread 'main' panicked at 'Failed to write to supplied stream: 
    Error { repr: Os { code: 32,message: "Broken pipe" } }', src/libcore/
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Hmm, that’s not very friendly. Unix-y command line tools should be able to redirect output to each other without crashing.

Looking through my code, I had this line:

    writeln!(w, "{}{}", offset_string, string)
        .expect("Failed to write to supplied stream");

I had added “expect” because if it’s not there, the Rust compiler will issue a warning that I’ve called a function that returns Result<T,E> but doesn’t check whether the call was successful. (On a side note, this is related to the first good question I asked on StackOverflow)

In Rust, the expect() method basically says, “This method is returning a Result<T,E>, but I expect it to succeed, so panic with the given message if this call isn’t successful.” And of course, when I piped the output of wires into head -10, after 10 lines, the output stopped being successful, so my program panicked, just like I told it to. But having a tool that panics if you redirect it’s output to head isn’t good, so I kept looking. (Googling this issue is kind of funny, because “broken pipe in rust” has perfect keyword overlap with plumbing, and will not produce helpful results right away.)

I have been playing around with exa, which is a reimagining of ls, written in Rust. I tested, and sure enough, exa doesn’t panic when I pipe it to head -1. I eventually found this code: from exa on GitHub

match e.kind() { // e is the error from a failed write to stdout
    ErrorKind::BrokenPipe => exit(exits::SUCCESS),
    _ => {
        writeln!(stderr(), "{}", e).unwrap();

Basically, instead of just panicking if it can’t write to stdout, exa checks whether the write to stdout succeeded. If it failed, checks why. If it failed because of a broken pipe, exa just exits 0 because it’s done. If it failed for some other reason, exa will print that error to stderr and then exit with a failure code. So I made a similar change in the wires code

The next change I need to make in wires involved memory handling of large files. Currently, wires reads the entire file into memory and then loops over it, which is pretty inefficient. First off, it means that the memory footprint of the running program is at least as large as the size of the input file. Second, it means that even if we exit early (because of a broken pipe, for example), we read the entire file into memory. My next project with wires is to make it use a buffered reader instead of a vector of bytes.

Till then, happy learning!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s