July 21, 2007

Non-cooked mode input problems with piping over SSH

Posted in Coding, input redirection, Linux, Software, ssh at 8:46 pm by mj

Thought I’d share this, since what I thought was a 2-hour Saturday early morning side project went horribly awry.

Consider this Perl code snippet:

use Term::ReadKey;

ReadMode('cbreak');
while (1) {
    my $key = ReadKey(-1);
    if (defined($key)) {
        print "Got key $key\\n";
        last; 
   }
}
ReadMode('normal');

Simple enough: it does nothing until you press a key. This idiom might be used, for example, with non-blocking reads while you’re processing another file, as below:

use Term::ReadKey;
use IO::Select;

my $handle = IO::File->new("tail -f /some/file |");
my $selectable = IO::Select->new();
$selectable->add($handle);

my ($tailCommandPid) = getChildPids();

ReadMode('cbreak');
while (1) {
    my $key = ReadKey(-1);
    if (defined($key)) {
        print "Got key $key\\n";
        last; 
    }
    my @ready = $selectable->can_read(1);
    foreach my $h (@ready) {
        my $line = <$h>;
        print $line if defined($line);
    }
}
ReadMode('normal');
$selectable->remove($handle);
kill 9, $tailCommandPid;
$handle->close();

This also works. It continually echoes the lines it reads, until you press a key.

But what if you want to read the lines from a remote shell? Can we replace the line

my $handle = IO::File->new("tail -f /some/file |");

with

my $handle = IO::File->new("ssh someServer tail -f /some/file |");

and be on our merry way?

Well, no. In this case, your input on STDIN is completely ignored, unless you switch back to plain-old cooked mode.

Or is it really ignored? Maybe ssh is using your input for its own nefarious purposes. Makes perfect sense if your program is but a mediary between your user and an interactive remote shell.

Turns out, ssh accepts -f as an argument to force it into the background. Will that work?

my $handle = IO::File->new("ssh -f someServer tail -f /some/file |");

Success! But now we’re killing the wrong process at the end.

By forcing itself into the background, ssh is really daemonizing itself: its parent PID (on Linux) is 1. Your program will exit, but you’ll leave ssh happily running in the background, unless you result to less-than-safe tricks with ps and grep.

Further down in the man page, however, and we find the trick: -n keeps ssh running as a normal child process, but does not read from STDIN:

my $handle = IO::File->new("ssh -n someServer tail -f /some/file |");

And this, my friends, is a mistake I will never make again. And neither will you.

July 8, 2007

Creating a RESTful API Detailed Case Study

Posted in api, Development, Software, Web2.0 at 9:48 pm by mj

Joe Gregorio posted a great example of designing a RESTful API, and the brief ensuing discussion is instructive, too. (Yes, it’s a few weeks old.)

He walks through the steps of extracting the nouns, mapping the verbs, ensuring correctness and reliability, and generating and validating tickets to provide a RESTful API for a moderately complex situation.

It looks like development tools are finally catching up, such that it’s much easier to rely on HTTP verbs and response codes than it was even a year ago. From my limited experience, though, it seems there are still some gaps, and thus obstacles. If you have experience in this area, drop me a line.

Managing my feeds

Posted in Life, Me at 9:09 pm by mj

I’m not dead! Woohoo!

Since moving to Google Reader, I’ve realized just how much stuff I’d been missing doing things manually all this time. And, as with everybody who starts reading feeds, I’ve found it’s completely overwhelming–and trying to read too much is counter productive. (I’ve tried to share interesting posts on my link blog, but original thinking and writing has suffered.)

I knew this, but it all came to a head a few days ago when Scoble started putting LOL Cats and other crap in his link blog. Now, that in itself isn’t too bad, but one big problem with subscribing to the RSS feed of a shared Google Reader account is you don’t know whose feed you’re reading. You only see the name of the original blog and original author. So I spent an hour trying to track down if maybe Google had messed up, or some other feed was spewing trash, before I realized it was Scoble.

And that he was posting, on average, something like 20 stories a day.

And some of the blogs I thought I was already subscribed to were really coming from Scoble’s feed.

And many other blogs he linked to I was already subscribed to. (Call it osmosis.)

In other words, I didn’t have a handle on my feeds.

So, since I was already feeling miserable this weekend due to my allergies blocking up my ears, I pruned my feeds. Post too much? Gone. Low signal to noise? Gone. Have to click 2 or more times to get to mediocre content (ahem, Artima)? Gone.

I feel so much better about it, and I think I’m down to a reasonable enough traffic that I can at least get through the software, industry analysis, and humor blogs every day. And, while I didn’t completely unsubscribe from all partial feeds, I have segregated them to their own folder, which I will check far less often.

The diff on my OPML file is ugly. I’m keeping it for future reference.

I’m also abandoning Google’s “sort by auto” feature, which prioritizes feeds that are updated less often. It gives a false sense of how much progress I’ve made.

I have in mind a few interesting (to me) things to write about, and with my feeds now under control, and feeling better about things at work, I should get to them in the coming weeks.