September 4, 2018

tailgrep

Following a file, grepping for lines, watching a spinner.

I sometimes use tail -f to watch a file, and also grep the output for a particular phrase.

For instance, I might look for a URL in an nginx log file. I might start a screen session, send the output to a log, and filter for phrases in a long running command. Or use script, which I recently learned about, to generate a real time log of a shell session.

In a perfect world, that's all I need. But sometimes things go wrong. The web server stops. The long running command gets killed. And tail -f | grep don't help me realize that the output has stopped.

$ tail -f /var/log/nginx/access.log | grep special
1.2.3.4 - - [29/Aug/2018 -0400] "GET /special"…
█

$ screen -L
$ script -F /tmp/out
$ echo "Read me back the last line"

$ tail -f screenlog.0 | grep Read
Read me back the last line
█

$ tail -f /tmp/out | grep Read
Read me back the last line
█

What would be nice would be a spinner to indicate that output is still being generated.

It would also be nice if the spinner stops spinning if there is no output.

So, here's a little program to accomplish that. I called it tailgrep.

I don't reimplement tail -- I'm too lazy to think about opening and seeking to the end of a file -- I just use make a Proc::Async object and then watch stdout.

There are some subtleties -- whenever looks like a loop but isn't. It sets up callbacks. And react declares an event loop. start puts it in a separate thread.

Also you have to get the order right: Make the Proc::Async object, then call .stdout to create a Supply. Set up your the callbacks on your event loop with whenevers. Then start the process.

It returns a Promise which you can await.

What else.

shell "tput 'civis'"; (and cnorm) make the cursor invisible (or visible). signal(SIGINT) is another Supply that is called whenever ^C is pressed (i.e. a sigint signal is received).

< > splits up a string into an array. <<...>> interpolates too. $++ maintains a stateful variable and (post-)increments it.

#!/usr/bin/env perl6

sub spinner() {
  <\ - | - / ->[$++ % 6]
}

sub MAIN($expr, $filename) {
  shell "tput 'civis'";
  my $proc = Proc::Async.new:
    <<tail -f $filename>>;
  my $out = $proc.stdout;
  start react {
    whenever $out.lines.grep( / "$expr" / ) {
      .say
    }
    whenever $out.lines {
      print spinner() ~ "\r";
    }
    whenever signal(SIGINT) {
      shell 'tput cnorm';
      exit;
    }
  }
  await $proc.start;
}

That's better:

Oh, yeah, MAIN($expr,$filename) declares both the parameters and the command line arguments, and also generates a usage message which is displayed when you start the program with -h.

First I want to ensure that the file exists. To do this, I add a constraint on the $filename parameter.

sub MAIN($expr, $filename where *.IO.e) {

 

Also let's write a message if there hasn't been output for a few seconds.

We make another Supply that calls a little routine every second to check for that.

my $last-seen = DateTime.now;
...
  whenever $out.lines {
    print spinner() ~ "\r";
    $last-seen = DateTime.now;
  }
  whenever Supply.interval(1) {
    if DateTime.now - $last-seen > $wait {
      say "--no lines for $wait seconds--";
    }
  }

And by "a few seconds" I mean whatever numeric value is given on the command line, but let's default to 2.

sub MAIN(
  $expr,                   #= what to search for
  $filename where *.IO.e,  #= a filename to grep
  Numeric :$wait = 2,      #= when to notify
) {

Did I mention that inline comments attached to the parameters become messages in the help output?

Or that named parameters (which start with a :) become named command line arguments?

./tailgrep -h
Usage:
  ./tailgrep [--wait=<Numeric>] <expr> <filename>

    <expr>              what to search for
    <filename>          a filename to grep
    --wait=<Numeric>    when to notify

That about wraps it up, but just for fun, I also decided to look for some nicer unicode spinners and add some color. For these features I needed to use a few modules that may need to be installed separately.
(zef install JSON::Fast Terminal::ANSIColor).

So, I found some spinners here, with a nice JSON file, so we just download that and store it locally.

#!/usr/bin/env perl6
use JSON::Fast;
use Terminal::ANSIColor;

my @frames = < / - | - \ - >;
sub spinner {
  @frames[$++ % +@frames];
}

sub download-spinners {
    my $store = "{ %*ENV<HOME> }/.spinners.json";
    unless $store.IO.e {
        say "downloading spinners";
        my $url='https://raw.githubusercontent.com'
          ~ '/sindresorhus/cli-spinners'
          ~ '/HEAD/spinners.json';
        shell "curl -s $url > $store";
    }
    from-json( $store.IO.slurp );
}

Also multi-dispatch is handy -- a named boolean argument for another dispatch candidate can show a list of all the spinners.

multi MAIN(
    Bool :$list-spinners!, #= list spinners
) {
    say download-spinners.keys.sort.join("\t");
}
...

If you want to try them out, here are some handy urls for curling.

curl -L https://git.io/fA8MV > ~/bin/tg-fancy
curl -L https://git.io/fA8MM > ~/bin/tailgrep

Just chmod +x and there you are.

$ tg-fancy -h
Usage:
  tg-fancy --list-spinners
  tg-fancy [--wait=<Numeric>] [--spinner=<Any>] <expr> <filename>

    --list-spinners     list spinners
    <expr>              what to search for
    <filename>          a filename to grep
    --wait=<Numeric>    when to notify
    --spinner=<Any>     which spinner

$ tg-fancy --spinner=pong tailgrep access.log

$ tg-fancy --spinner=weather tailgrep access.log