[flow-tools] flow-print pipe stalling on large files

[Mike Hunter]

Hi everybody,

I'm not convinced that this is a flow-tools problem, but hopefully some of
the older and wiser folks out there can give me a hint on this.

I'm working on some custom scripts that take "flow-print -f 5" style text
as input.  My perl command looks something like this:

open DATA, "$flow_print_exe -l -f 5 < $tempdir/poi.$$ |"

...

while (<DATA>) 
...

(I added the "-l" recently to try to address my problem, but it doesn't
seem to have helped.)

The script churns away just great at first, but over time, `top` shows both
the perl program and the flow-print process using less and less CPU (this
is a dual-processor machine) until finally they are both using less than
2 percent.

I'm doing this on a large amount of data (7 gigs).  When I run the script
on a more sane amount of data, I never run into this trouble.  The machine
has a gig of memory IIRC.  Sometimes, the script does indeed finish.

Any advice would be greatly appreciated.

Good luck with the rest of 2002,

Mike