Nonblocking/asynchronous fifo/named pipe in shell/filesystem?

You can use special program for this purpose - buffer. Buffer is designed to try and keep the writer side continuously busy so that it can stream when writing to tape drives, but you can use for other purposes. Internally buffer is a pair of processes communicating via a large circular queue held in shared memory, so your processes will work asynchronously.

Your reader process will be blocked in case the queue is full and the writer process - in case the queue is empty. Example.

You can use special program for this purpose - buffer. Buffer is designed to try and keep the writer side continuously busy so that it can stream when writing to tape drives, but you can use for other purposes. Internally buffer is a pair of processes communicating via a large circular queue held in shared memory, so your processes will work asynchronously.

Your reader process will be blocked in case the queue is full and the writer process - in case the queue is empty. Example: bzcat archive. Bz2 | buffer -m 16000000 -b 100000 | processing_script | bzip2 > archive_processed.

Bz2 linux.die.net/man/1/buffer.

Thank you for pointing out program that I didn't knew, seems interesting and might be useful in some cases. Though its not exactly what I wanted and I will not be able to do what I want with this I have decided that most likely I'll implement my own program/daemon for cases like mine. – morphles Jul 14 '11 at 10:38 Another solution would be to use queue daemon, something like gearman.

Your "lines" could persist in memcache as jobs. – ieskok Jul 19 '11 at 6:16.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.