Fork in multi-threaded program?

The problem with forking when you do have some threads running is that the fork only copies the CPU state of the one thread that called it. It's as if all of the other threads just died, instantly, wherever they may be.

Up vote 5 down vote favorite share g+ share fb share tw.

I've heard that mixing forking and threading in a program could be very problematic, often resulting with mysterious behavior, especially when dealing with shared resources, such as locks, pipes, file descriptors. But I never fully understand what exactly the dangers are and when those could happen. It would be great if someone with expertise in this area could explain a bit more in detail what pitfalls are and what needs to be care when programming in a such environment.

For example, if I want to write a server that collects data from various different resources, one solution I've thought is to have the server spawns a set of threads, each popen to call out another program to do the actual work, open pipes to get the data back from the child. Each of these threads responses for its own work, no data interexchange in b/w them, and when the data is collected, the main thread has a queue and these worker threads will just put the result in the queue. What could go wrong with this solution?

Please do not narrow your answer by just "answering" my example scenario. Any suggestions, alternative solutions, or experiences that are not related to the example but helpful to provide a clean design would be great! Thanks!

Multithreading operating-system fork link|improve this question asked Aug 5 '09 at 20:18jimx18529 40% accept rate.

The problem with forking when you do have some threads running is that the fork only copies the CPU state of the one thread that called it. It's as if all of the other threads just died, instantly, wherever they may be. The result of this is locks aren't released, and shared data (such as the malloc heap) may be corrupted.

Pthread does offer a pthread_atfork function - in theory, you could take every lock in the program before forking, release them after, and maybe make it out alive - but it's risky, because you could always miss one. And, of course, the stacks of the other threads won't be freed.

– jimx Aug 6 '09 at 0:22 Correct. The fork clones all locks while they're still in the locked state. – bdonlan Aug 6 '09 at 1:45.

It is really quite simple. The problems with multiple threads and processes always arise from shared data. If there is not shared data then there can be no possible issues arising.

In your example the shared data is the queue owned by the main thread - any potential contention or race conditions will arise here. Typical methods for "solving" these issues involve locking schemes - a worker thread will lock the queue before inserting any data, and the main thread will lock the queue before removing it.

– bdonlan Aug 5 '09 at 20:31 I dunno - it could but normally your standard libraries are written in such a way as to be thread safe (sometimes you have to select thread safe versions of them). It depends on what your definition of shared data is and what the impact is. – 1800 INFORMATION Aug 5 '09 at 20:34 often, we can't avoid to have shared data.

Like pipes, file descriptors, etc. They always are shared across fork. Now under linux, one may set O_CLOEXEC flag so a fd can be closed when forking, (I guess what that mean is close the fd on the child's address space), though I don't know if that would help if we add threads in? E.g.

What if I open pipes in one thread and fork? What if another thread also does fork? Which child will be able to see the pipe?

– jimx Aug 6 '09 at 0:38 @bdonlan - no, malloc() only utilizes process memory. @jimx - threads within a process share FD, and while I didn't know O_CLOEXEC - the effects are the same regardless of which thread calls fork(). – NVRAM Nov 5 '09 at 21:27.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.