Hadoop Datanode, namenode, secondary-namenode, job-tracker and task-tracker?

Currently hadoop cluster has a single point of failure which is namenode.

Currently hadoop cluster has a single point of failure which is namenode. And about the secondary node isssue (from apache wiki) : The term "secondary name-node" is somewhat misleading. It is not a name-node in the sense that data-nodes cannot connect to the secondary name-node, and in no event it can replace the primary name-node in case of its failure.

The only purpose of the secondary name-node is to perform periodic checkpoints. The secondary name-node periodically downloads current name-node image and edits log files, joins them into new image and uploads the new image back to the (primary and the only) name-node. See User Guide.So if the name-node fails and you can restart it on the same physical node then there is no need to shutdown data-nodes, just the name-node need to be restarted.

If you cannot use the old node anymore you will need to copy the latest image somewhere else. The latest image can be found either on the node that used to be the primary before failure if available; or on the secondary name-node. The latter will be the latest checkpoint without subsequent edits logs, that is the most recent name space modifications may be missing there.

You will also need to restart the whole cluster in this case. There are tricky ways to overcome this single point of failure. If you are using cloudera distribution, one of the ways explained here.

Mapr distribution has a different way to handle to this spof. Finally, you can use every single programing language to write map reduce over hadoop streaming.

A lot of people are calling the secondary namenode the "checkpoint node" now, which is a good thing. – orangeoctopus Oct 19 at 12:19 Any programming language which can read/write to STDIN/STDOUT can be used with Hadoop Streaming. There are a couple of frameworks which make Hadoop Streaming easier.

– Praveen Sripati Nov 3 at 15:13.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.