MongoDB map/reduce over multiple collections?

A reduce function may be called once, with a key and all corresponding values.

A reduce function may be called once, with a key and all corresponding values. It may also be called multiple times, each time with a key and only a subset of the corresponding values, and the previous reduce results for that key. This scenario is called a re-reduce.In order to support re-reduces, your reduce function should be idempotent.

There are two key features in a idempotent reduce function: The return value of the reduce function should be in the same format as the values it takes in. So, if your reduce function accepts an array of strings, the function should return a string. If it accepts objects with several properties, it should return an object containing those same properties.

This ensures that the function doesn't break when it is called with the result of a previous reduce. Don't make assumptions based on the number of values it takes in.It isn't guaranteed that the values parameter contains all the values for the given key. So using values.

Length in calculations is very risky and should be avoided. If your reduce function is idempotent, you shouldn't have any problems map-reducing multiple collections. Just re-reduce the results of each collection: Step 1 Run the map-reduce on each required collection and save the results in a single, temporary collection.

You can store the results using a finalize function: finalize = function (key, value) { db.tempResult. Save({ _id: key, value: value }); } db.someCollection. MapReduce(map, reduce, { finalize: finalize }) db.

AnotherCollection. MapReduce(map, reduce, { finalize: finalize }) Step 2 Run another map-reduce on the temporary collection, using the same reduce function. The map function is a simple function that selects the keys and values from the temporary collection: map = function () { emit(this.

_id, this. Value); } db.tempResult. MapReduce(map, reduce) This second map-reduce is basically a re-reduce and should give you the results you need.

– ibz Oct 1 '10 at 9:43 @ionut bizau: You can use a finalize function for that. See my updated answer for details. – Niels van der Rest Oct 1 '10 at 10:06 Awesome, didn't think about that!

Thanks! – ibz Oct 1 '10 at 12:30 1 Niels, your answer is pretty good. But what if we have duplicated keys in both reduces?

I suggest to save tempResult data as {id, value} under usual ids and map them for reduce with map = function () { emit(this. Id, this. Value); } Oh!

I've found usefull feature of mapReduce starting from MongoDB 1.7.4 mongodb. Org/display/DOCS/MapReduce#MapReduce-Outputoptions > { reduce : "collectionName" } - If documents exists for a given key in the result set and in the old collection, then a reduce operation will be performed on the two values and the result will – lig 09/121 at 19:06 1 Now in 1.8 you can use {out:{reduce: 'collectionName'}} just like you mentioned.It's perfect for aggregating stats together incrementally. Check out this tutorial: kylebanker.

Com/blog/2009/12/mongodb-map-reduce-basics – Clint 09/122 at 16:49.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.