2014년 12월 29일 월요일

Nested lock error in MongoDB MapReduce

I am using MongoDB-2.6.1. 
I tried `db.coll.mapreduce()` storing in `system.js` and accessing via `db.eval()`.

I faced the problem of **nested lock** (only for 100 documents) when running using `db.eval()`. But direct running of `mapreduce()` worked fine.


Snapshot of error message (nested lock).



**Is it not possible for storing `db.coll.mapreduce()` in `system.js` and accessing via `db.eval()` ??**
If not, what is the alternative way to this.

Could any one please help me out....



This is not a proper way to run mapReduce.
Why do you want to store it in system.js?



I want to store in system.js because I am using custom functions in scope: {} (around 5-8 functions in scope). Is it possible to access the custom functions inside scope; if these custom functions stored in system.js???



That's not supported in MongoDB and it's really not the right thing to do anyway - as you scale to systems which are sharded, you no longer have a single "server" - you have many servers, so it doesn't make any sense to have "server-side" functions.

What do these mapReduce "functions" do?  Maybe there is a more fitting way to do the same thing in MongoDB?



I am using MapReduce to get the aggregated results.

For example: for 1 ObjectId (not the _id), the count I am getting is correct for the documents less than 100.

I am not getting where I went wrong. The emit function which I used is:

emit( this.<ObjectId>, this
***Results which I got are: For 1 ObjectId, it will emit number of documents and then I am iterating those number documents to get aggregated results. But for 100 documents, the count I am getting is 2, for 105, I got 6. I wrote this logic in scope: {}. ***

What is the problem in the above results?? Is that can be solved?? Please help me to solve that problem.

One more point I would like put here......
"server-side JavaScript is discouraged as there are significant performance and scalability considerations" 

I got the above statement from one forum. If that is the case, then what is the advantage of server side scripting.

In what cases that should be used and should not be used.


Could any one give me insights about this.



I am using MapReduce to get the aggregated results.

For example: for 1 ObjectId (not the _id), the count I am getting is correct for the documents less than 100.

I am not getting where I went wrong. The emit function which I used is:

emit( this.<ObjectId>, this

MapReduce is working like this.... 

1st case: For Example: for 1 ObjectId() (Not the MongoDB generated _id) there are 99 documents, Mapreduce is working fine to count the number of documents (count is 99).

2nd case: For the same ObjectId(), if there are more than 101 documents; its not working properly. In this case, count is 2
 But for 101 documents, the count I am getting is 2, for 105, I got 6. I wrote this logic in scope: {}. ***

What is the problem in the above results?? Is this can be solved?? Please help me to solve that problem.

One more point I would like put here......
"server-side JavaScript is discouraged as there are significant performance and scalability considerations" 

Questions:
1) I got the above statement from one forum. If that is the case, then what is the advantage of server side scripting.

2) In what cases that should be used and should not be used. What are the best practices??

3) Getting the aggregated results and doing some changes to that output as required from Java. Is this the right way?? Or doing this in server side itself is the right way??

Could any one give me insights about this.




When you are aggregating, your reduce function is making an assumption that it can only be called once, but it can be called multiple times.   So it's never correct to emit "this" as value since reduce must return the same type as "this" which seems unlikely.
Maybe if you provide your reduce function we can suggest a better way to do what you are trying to do.
And I assume you're doing more than counting since you don't need map reduce to get counts.



As far as your question about server-side - you should do the work server-side rather than in your code, but not via scripting.
If possible you want to use DB itself - queries, commands, aggregation pipeline.
Not server-side JavaScript.



Yes I am doing more than counting in mapreduce. As I mentioned, for 1 ObjectId(), I have to count and get the average (looping each document in values emitted) for  6 different fields and I need to modify that output a bit. If I use aggregation, I will have to hit the database 6 times.

So when can server side scripting be used and what is the advantage??



You should show your actual MapReduce code - you are not emitting the same type that your reduce function is returning and you are assuming that reduce function will only get called once per each key, which is not accurate.




MapReduce function which I used is:

db.collection.mapreduce(
   function() { emit(this.company_id, this);},

    function(key, values) { 
        return groupingObject(values);
    },

    {
        out: { inline : 1 },
        scope: {

            groupingObject: function (values) {
                            ................ 
                            ................ // In this function I am iterating the values to count and get the average for 6 different fields (values.length is the number of documents for 1 company_id).

}});

Mostly I went wrong in using mapreduce.. Please tell me where I went wrong. What corrections need to be made?? 

For what reason I am getting wrong result??



Really - just explain what you are trying to do, give some sample data and we can show you a proper way to do it.

As it is, we're sort of guessing.   

If you want to iterate over some values to get some counts and averages, you may be able to do this with simple aggregation pipeline query.


댓글 없음:

댓글 쓰기