I got the following at program start:
mongo::ScopedDbConnection* c;
Then I launch a thread pool. Each thread should be able to upsert records based on a specific logic.
This program was not threaded before and was working OK. It was just too slow and that's why I'm making it threaded (Using async and futures)
Anyway, each thread has a function does the following:
void saveToMongo(){
c = mongo::ScopedDbConnection::get
mongo::BSONObj query;
.... other BSON objects...
there's a list which contains the records for the upsert, through which I iterate:
for (it = myLocalPacketList.begin(); it != myLocalPacketList.end(); it++) {
.... other operations ....
query_upsert_document = BSON( "update" << "packets" << "updates" << BSON_ARRAY( query_upsert_query ));
try {
c->get()->runCommand("tracer", query_upsert_document, objError);
std::cout << "Command OK: Message size: " << query_upsert_query.jsonString(
c->done();
} catch ( std::exception& e ) {
std::cout << "We got an Exception: " << e.what() << std::endl << "Message size: " <<query_upsert_query.jsonString(
std::exit(-1);
}
The thing is, I does execute a few times, but then terminates with:
starting thread...
starting thread...
starting thread...
Command OK: Message size: 6513
starting thread...
Command OK: Message size: 7096
Command OK: Message size: 7072
starting thread...
Command OK: Message size: 7073
starting thread...
starting thread...
Command OK: Message size: 5589
Command OK: Message size: 6317
Command OK: Message size: 5902
starting thread...
Command OK: Message size: 7366
Command OK: Message size: 9332
starting thread...
Command OK: Message size: 5796
Command OK: Message size: 9218
object valid: starting thread...
Command OK: Message size: 6422
Command OK: Message size: 5827
starting thread...
Command OK: Message size: 6426
Command OK: Message size: 6454
starting thread...
Command OK: Message size: 6488
starting thread...
We got an Exception: connection was returned to the pool already
Message size: 6423 We're terminating
Anyone got any ideas what may be wrong?
Hello All,I got the following at program start:mongo::ScopedDbConnection* c;
You should almost never have a pointer to a ScopedDbConnection. ScopedDbConnection is intended to be used as a value, implementing the RAII pattern (http://en.wikipedia.org/wiki/
Also, the code below suggests that you are sharing this 'c' variable among threads? If so, that is definitely problematic, as ScopedDbConnection objects are not intended for concurrent access.
Then I launch a thread pool. Each thread should be able to upsert records based on a specific logic.This program was not threaded before and was working OK. It was just too slow and that's why I'm making it threaded (Using async and futures)Anyway, each thread has a function does the following:void saveToMongo(){
c = mongo::ScopedDbConnection::getScopedDbConnection("127.0.0.1" );
mongo::BSONObj query;
.... other BSON objects...
I'm puzzled by the above, since there does not appear to be any such static method as 'getScopedDbConnection' in class ScopedDbConnection.
It is a little hard for to answer because it is not entirely clear what you are trying to do, given the incomplete code. However, you definitely should not be sharing ScopedDBConnection objects between threads, and you definitely shouldn't be holding on to them by pointer.
Also, I'd like to mention that ScopedDBConnection (and the associated ConnectionPool class and 'pool' global variable) have all been removed from the driver in the latest legacy-1.0-rc2 release. The upcoming legacy-1.0.0 driver will ship without support for these classes.
If your application requires pooling of connections, you would be better off directly building such pooling as appropriate for your application, rather than relying on the deprecated ScopedDBConnection and ConnectionPool classes.
Thanks all for your help!
Thanks,
Yes, I read about ScopedDBConnection being removed, thanks for that.
About the class itself, there is a member getScopedDbConnection here:
Static Public Member Functions | |
| static ScopedDbConnection * | getScopedDbConnection (const string &host, double socketTimeout=0) |
| static ScopedDbConnection * | getScopedDbConnection () |
| static ScopedDbConnection * | getInternalScopedDbConnection |
| static ScopedDbConnection * | getInternalScopedDbConnection |
So my implementation is the following on every thread....
on program startup:
mongo::ScopedDbConnection* c;
Then on the threads:
try
{
c = mongo::ScopedDbConnection::get
if( !c->ok() )
{
std::cout << "Connection NOK, getting connection" << std::endl;
c->get()->runCommand("tracer", query_upsert_document, objError);
std::cout << "Command OK: Message size: " << query_upsert_query.jsonString(
c->done();
}
} catch ( std::exception& e ) {
std::cout << "We got an Exception on the Upsert: " << e.what() << std::endl << "Message size: " <<query_upsert_query.jsonString(
std::exit(-1);
}
Is having "mongo::ScopedDbConnection* c;" on program startup wrong?
If i create "c" on every thread wouldn't that kill the whole idea behind connection pool?
Also, C#'s implementation has a connection pool, why is it being removed from c++?? Not smart...
Yes, I read about ScopedDBConnection being removed, thanks for that.About the class itself, there is a member getScopedDbConnection here:
Static Public Member Functions
static ScopedDbConnection * getScopedDbConnection (const string &host, double socketTimeout=0) static ScopedDbConnection * getScopedDbConnection () static ScopedDbConnection * getInternalScopedDbConnection (const string &host, double socketTimeout=0) static ScopedDbConnection * getInternalScopedDbConnection ()
You are looking at the API docs for the server (and for a very old version of the server: 2.3.0).
The driver API docs are here: http://api.mongodb.org/
The C++ driver repo has never contained the functions you reference above.
Are you actually using the "server C++ driver" from 2.3.0? That is an extremely old version, and, additionally, is a development version. If so you should probably update to the 26compat driver from here: https://github.com/
So my implementation is the following on every thread....on program startup:mongo::ScopedDbConnection* c;Then on the threads:try
{
c = mongo::ScopedDbConnection::getScopedDbConnection("127.0.0.1" );
if( !c->ok() )
{
std::cout << "Connection NOK, getting connection" << std::endl;
c->get()->runCommand("tracer", query_upsert_document, objError);
std::cout << "Command OK: Message size: " << query_upsert_query.jsonString().length() << std::endl;
c->done();
}
} catch ( std::exception& e ) {
std::cout << "We got an Exception on the Upsert: " << e.what() << std::endl << "Message size: " <<query_upsert_query.jsonString().length() << " We're terminating" << std::endl;
std::exit(-1);
}
You cannot share a ScopedDBConnection pointer across threads like this. Consider what happens if, while using 'c' in one thread, another thread executes 'getScopedDBConnection' and assigns the result to 'c'. The two threads will now be using the same connection object, and this is not permitted.
Is having "mongo::ScopedDbConnection* c;" on program startup wrong?
Yes, see above.
If i create "c" on every thread wouldn't that kill the whole idea behind connection pool?
No. ScopedDBConnection isn't the pool object. It is an object which, when constructed, obtains a new connection from the existing pool, and when 'done' is called, it returns the obtained connection to the pool. The pool is already a shared global object (and thread safe). You should simply construct a new ScopedDbConnection in each thread.
Also, C#'s implementation has a connection pool, why is it being removed from c++?? Not smart...
For many reasons. Principal among them is that it is straightforward for users to implement application specific pooling, rather than requiring the driver to offer a one-size-fits-all-poorly implementation. Additionally, the ScopedDBConnection class only worked with the global pool object, and couldn't easily be used with separately configured pools. The global pool object also necessitated complex init processing before main, complicated driver initialization and termination considerably, was a frequent source of bugs, and overall had a low quality of implementation. There were also conflicts between supporting MongoDB URI syntax and honoring the pool API. Finally, the pool didn't function correctly with authorization.
Overall, our decision was that it was better to remove the existing pool and offer users whatever facilities are required to correctly implement application specific pooling. It really isn't that hard to do. If we find that this is too difficult or onerous for users of the C++ driver, we will consider re-introducing a redesigned pool in a subsequent release.
댓글 없음:
댓글 쓰기