Concurrent switch_db failure

[goldbaum]

I have multiple green threads that try to with switch_db(SomeDocument, alias):... on the same document in order to query multiple aliases.
Once in a while I get an exception:

File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/queryset.py", line 104, in count
return super(QuerySet, self).count(with_limit_and_skip)
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/base.py", line 347, in count
return self._cursor.count(with_limit_and_skip=with_limit_and_skip)
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/base.py", line 1481, in _cursor
self._cursor_obj = self._collection.find(self._query,
AttributeError: 'NoneType' object has no attribute 'find'
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/queryset.py", line 104, in count
return super(QuerySet, self).count(with_limit_and_skip)
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/base.py", line 347, in count
return self._cursor.count(with_limit_and_skip=with_limit_and_skip)
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/base.py", line 1481, in _cursor
self._cursor_obj = self._collection.find(self._query,
AttributeError: 'NoneType' object has no attribute 'find'

It happens possibly due to self.cls._collection = None in switch_db.enter function.
I'm using mongoengine 0.10.6

[wojcikstefan]

Hi @goldbaum, I'm afraid switch_db has many flaws in its current form. Your issue sounds very similar to #788. There's already some progress on this stuff via @anih's #1457 PR.

[bagerard]

Relates to #788

[josepato]

We ran into this issue when using multiple treads hitting different databases.

We realize that the db_alias is changing on all the class instances, so if I have

• instance_1 of the some class with db_alias='user_1'
• instance_2 of the same class with db_alias='user_2'

The instance_1, db_alias attribute will be changes to 'user_2'. So in that fraction of a second the instance_1 will hit the wrong database.

We try to init() the class but we had a lot of problems with that. So we added a property to the document (the actual document that is being created or updated) that is called 'db_alias'. So when the _save_create and _save_update are called, we check if the document has the attribute and if so, we call directly the get_db and return the proper db and collection. Still making some adjustments but so far so good.

https://github.com/linkaform/mongoengine