SQLAlchemy 0.5.3 and 0.5.5 error: Set changed size during iteration
Update 1: See Mike Bayer’s comment below – looks like there’s a fix for this coming up in 0.5.6. Very responsive work by Mr. Bayer. Thanks!
YippieMove has been running into some spurious problems with SQLAlchemy. When upgrading Python from 2.5 to 2.6 the errors became a lot more frequent. The error is “
Set changed size during iteration” and happens in all kinds of random code paths when using the ORM layer. Today I saw it happen when getting a result object from a query by index, and then again when iterating over the result of another query.
The end of the traceback looks something like this:
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/dynamic.py", line 192, in __iter__ sess = self.__session() File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/dynamic.py", line 181, in __session sess.flush() File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/session.py", line 1354, in flush self._flush(objects) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/session.py", line 1359, in _flush if (not self.identity_map.check_modified() and File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/identity.py", line 56, in check_modified for state in self._mutable_attrs: RuntimeError: Set changed size during iteration
The code in question looks like this:
for state in self._mutable_attrs: if state.modified: return True
The program is single threaded and it sure didn’t look like “state.modified” would cause the iterated over set to change. Weird. After snooping around for a while I found this post which suggests an async GC being the problem.
Sure enough the problem appears to have gone away when automatic GC is disabled. It’s not a nice workaround because the garbage collector in Python collects cyclic garbage. But it did solve the problem for now so here it is if anyone else needs it:
import gc gc.disable()
Unless you want to be buried in garbage, make sure to collect it once in a while in a safe spot in your code using
Hopefully this will be fixed in a future version.