RabbitMQ, Celery and Django

It’s really not a very pretty script but since I won’t be doing any more work on it I’ll release it here in the spirit of open source. Whether it works for you or not is anyone’s guess, but one thing is for sure: if it was just sitting on my hard drive it definitely would not.

Download: migrate_from_eventum.rake

See instructions in the beginning of the file.

The first problem was that the deserialization class could no longer find the class to deserialize to. No wonder since it was in a new package and had a new name. No problem. I subclassed ObjectInputStream and overrode resolveClass(ObjectStreamClass desc) to properly load the class from its new name and location.

That’s when Java freaked out with the following piece of brilliance:

local class name incompatible with stream class name X

Well, duh. Of course the name is different, that’s the point of a refactoring. What exactly were they trying to accomplish when they added this name check? The class has already been found, it’s interface is an exact match. The name of the class is irrelevant at this point.

I proceeded by overriding readClassDescriptor() with the intention to simply replace the name of the class with the new name when reading the stream. Java would never even realize the class used to have another name. But no: that method must return a java.io.ObjectStreamClass object, which you can’t instantiate or override unless you’re in the java.io package. And if you just grab the super.readClassDescriptor() instance you aren’t allowed to make any changes to it.

At this point I considered my favorite solution to Java damage. I’d like to call it “corrective Java surgery” – use reflection to force your way into the class and fix it. This is a very gratifying solution since you can undo the mistakes in Java and set things right. The drawback of course is that different versions of Java may require different corrective code.

That wasn’t necessary in this case since reading the source code of ObjectInputStream revealed a static method for creating ObjectStreamClasses: ObjectStreamClass.lookup. With that tool things fell into place very simply like so:

@Override
protected java.io.ObjectStreamClass readClassDescriptor() throws IOException ,ClassNotFoundException {
	ObjectStreamClass desc = super.readClassDescriptor();
	if (desc.getName().equals("com.trueship.base.Preferences")) {
		return ObjectStreamClass.lookup(LegacyPreferences.class);
	}
	return desc;
};

Hope that’ll help anyone else looking to rename and deserialize classes in Java.

YippieMove has been running into some spurious problems with SQLAlchemy. When upgrading Python from 2.5 to 2.6 the errors became a lot more frequent. The error is “Set changed size during iteration” and happens in all kinds of random code paths when using the ORM layer. Today I saw it happen when getting a result object from a query by index, and then again when iterating over the result of another query.

The end of the traceback looks something like this:

  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/dynamic.py", line 192, in __iter__
    sess = self.__session()
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/dynamic.py", line 181, in __session
    sess.flush()
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/session.py", line 1354, in flush
    self._flush(objects)
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/session.py", line 1359, in _flush
    if (not self.identity_map.check_modified() and
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/orm/identity.py", line 56, in check_modified
    for state in self._mutable_attrs:
RuntimeError: Set changed size during iteration

The code in question looks like this:

    for state in self._mutable_attrs:
        if state.modified:
            return True

The program is single threaded and it sure didn’t look like “state.modified” would cause the iterated over set to change. Weird. After snooping around for a while I found this post which suggests an async GC being the problem.

Sure enough the problem appears to have gone away when automatic GC is disabled. It’s not a nice workaround because the garbage collector in Python collects cyclic garbage. But it did solve the problem for now so here it is if anyone else needs it:

import gc
gc.disable()

Unless you want to be buried in garbage, make sure to collect it once in a while in a safe spot in your code using gc.collect().

Hopefully this will be fixed in a future version.

Instead I hacked up Carl Nygard’s MediaWiki to Redmine script. It’s an awful hack really but it worked for the pages I needed, and it does preserve revision history.

• Download the script: migrate_from_moinmoin.rake
• Edit the source and enter your own email address instead of admin@example.com.
• Place the file in lib/tasks/ in your Redmine installation.
• Run the command with

  rake redmine:migrate_from_moinmoin RAILS_ENV="production"

• Enter a project identifier.
• Enter an absolute path to the data/pages folder of your MoinMoin install.

/html/body/table[3]/tbody/tr/td/table/tbody/tr

My first attempt to convert it looked something like this:

$("body > table:eq(2) > tbody > tr > td > table > tbody > tr")

This gave no results. I simplified it a little bit,

$("body > table:eq(2) tr tr")

Still no luck. After experimenting for a while it looked to me like :eq wasn’t playing well in this expression. The following worked:

$("body > table").eq(2).find('tr tr')

Weird.

Here’s the solution: puttycyg. It’s a modification of putty which allows it to act as a cygwin terminal in addition to the normal ssh and telnet functionality. Very cool!

Found it through this blog post. Thank you Andreas Hochsteger!

Example:

cat myfile.txt | pbcopy

The solution was to add an SWT.KeyDown listener on the table and intercept any Home or End keyboard events. Set e.doit = false, then use table.setTopIndex() to change the viewing position in the table by hand. E.g. just reimplement the Home and End keys in your own code.

This worked fine for me.

Now this would work except that by default iReport 1.2.7 creates the PDF file in its home folder ".". This is a relative path and of course Acrobat has no idea what its relative to.

To fix this, go to Settings/Compiler and change ‘Default compilation directory’ from "." to a complete path like "C:\" or better yet "C:\Documents and Settings\USERNAME\My Documents".