Setting up Django with MySQL on Mac OS X Lion

Surprisingly, this is a lot more annoying that I thought it would be. Some notes to the forgetful future me:

  1. Download the MySQL .dmg installer (64-bit). The registration form can be skipped.
  2. Install MySQL, the MySQLStartupItem, and the MySQL.prefPane in that order (See http://stackoverflow.com/questions/6317614/getting-mysql-work-on-osx-10-7-lion)
  3. Open up the MySQL preference pane and make sure the server is started.
  4. Edit ~/.bash_profile and make sure that the mysql bin directory is in the path (/usr/local/mysql/bin). (The MySQL-python package needs to call mysql_config)
  5. Add a libmysqlclient symlink: sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib
  6. pip install MySQL-python

Phew. Don’t forget to update your django SETTINGS file for running locally:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': DATABASE_NAME,
        'USER': USER,
        'PASSWORD': PASSWORD,
        'HOST': 'localhost'
    }
}
Posted in Software | Leave a comment

Instagram Engineering Challenge

Darn, I’m pretty late to the Instagram Engineering Challenge party but it was still a fun little puzzle to solve even if there is no free schwag left.

Some of the Hacker News crowd felt it was too easy but I thought it struck the right balance between too easy and too much work for a take-home recruiting problem. Besides, as others pointed out, you can always remove some of the simplifying assumptions (images don’t have large blocks of repeating colors, adjacent shreds will never be in the correct order, etc) and you can make the problem as difficult as you want it to be.

The general solution sketch is fairly obvious: Compare colors in adjacent vertical pixel strips and if they are “close” to each other then the two strips should be adjacent to each other in the unshredded image. As far as distance metrics go, measuring the euclidean distance between two RGBA pixels and summing up the values for each pixel in the strip is the first thing that comes to mind.

The above seemed like a reasonable avenue of attack but, alas, the devil is in the details as they say.

The first bump in the road that came up was trying to determine what the first shred in the image is. For this, I ended up measuring the distance between every pair of shreds. For each shred, you can then determine what its preceding shred is by picking the one with the smallest distance value. The shred whose smallest distance value is the greatest amongst all shreds likely doesn’t have a previous shred and is thus the left-most shred in the image.

Next, I noticed that the algorithm was having problems with the tall black and white building in the photo. The building is unfortunately split right along a white-black transition. This causes the adjacent pixels to be rather different from each other:

To fix this, I needed a way to smooth out the colors used in the strips. So, rather than consider each pixel in isolation, I calculated the average color of the pixel and its surrounding neighbors (a 3×3 mask seemed to work ok on this image).

Finally, for the extra credit problem of automatically determining the shred width, I measured the distance between adjacent vertical pixel strips. My thinking was that the distance values should be relatively small until it reaches another shred at which point the distance value will spike. I then normalized the distance values by recasting them as the number of standard deviations away from the mean. Since there is a simplifying assumption that each shred has a uniform width, the normalized distance can be checked at those discrete spots and if each value exceeds a threshold, then we’ll use that as the shred width.

Fun way to spend an afternoon :). The python code is below and it is completely untested on any other image of course.

Posted in Software | Leave a comment

Using a custom IValueProvider in ASP.NET MVC

I’ve been working with an ASP.NET MVC application at the day job and I am impressed at how nicely the framework allows users to modify the behavior of the out-of-the-box functionality without jumping through ridiculous hoops (I’m looking at you SharePoint). The MVC team at Microsoft did a nice job of sizing the Lego pieces appropriately. Kudos!

One of the automagic pieces of functionality that comes with the framework is the ability to bind http request fields to strongly typed C# objects automatically via the DefaultModelBinder.

At first glance, creating a custom model binder seemed like a good fit for taking an encoded set of name/value pairs stored in a hidden input form field and binding them to a POCO. With this particular application, there is an external tool that will automatically set the hidden field value using the following format:

<name>:<value>|<name2>:<value2>

and then post the form to the server. It would have been nice if this external tool used a more standard encoding like json but unfortunately this is outside of our control.

In addition to binding the request data to a POCO, I also wanted to use data annotations for validation purposes. However, it seemed like there was a fair bit of trickery involved in getting a custom IModelBinder implementation to perform the validation.

I was about to head down the derived DefaultModelBinder route when I stumbled across Phil Haacked’s post about IValueProviders. The particularly important bit is one of the contributions from his co-worker:

…value providers provide an abstraction over where values actually come from. Value providers are responsible for aggregating the values that are part of the current request, e.g. from Form collection, the query string, JSON, etc. They basically say “I don’t know what a ‘FirstName’ is for or what you can do with it, but if you ask me for a ‘FirstName’ I can give you what I have.”

Eureka! I decided that a much simpler solution would be to create a custom IValueProvider that parses the encoded name/value pairs in the hidden field and presents a dictionary to the plain ole’ DefaultModelProvider. That way, we get all the DataAnnotation validation goodness for free. This turned out to be really easy to do:

    public class EncodedDataValueProviderFactory : ValueProviderFactory
    {
        public override IValueProvider GetValueProvider(ControllerContext controllerContext)
        {
            String rawCallData = controllerContext.HttpContext.Request.Form["encoded_data"];
            NameValueCollection nameValues = new NameValueCollection();
            if (!String.IsNullOrWhiteSpace(rawCallData))
            {
                var q = from nvp in rawCallData.Split(new char[] {'|'}, StringSplitOptions.RemoveEmptyEntries)
                        let tokens = nvp.Split(':')
                        where tokens.Length == 2
                        select new
                        {
                            Name = tokens[0],
                            Value = tokens[1]
                        };
                        
                foreach (var pair in q)
                {
                    nameValues.Add(pair.Name, pair.Value);
                }
            }

            return new NameValueCollectionValueProvider(nameValues, CultureInfo.CurrentCulture);
        }
    }

I decided to delegate to the out-of-the-box NameValueCollectionValueProvider instead of rolling my own IValueProvider.

Then, the EncodedDataValueProviderFactory simply needs to be registered in the Application_Start even in the Global.asax.cs class:

protected void Application_Start()
{
    ValueProviderFactories.Factories.Add(new EncodedDataValueProviderFactory());
}

That’s it! The DefaultModelBinder will automatically query the new value provider when it attempts to bind values to the POCO.

Posted in Software | Leave a comment

Django mobile site template loading

Building out a mobile version of a site is increasingly becoming a must-have feature these days. From a maintainability perspective however, it is important not to intermingle too much conditional markup within a single template (eg, “If this is the mobile version, render this, otherwise render that”). A better approach is to use a separate mobile template and fallback to the regular site template if the mobile one cannot be found.

http://sullerton.com/2011/03/django-mobile-browser-detection-middleware/ outlines a nice, clean way of accomplishing this in Django with middleware and a custom template loader:

  1. Use middleware to detect whether the user-agent string in the request is from a mobile browser. Store a value indicating what version of the templates to load in a thread-local.
  2. Create a custom template loader that reads the flag from the thread-local and loads the appropriate version of the template or fall back to the default version if it doesn’t exist.

The thread-local hackery is actually a nice way of getting around the issue of trying to pass values from the middleware to the template loader. There are a couple of small tweaks that can be made to the template loader however to improve things a bit.

The template loader outlined in the post above makes an assumption around where the mobile templates will be stored on disk. It would be nicer if you could use the existing template loaders that come with django to load your mobile template. For example, if you are creating a reusable django app, it would be nice to bundle your mobile templates in the same directory as your full site templates. Or, if you are one of the crazy people loading their templates from a Python egg, it would be nice to be able to stick your mobile templates into the same egg and use the built-in egg loader to load the appropriate one.

from django.template.base import TemplateDoesNotExist
from django.template.loader import BaseLoader
import django.template.loader

class Loader(BaseLoader):
    is_usable = True
            
    @property
    def other_loaders(self):
        for loader in django.template.loader.template_source_loaders:
            if loader != self:
                yield loader
    
    def load_template_source(self, template_name, template_dirs=None):
        site_version = get_version()
        if site_version != 'full':
            mobile_template_name = "%s/%s" % (site_version, template_name)
            
            for loader in self.other_loaders:
                try:
                    return loader.load_template_source(mobile_template_name, template_dirs) 
                except TemplateDoesNotExist:
                    pass
        
        raise TemplateDoesNotExist(template_name)

First thing to note is that this template loader adheres to the new class-based API that was introduced in Django 1.2.

getVersion() is the method that returns the string that was stored in the thread-local by the middleware. It returns either ‘full’ or ‘mobile’. Depending on how many different site variations you wanted to support, you could modify the middleware to set more granular identifiers like ‘ios’ or ‘android’ instead of just ‘mobile’.

Rather than load the contents of a template directly, this loader will delegate to the ‘real’ loaders that are defined in the settings file when it looks for the mobile version of a template. So, suppose your settings file looks like this:

TEMPLATE_LOADERS = (
    'templateloader.Loader',
    'django.template.loaders.filesystem.Loader',
    'django.template.loaders.app_directories.Loader',
)

If a request is made for the mobile version of “base.html”, the loader will see if it can find the “mobile/base.html” with the filesystem Loader, then it will try the app_directory Loader. If it still doesn’t find it, the loader will then fallback to the initially requested “base.html”. Of course, in order for this to work, it is critical that the custom template loader class appear first in the TEMPLATE_LOADERS tuple.

The templates are organized into parallel directories that are structured as follows:

/templates/base.html
/templates/home.html
/templates/mobile/base.html
/templates/mobile/home.html

The django.template.loader.template_source_loaders used in the other_loaders property is a global array that gets initialized in the django.template.loader.find_template method and contains instances of the template loaders that are defined in the settings file. I’m assuming that find_template will always be called before control passes to the load_template_source method in this custom template loader.

Would love to hear any feedback on this approach.

Posted in Software | Leave a comment

Recipe Distiller site domain popularity and the 80-20 rule

I’ve previously claimed that the main compelling feature of Recipe Distiller is its ability to automatically extract ingredients from any recipe site. However, it is quite possible that I am blinded by the fact that this is the most technically interesting part of the application. It does not necessarily follow that it is a useful feature for people.

So, I thought I would take the blinders off and put together a little chart showing the number of recipes that have been saved by domain to see if this is actually the case:

There are 1010 different site domains represented in the chart. As you can see, the top 10 sites dominate the overall saved recipe counts. They represent about 50% of the total number of saved recipes.

allrecipes.com 1885
foodnetwork.com 1014
recipes.sparkpeople.com 351
cooks.com 306
food.com 257
tasteofhome.com 223
epicurious.com 185
find.myrecipes.com 140
foodnetwork.mobi 122
southernfood.about.com 121

(The data is a tad dirty in that foodnetwork.com and footnetwork.mobi are counted as two separate domains. A more thorough analysis would try and conflate these types of duplicates)

The 80-20 rule appears to apply here as well. If you scroll the chart out to the right, you will see an annotation indicating the domain that marks the end of the top 20% of the recipe site domains. The domains in the top 20% account for 86% of all saved recipes on the site.

That’s a pretty high percentage so it would make sense to focus exclusively on these. A custom screen scraper is a lot more accurate than the automated ingredient extraction algorithm that Recipe Distiller uses so why not just build out a bigger library of scrapers that work with the top 20%? The problem is that the top 20% represents just over 200 different screen scrapers that need to be built and then maintained if sites change their markup. If you look at the competing products out there, they generally only support somewhere between 10-30 different recipe sites. Building out a scraper library that is large enough to capture a sizable portion of the long tail is subject to seriously diminishing returns on developer time once you get passed that top 30 point. Furthermore, the counts drop off a cliff very quickly and I suspect there will be a lot of churn at the bottom end of that 20%. You will always be in a reactive mode if you try to use scrapers exclusively.

A hybrid approach might be better: Build out special scrapers for the top 10-30 but then fallback to the automated (but less accurate) algorithm for the rest.

A couple of caveats to all of this:

  • Small sample size, yadda yadda yadda
  • The data is being drawn from a potentially biased set of users. Recipe Distiller is marketed as being usable with any recipe site so it is certainly possible that the people using it are the types to specifically seek out the non-mainstream sites for recipes.
Posted in Software | Leave a comment

Lost in the App Store

Well, that was certainly quicker than expected.

The good news is that Recipe Distiller for iPhone only took 4 days to approve. That was a lot faster than I had expected although the app is really quite simple and I can’t imagine that it would take long to review. A testament to the minimalist design, that. 🙂

The bad news? The sobering reality that trying to sell something is exponentially harder than giving it away for free. In my head I didn’t think it was possible to gain much traction without doing any marketing but I have to confess that there was some small part of me that was secretly hoping that millions of cooks would happen to check the App Store on the day of release and be compelled to buy a copy. Not too surprisingly, things didn’t quite work out that way and sales are dismal to non-existant after Week 1. Maybe more non-existant than dismal if I’m being honest.

So, needless to say, I won’t be quitting the day job anytime soon. I had no dreams of making much of a profit, this entire project has mostly been a learning exercise after all, but I was hoping to recoup the costs of the $99 developer’s license.

The main issue to solve is a non-technical one (gulp) which is how to effectively promote one’s app and prevent it from drowning amongst the vast sea of apps that are added to the store every day. The Android version is free and I was quite surprised to build up a small user base on that platform with no advertising done at all. There are a handful of recipes being saved by users every day and that is some nice validation that others also see value in the product. On the iPhone side of things however, it has just been the sound of crickets. Part of this is due to the slightly higher than normal price point ($1.99, which is infinitely more than free I realize) but I also think it is a case of being a small fish in a big pond.

I did a little bit of searching around and found a couple of other applications, Paprika and Pepperplate that essentially do the same thing except with a much more professional looking UI and lots of fancy meal planning and search features. Both appear however to require specialized scrapers for every recipe site they want to support.

And therein lies one of the main differentiating features of Recipe Distiller.

I’ve submitted some review requests to a few app review sites but I have no idea what the acceptance rate is for them nor who their main audience is. I think I will have more success if I can give away a few copies to recipe blogging community and try to get more visibility with them. These writers and their readership don’t just build their recipe collections from the top cooking sites but instead peruse recipes farther down the long tail. These sites are not going to be supported by the other apps and I’m hoping that Recipe Distiller’s automated ingredient extraction performs well enough to be a step up from the manual data entry required by the others.

Finally, if anybody out there with an iPhone wants to try it out for free, lemme know and I’ll send you a promo code.

Posted in Android, iOS, Software | 2 Comments

Newbie iOS dev thoughts from a newbie Android dev

Egads, looks like I slipped my self-imposed release date for the iOS version of Recipe Distiller by about a month. Mea culpa! Beyond the obvious issue of not scrounging together enough free time to work on it, there were several non-obvious issues that cropped up along the way that took longer to deal with than I had initially expected:

Calling a series of web service methods sequentially

Recipe Distiller exposes a RESTful web service that clients can use to extract recipe ingredients and manage grocery lists. There are several places where a client needs to call a service method A, read some values from the response message and then pass values from A’s response to another service method B. Unfortunately, the dependencies between service method calls doesn’t jibe well with the asynchronous connection methods of the NSURLConnection. While it is possible to initialize and kick off another connection request for service method B after A completes successfully, this quickly leads to spaghetti code and it can be difficult to follow the “call method A then method B” logic. Things can spiral out of control once you have additional dependencies between service calls. This was easier to manage with the Android version of the app because it was possible to spawn a worker thread and then make synchronous calls to the service in the background. While the NSURLConnection does respond to a message that sends a synchronous request to the service, it supports fewer options than the asynchronous call.

What I ended up doing was creating separate NSOperation subclasses for each service method call, creating dependencies between them, and running them on a shared NSOperationQueue. This actually ended up working out pretty well in practice as the NSOperation subclasses are nicely encapsulated and can be assembled together in different combinations if there are different dependency requirements. The only strange thing about all of this is that it seems like the asynchronous connection methods on the NSURLConnection do not work if it they are kicked off on a background thread and by default that is the thread that the NSOperation start message runs on. You can workaround this issue by marshalling the start method back to the main UI thread:

-(void)start {
	if (![NSThread isMainThread]) {
        [self performSelectorOnMainThread:@selector(start) withObject:nil waitUntilDone:NO];
        return;
    }
	
	NSURLRequest *request = nil;
	if ([self isCancelled] || !(request = [self createRequest])) {
		// Must move the operation to the finished state if it is canceled.
		[self willChangeValueForKey:@"isFinished"];
		_isFinished = YES;
		[self didChangeValueForKey:@"isFinished"];
		return;
	}
	
	// If the operation is not canceled, begin executing the task.
	[self willChangeValueForKey:@"isExecuting"];

	NSURLConnection *aConnection = [[NSURLConnection alloc] initWithRequest:request
																   delegate:self];
	self.connection = aConnection;
	[aConnection release];
	
	_isExecuting = YES;
	[self didChangeValueForKey:@"isExecuting"];	
}

Keeping strong references to NSManagedObject instances

I thought CoreData was pretty nifty way of managing persistent data. It definitely beats the homegrown, half-implemented ORM I ended up creating for the Android version of the app. The general CoreData usage pattern I started with was basically copied from a couple of sample projects I had seen: A UITableView populated with a NSFetchedResultsController displaying a list of recipes or grocery items. Tapping a row initializes a new details controller that maintains a strong reference to the NSManagedObject.

The Recipe Distiller app supports a background sync operation with the server that causes havoc with the above approach. It is possible for a NSManagedObject to be deleted as part of the background sync while a details controller view is being displayed to the user (note that the actual manipulation of the NSManagedObject is happening on the UI thread so there is no issue of trying to use the NSManagedObjectContext from multiple threads going on here). Once the NSManagedObject is deleted from the context, it is automatically faulted out but cannot be deleted because the details controller is still hanging onto it. Now, when the controller attempts to read values from the managed object to display in the view, a fault cannot be fulfilled error will occur.

To workaround this, I ended up having the details controller store a reference to the NSManagedObjectID of the managed object along with any other non-managed object fields that it needs to display. Then, when the user attempts to save any changes, the object ID is used to lookup the managed object. If the object still exists, any changes made on the details view are set into the managed object and saved, otherwise, the details controller is simply popped off the navigation stack.

Programmatically adding Mobile Safari bookmarklets

Can’t do it.

Recipe Distiller relies on integrating itself into the browser to allow users to capture recipes on a web page being viewed. Android makes it easy to do this by registering an intent filter in your application manifest. Unfortunately, it is hugely painful to do this on iOS devices and relies on a lot of manual steps from the user and I’m positive that this is going to be a dealbreaker for a lot of users. I’m pretty sure there is no better way to do this as of right now (iOS 4.3). Other apps that behave similarly to Recipe Distiller like Instapaper also have this same issue.

In an ideal world, every recipe site owner would add a “Save Recipe” link to their recipes. This would be very similar to the Reddit, Facebook Like, Digg, et. al buttons that litter then web. I doubt that the big sites would be willing to do this as they want to lock users into their own grocery list and recipe management toolsets but I can see Recipe Distiller being a nice value add for blog writers that don’t have the time or wherewithal to build this stuff on their own. Getting the long, long tail of bloggers to buy in is for Part 2 of Recipe Distiller World Domination.

Inability to reuse app names in itunes Connect

My typical click first and ask questions later approach really screwed me up here. In my impatience, I entered the incorrect Bundle ID when I created the application in iTunes Connect so the app validation was failing in XCode. I figured I’d be able to just delete the app I had created and recreate it. I’m an idiot for not heeding the warning message which appeared when I selected Delete App but I incorrectly assumed that I’d be able to reuse an app name if it had never been given an associated binary or made available in the app store. Imagine my horror once I stumbled this Stack Overflow post telling me I was out of luck. I also learned that iTunes Connect will also consider app names that differ only by casing or whitespace as being the same. Sad panda.

Anyhow, I have to admit that all in all, it was a semi-enjoyable ride dabbling with the iOS SDK. The out-of-the-box UI controls and animations are much more polished-looking than the Android equivalents. I found the syntax rather inelegant-looking and the old school split of header and implementation files meant that I found myself constantly copy/pasting between (it is somewhat frightening how many different things you need to change . Recipe Distiller, err “Recipe Distiller for iPhone”, was submitted for review last night and I’m hoping to have it available on the App Store soon.

Posted in Android, iOS | Leave a comment

Quarter Year Resolutions

I think New Year Resolutions put a little too much pressure on the resolver. Witness the hordes of people at the gym in January and February and then the inevitable drop off in March. It makes a lot more sense to avoid the societal pressures of the New Year and just set a few modest, tangible goals at a non-traditional time. Seeing incremental improvements can be tremendously motivating. Poco a poco as they say. OK, so here’s my very modest short-term goal:

Release a first version of Recipe Distiller for iOS in April.

After a couple of failed starts, I’m hoping that putting this down on (virtual) paper will serve as the swift kick in the pants I need to get this out the door.

I’m certainly not planning on being as ambitious as the iPhone app in a week guy but I’m hoping to steal his idea of blogging about the process of ramping up from zero. I hate buying books about specific technologies as these get outdated so quickly but I gave in to temptation and I am now armed with a copy of iPhone SDK Development and am ready to begin stumbling my way through it. Reading and going through code samples only takes me so far though. I generally learn better when I have a project more meaty and interesting than a simple Hello World to work on. Still, nothing like investing a little bit of cash to motivate you to start doing something (although I suppose the beer making kit sitting idle in the basement would say otherwise).

There’s also a half-working (ok maybe more like 20% done) version of a Windows Phone 7 client for Recipe Distiller waiting for me after this gets done. Part of me thinks it would have been better to go down the Appcelerator route to crank out these admittedly rather simplistic looking grocery list apps out the door as fast as possible but there is another part of me that thinks it is just more fun and interesting to dabble with languages that I don’t get to use in my day job.

Posted in Software | Leave a comment

Android AsyncTasks during a screen rotation, Part II

Part I went over a couple of proposed solutions for dealing with the problem of the Activity being destroyed during a screen orientation change while an AsyncTask is running.

The technique of using onRetainNonConfigurationInstance and getLastNonConfigurationInstance to pass an AsyncTask reference from the Activity instance being destroyed to the new one being created is a good one but does not appear to handle the case when an Activity is destroyed and recreated outside of a configuration change (eg, Android might kill your non-foreground Activity in low-memory situations).  In that situation, the reference to the AsyncTask is essentially lost and there isn’t any way to tell it about the new Activity instance.

What I ended up doing to workaround this issue was to store references to the AsyncTask inside of the Application object instead.  The major assumption that I am making here is that the lifetime of the Application instance matches that of the process and will stick around even as Activities are destroyed and created (crossing fingers).  A custom AsyncTask class automatically saves a reference to itself with the Application when it is executed and will also remove itself from the Application when it completes to prevent a memory leak.

The Activity is responsible for notifying the application when it is being destroyed and restarted in onSaveInstanceState.  The CustomApplication class will lookup all AsyncTasks that have been started on behalf of the Activity and null out their Activity reference.  When the new Activity instance is created and initialized, the onRestoreInstanceState method notifies the CustomApplication again and it will pass the new Activity reference to all of the AsyncTasks that are still running.

CustomApplication.java:

public class CustomApplication extends Application {
	/**
	 * Maps between an activity class name and the list of currently running
	 * AsyncTasks that were spawned while it was active.
	 */
	private Map<String, List<CustomAsyncTask<?,?,?>>> mActivityTaskMap;

	public CustomApplication() {
		mActivityTaskMap = new HashMap<String, List<CustomAsyncTask<?,?,?>>>();
	}

	public void removeTask(CustomAsyncTask<?,?,?> task) {
		for (Entry<String, List<CustomAsyncTask<?,?,?>>> entry : mActivityTaskMap.entrySet()) {
			List<CustomAsyncTask<?,?,?>> tasks = entry.getValue();
			for (int i = 0; i < tasks.size(); i++) {
				if (tasks.get(i) == task) {
					tasks.remove(i);
					break;
				}
			}

			if (tasks.size() == 0) {
				mActivityTaskMap.remove(entry.getKey());
				return;
			}
		}
	}

	public void addTask(Activity activity, CustomAsyncTask<?,?,?> task) {
		String key = activity.getClass().getCanonicalName();
		List<CustomAsyncTask<?,?,?>> tasks = mActivityTaskMap.get(key);
		if (tasks == null) {
			tasks = new ArrayList<CustomAsyncTask<?,?,?>>();
			mActivityTaskMap.put(key, tasks);
		}

		tasks.add(task);
	}

	public void detach(Activity activity) {
		List<CustomAsyncTask<?,?,?>> tasks = mActivityTaskMap.get(activity.getClass().getCanonicalName());
		if (tasks != null) {
			for (CustomAsyncTask<?,?,?> task : tasks) {
				task.setActivity(null);
			}
		}
	}

	public void attach(Activity activity) {
		List<CustomAsyncTask<?,?,?>> tasks = mActivityTaskMap.get(activity.getClass().getCanonicalName());
		if (tasks != null) {
			for (CustomAsyncTask<?,?,?> task : tasks) {
				task.setActivity(activity);
			}
		}
	}
}

CustomAsyncTask.java:

The task registers itself with the CustomApplication in onPreExecute and then removes itself once it has finished running (onCancelled and onPostExecute).  The setActivity method is used to null out the Activity reference when the spawning Activity is getting destroyed and also to push in the new Activity instance.

public abstract class CustomAsyncTask<TParams, TProgress, TResult> extends AsyncTask<TParams, TProgress, TResult> {
	protected CustomApplication mApp;
	protected Activity mActivity;

	public CustomAsyncTask(Activity activity) {
		mActivity = activity;
		mApp = (CustomApplication) mActivity.getApplication();
	}

	public void setActivity(Activity activity) {
		mActivity = activity;
		if (mActivity == null) {
			onActivityDetached();
		}
		else {
			onActivityAttached();
		}
	}

	protected void onActivityAttached() {}

	protected void onActivityDetached() {}

	@Override
	protected void onPreExecute() {
		mApp.addTask(mActivity, this);
	}

	@Override
	protected void onPostExecute(TResult result) {
		mApp.removeTask(this);
	}

	@Override
	protected void onCancelled() {
		mApp.removeTask(this);
	}
}

TestActivity.java:

Note that the DoBackgroundTask inner class is static so as to avoid creating an implicit reference to the Activity.  Also note that the mActivity reference needs to be checked for null in all the UI-thread callbacks to handle the case where the spawning Activity has been destroyed.

public class TestActivity extends Activity {

	private static class DoBackgroundTask extends CustomAsyncTask<Void, Integer, Void> {
		private static final String TAG = "DoBackgroundTask";

		private ProgressDialog mProgress;
		private int mCurrProgress;

		public DoBackgroundTask(TestActivity activity) {
			super(activity);
		}

		@Override
		protected void onPreExecute() {
			super.onPreExecute();
			showProgressDialog();
		}

		@Override
		protected void onActivityDetached() {
			if (mProgress != null) {
				mProgress.dismiss();
				mProgress = null;
			}
		}

		@Override
		protected void onActivityAttached() {
			showProgressDialog();
		}

		private void showProgressDialog() {
			mProgress = new ProgressDialog(mActivity);
			mProgress.setProgressStyle(ProgressDialog.STYLE_HORIZONTAL);
			mProgress.setMessage("Doing stuff...");
			mProgress.setCancelable(true);
			mProgress.setOnCancelListener(new OnCancelListener() {
				@Override
				public void onCancel(DialogInterface dialog) {
					cancel(true);
				}
			});

			mProgress.show();
			mProgress.setProgress(mCurrProgress);
		}

		@Override
		protected Void doInBackground(Void... params) {
			try {
				for (int i = 0; i < 100; i+=10) {
					Thread.sleep(1000);
					this.publishProgress(i);
				}

			}
			catch (InterruptedException e) {
			}

			return null;
		}

		@Override
		protected void onProgressUpdate(Integer... progress) {
			mCurrProgress = progress[0];
			if (mActivity != null) {
				mProgress.setProgress(mCurrProgress);
			}
			else {
				Log.d(TAG, "Progress updated while no Activity was attached.");
			}
		}

		@Override
		protected void onPostExecute(Void result) {
			super.onPostExecute(result);

			if (mActivity != null) {
				mProgress.dismiss();
				Toast.makeText(mActivity, "AsyncTask finished", Toast.LENGTH_LONG).show();
			}
			else {
				Log.d(TAG, "AsyncTask finished while no Activity was attached.");
			}
		}
	}

    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        Button b = (Button) findViewById(R.id.launchTaskButton);
        b.setOnClickListener(new OnClickListener() {
			@Override
			public void onClick(View v) {
				new DoBackgroundTask(TestActivity.this).execute();
			}
        });
    }

    @Override
    public void onSaveInstanceState(Bundle outState) {
    	super.onSaveInstanceState(outState);

    	((CustomApplication) getApplication()).detach(this);
    }

    @Override
    public void onRestoreInstanceState(Bundle savedInstanceState) {
    	super.onRestoreInstanceState(savedInstanceState);

    	((CustomApplication) getApplication()).attach(this);
    }
}

One of the nice things about this approach is that the Activity does not need to keep an explicit reference to the AsyncTask. The tasks can be used in a fire and forget fashion similar to the way it is described in the official documentation.

The full project is available on github for the curious: https://github.com/callorico/CustomAsyncTask

Posted in Android | 19 Comments

Android AsyncTasks during a screen rotation, Part I

During development of the Android Recipe Distiller app, I ran into a subtle issue around the usage of the AsyncTask class.  It ended up being a little trickier than I thought it would be to perform the relatively mundane task of bringing up a progress dialog and making a webservice call in the background. This first part goes over some of the suggested solutions and the next post will go over what I ended up implementing.

My initial implementation started with the AsyncTask as an inner class of the Activity.  A progress dialog was created in onPreExecute, the web service call made in doInBackground, and then the dialog was dismissed in onPostExecute.  This was all well and good until you tried rotating the screen at which point the app would crash.

Basically, the issue is that once you call execute() on the AsyncTask, the thread performing the long running operation will continue doing its thing regardless of the state of the Activity that spawned it.  When the screen is rotated, the Activity is destroyed and a new instance is created.  The problem is that any currently running AsyncTasks will now be operating against the destroyed Activity instance and Very Bad Things will ensue.

There are a number of threads at StackOverflow going over this issue.  Here is one of them:

http://stackoverflow.com/questions/2620917/how-to-handle-an-asynctask-during-screen-rotation

In the Shelves application, within the onSaveInstanceState method, there is a check to see if there are any currently running AsyncTasks.  If so, it is cancelled and its current state is saved to the bundle.  In onRestoreInstanceState, if the AsyncTask state exists in the bundle, a new AsyncTask instance is created for the new Activity and it is immediately executed.

(Note: This has been edited for brevity)

@Override
protected void onSaveInstanceState(Bundle outState) {
        super.onSaveInstanceState(outState);
        final AddTask task = mAddTask;
        if (task != null && task.getStatus() != UserTask.Status.FINISHED) {
            final String bookId = task.getBookId();
            task.cancel(true);

            if (bookId != null) {
                outState.putBoolean(STATE_ADD_IN_PROGRESS, true);
                outState.putString(STATE_ADD_BOOK, bookId);
            }

            mAddTask = null;
        }
}

@Override
protected void onRestoreInstanceState(Bundle savedInstanceState) {
        super.onRestoreInstanceState(savedInstanceState);
        if (savedInstanceState.getBoolean(STATE_ADD_IN_PROGRESS)) {
            final String id = savedInstanceState.getString(STATE_ADD_BOOK);
            if (!BooksManager.bookExists(getContentResolver(), id)) {
                mAddTask = (AddTask) new AddTask().execute(id);
            }
        }

This feels somewhat unsatisfactory since it seems rather inefficient to potentially repeat the same expensive network call just because the device was rotated in the middle of the operation.

This thread outlines another approach that leverages the onRetainNonConfigurationInstance callback.  In a nutshell:

  • AsyncTasks are made static inner classes so they do not retain the implicit reference to the parent Activity.  Instead, an Activity reference is explicitly passed to the task.
  • When the Activity is being destroyed as part of a screen orientation change, the AsyncTask’s Activity reference is nulled out.  The AsyncTask callback methods that run in the UI thread perform a null check on the Activity before trying to make any UI changes.
  • When the new Activity gets created, the new instance is passed to the AsyncTask.

Taking a quick glance at the code should make this clear.

While this solution seems fine for the screen rotation case, I don’t think it will work correctly in the following situation:

  1. AsyncTask started inside of Activity A
  2. Phone call comes in and a new Activity is brought to the foreground.
  3. Android decides to destroy Activity A

In this situation I don’t believe the onRetainNonConfigurationInstance method will ever be called if I understand the Activity lifecycle correctly. Anytime another Activity comes to the foreground, it is possible that Android will kill the previous Activity in certain low-memory situations.

The nice thing about this approach however is that it provides a way to hand off the AsyncTask instance from the Activity being destroyed to the new one that gets created after the screen rotation.  This way, the expensive operation only needs to happen once.  With the above scenario however, there isn’t any way for the newly created Activity to get a reference to the previously spawned AsyncTask.

Part II will go over the workaround for this that I cobbled together.

Posted in Android | 5 Comments