This supersedes the patch I sent earlier today.
I tested this by manually modifying the user database and confirming that FAS overrode all settings I made locally, either removing or adding group memberships as appropriate.
When the groups are unchanged, we will only require a single lookup into the database, as opposed to one per group the user was a member of.
It also addresses a bug where users whose group membership was revoked would still appear to be a member within Django apps.