If we don't expire the cached user IDs, the list of IDs would become
outdated when a new member is added, or when a member ios removed from
the allowed group.
Signed-off-by: Rémy Coutable <remy@rymai.me>