Ben Affleck fears he's 'one errant remark away from being canceled'