They used to be the president's biggest fans. So what happened?