For the Enterprise-level license, you have to be an Admin to be able to set up Ideas Portal. Once set up, why can't you enable Contributors to add or remove users to that Portal?