This error can occur when you try to configure a webhook that is not currently supported, such as setting up a PagerDuty webhook. PagerDuty needs to be configured under connected services, not as a webhook.
When you remove a user from your account via Manage Account > Members in the Cloudflare dashboard, their email address is not removed from existing alerts. You need to remove their email address by editing the alert.