What you lose there is Unity will not know that a user has been deleted from the directory external to us and we will have bogus entries in SQL - this is not going to break anything but it can result in lots of error messages being logged and messages being dumped into the undeliverable messages distribution and the like. Generally not a tidy situation, but not the end of the world.
In this case it's the admins duty to remove the subscribers from Unity manually if they are deleting users in AD.