Developer should provide contact deactivation feature. In case Customer presently working in XYZ organization and after sometime if he/she change it and start to work with ABC Organization then we should be able to deactivate fist entry. If we label it as inactive then it will pop up in duplicate entry. If we delete contact then respective past records also get lost.
So If we inactive the customer then it should not reflect in duplicate section and past record also keep as it is.