"Merge duplicates" algorithm is flawed and needs improvement

Thomas_47502
Thomas_47502 Member Posts: 2

I have tons of duplicates on pipedrive because pipedrive is not very good at detecting duplicates based on the phone number. For example, pipedrive won't recognise two duplicates (with the same name) if their phone numbers are written in two different ways: 08 52 78 94 56 and 0852789456 (without space), or 08 52 78 94 56  and + 33 8 52 78 94 56 (with international prefix).

This is quite annoying as our company uses quite a few different databases to find clients and combined, those databases have a lot of duplicates that pipedrive doesn't recognise. This means our sales team have to double check if the contact doesn't already exists in the database before calling or mistakenly call someone they shouldn't have (thinking it was a new lead when it's an existing client or something similar). It's also generally annoying as it pollutes our database with duplicates and doesnt give us an accurate number of how many leads we have.

 

Another issue is that for a duplicate to be considered a duplicate, the name AND something else need to match. I quote pipedrive on this: "People are considered duplicates when their names and one or more of the following fields are the same: email, phone or organisation."

This doesnt make sense for me as two contacts who have the same email or phone number is obviously the same person. If this can't be generalised to everyone using pipedrive, then the duplicate criterias should be customisable by each company to fit their needs. 

In my case, the name requirement doesn't help because there are a lot of contacts who go by different names and thus pipedrive doesn't recognise the duplicate. For example, contact name is the company name instead of the CEO name, or a company is operated by husband and wife (who have different names).

Tagged:
0
0 votes

· Last Updated -

Comments

  • Kreete K
    Kreete K Pipedrive Team Posts: 356
    5 Likes First Anniversary First Answer Name Dropper
    PIPEDRIVE TEAM
    edited February 2022 #2

    Hi @Thomas!
    Indeed merge duplicate rules only check exact matches. . However, if you need to change the rules overall for your Company, please reach out to our Support team (via chat or email [email protected]) to understand what can be done for your account.

  • Tom Chapman
    Tom Chapman Member Posts: 14
    First Anniversary Name Dropper First Comment Photogenic
    edited July 2021 #3

    Hi, i agree.  The duplication is simply not fit for purpose, probably deliberately so that you end up using a third party to service duplicates.

     

    We use Insycle for this which works fine.

     

  • Thomas_47502
    Thomas_47502 Member Posts: 2
    edited July 2021 #4

    For the first issue, i have been told by the support team that it's not possible but as Tom said, Insycle and Dedupely can do that for me. I've came up with my own solution where I standardised all of the phone numbers on pipedrive by writting a code that reads all the phone numbers from a spreasheet and standardises the formatting. I then reupload the spreadsheet and use the pipedrive person ID to modify the phone number. Only problem I have now is that I have 900 duplicates and I can't duplicate them all at once.

    For the second issue, the support team modified the criterias so that a duplicate is detected when only the phone OR email is an exact match.

  • Martin Pecha_2338
    Martin Pecha_2338 Member Posts: 146
    First Anniversary 5 Up Votes 5 Likes First Comment
    edited July 2021 #5
    Kreete K said:

    Hi @Thomas!
    Indeed merge duplicate rules only check exact matches. . However, if you need to change the rules overall for your Company, please reach out to our Support team (via chat or email [email protected]) to understand what can be done for your account.

    @Thomas  

    it is as @Kreete K is saying and there are reasons for that.

     

    Same organisation = Name + address

    Same person = Name + working email

    The reasons are due to handling importing the data and automation of the data correctly. Based on this criteria, pipedrive is then able to distinguish different data in the import. The same concept is inherited for duplicates.

    If you would like to see duplicates based on other factors, you can co tact support, but you have to be aware, that also the import will behave differently.

  • Ryan Bozeman
    Ryan Bozeman Member Posts: 12
    First Comment First Anniversary Name Dropper Integration Partner

    Hey @Thomas_47502

    Pipedrive will monitor duplicates using the Merge Duplicates feature. But yes, it is not able to match phone numbers using different formats and there is not a lot of customization options for identifying duplicates using other fields.

    If you need to merge in bulk, you'll either need to custom code something for the Pipedrive API.

    I wanted also to mention that Insycle (and full disclosure, I work for them) could help you to catch more duplicates. Insycle allows you to use any Pipedrive field as a potential matching field. For phone numbers, by default, Insycle ignores formatting and just looks at the underlying data and ignores formatting considerations when identifying matching. Of course, if the actual phone numbers of the record are different (say one includes country code) you may have to build an advanced template to deal with these situations.

    Along with advanced matching features such as Exact/similar matching. You also have full control over the resulting merged master record, using rules for the master record as a whole, and even setting rules for retaining data on a field-by-field basis.

    Hope that helps!


    Best,

    Ryan