Client Area · 0114 299 4050

How to remove duplicate emails from an Exchange mailbox

How to remove duplicate emails from an Exchange mailbox
Neil shows us how to remove duplicate emails from Exchange 2007 and above using a powershell script

Have you ever been importing .pst's into Exchange through Outlook and forgotten to tick the "do not import duplicates" check box, then spent the next week trying to remove all the duplicates? Be honest...

Well don't worry, there is a way to export duplicates from any mailbox through powershell, without disrupting the users!

This method works with Exchange 2007 onwards, but I have only tested this on Exchange 2013.

So let's get started....

You will need to create yourself an admin account for this job. I suggest creating an account called "ExportAdmin" or something similar. This new user needs to be a part of the Domain Admins group.

Next you need to grant your new admin account impersonation permissions to the mailboxes in the Exchange Management Shell using the following command:

New-ManagementRoleAssignment -Name:impersonationAssignmentName -Role:ApplicationImpersonation -User:ExportAdmin

**Change ExportAdmin to the name of your admin account

Assign impersonation permissions

Now you will need to download the Remove-DuplicateItems.ps1 powershell script ready to run, which you can find on the Microsoft Technet site here.

If you don't already have the Exchange Web Services feature installed (EWS), you will need to install this now. You can download it here.

When installing EWS, take note of the installation path, then put the Remove-DuplicateItems.ps1 file into this new folder. By default the install location isC:\Program Files\Microsoft\Exchange\Web Services\1.2

Now open up Powershell (not the Exchange shell) but run it as the admin account you created earlier. Then change directory to the folder mentioned in the last step. You are now ready to roll!

Be aware that this script will only remove duplicates that are in the same folder. If you have duplicate folders, you would first need to merge them, then run this script to remove any duplicates. The script can run on an entire mailbox, or on a particular folder, for example you can run it on the Calendar folder only to remove duplicate meetings!

The example below permanently deletes duplicate items from all folders in the user's mailbox, using the admin account you created, by matching the criteria set out in the script such as the date and time received and the sender along with the subject.

.\Remove-DuplicateItems.ps1 -Mailbox "" -Type All -Impersonation -DeleteMode HardDelete -Mode Full -Verbose

 Run the script

Change the  parameter to the email address of the mailbox you want to administer, and if you only want to move duplicates to the deleted items folder so you can check which items have been detected, change the -DeleteMode parameter to MoveToDeletedItems

You can change certain other parameters to meet your needs. For instance if you only want to remove duplicate items from the user's calendar, then change -Type All to Calendar. A full list of available parameters is shown below:

Server Exchange Client Access Server to use for Exchange Web Services. When omitted, the script will attempt to use Autodiscover.

Impersonation When specified, uses impersonation for mailbox access, otherwise current logged-on user is used.

DeleteMode Determines how to remove messages. Options are:

  • HardDelete:  Items will be permanently deleted.
  • SoftDelete:  Items will be moved to the dumpster (Default).
  • MoveToDeletedItems:  Items will be moved to the Deleted Items folder.

Type Determines what kind of folders to check for duplicates. Options are:

  • Mail
  • Calendar
  • Contacts
  • Tasks
  • Notes
  • or All (default).

ModeDetermines how items are matched. Options are:

  • Quick: Removes duplicate items with matching PidTagSearchKey attribute (default).
  • Full: Removes duplicate items with predefined matching criteria, depending on item class.

As you can see from the results above, the test mailbox I processsed had no duplicates (as I had already run the script and forgot to take a screen shot of the results...sorry!)

I presume that you could use the get-mailbox | command to run this script against multiple maiboxes but as of yet this is untested. I will post the results once this has been verified!

I hope this saves you a huge headache next time you (or a client) misses that all-important check box!

< Back to Blog

Related Posts:


  1. Mack Altman

    When referring to other users' scripts, it may be best to direct those who read the blog to the creator. For more information on the above, users can go to:

    Reply to this post

    Allowed tags: <b><i><br>

  2. Bob Mills

    I get an error when trying to run the script:
    File C:\Windows\system32\Remove-DuplicateItems.ps1 cannot be loaded. The file C:\Windows\system32\Remove-DuplicateItems
    .ps1 is not digitally signed. The script will not execute on the system. Please see "get-help about_signing" for more d
    At line:1 char:26
    + Remove-DuplicateItems.ps1

    Reply to this post

    Allowed tags: <b><i><br>

  3. Resolve

    Have you tried “set-executionpolicy remotesigned” before you run the script?

    Reply to this post

    Allowed tags: <b><i><br>

  4. Resolve

    I did indeed

    Reply to this post

    Allowed tags: <b><i><br>

  5. Resolve

    You could try using the below version of the command suggested above:

    Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

    This should ensure that nothing is blocked and there are no warnings or prompts. I would also suggest moving the PS1 file to a different, non-system folder, such as C:\temp or C:\scripts, then trying the process again.

    Reply to this post

    Allowed tags: <b><i><br>

Please leave a comment

Allowed tags: <b><i><br>

emergency it response : 0114 299 4050
View PAYG Options