Saturday, November 22, 2014

Mailbox Permission Issues After Cross-Forest Migration

Hi folks,

I would love to share with you one old story that I had 3 years ago during the cross-forest Exchange migration (2007 to 2010).

After migration of mailboxes some of them cannot be disabled using Microsoft Exchange Shell or console. In order to resolve this problem Exchange Trusted Subsystem group had to be assigned Read All Properties and Write All Properties permissions on the domain container.

1. Launch ADSI Edit
2. Connect ADSI edit to the Default Naming Context
3. Right click the container that is corresponding to the domain DC=contoso,DC=com and go to the Security tab
4. Click on the Advanced button
5. On the Advanced Security Settings for make sure you are on the Permissions tab and clock on the Add button
6. Select All Descendant objects in the Apply to section. In the list of permissions tick boxes next to Allow-Read all properties and Allow-Write all properties. Click OK
7. Wait for Active Directory replication to occur.

And it should do the magic.


Removing Email Addresses With Obsolete SMTP Domain From Users Mailboxes

Hi folks,

Just wanted to share with you a quick way to remove obsolete SMTP addresses from mailboxes in your Exchange estate.

Imagine a situation when you decommissioning a certain SMTP domain from your organization. First you perform all of the activities to prevent your environment to receive email from this domain. These activities include changes in DNS (removing MX,SPF and PTR records) and also removing accepted domains from your Exchange and hygiene appliances. Additionally you may edit your email address policies to stop provisioning decommissioned SMTP domain to the new mailboxes.

However you are still left with the mailboxes that still contain this email address in their EmailAddresses  attribute. Of course you can attend each mailbox one-by-one and scrap these addresses out, however for bigger environments it won't work.

The below script will help you to remove obsolete email addresses from mailboxes of users located in the OU which have email addresses with suffix. The script will go through all the users which may have this domain and scrap it out. Please note that when running this script you may need to adjust the name of the SMTP domain you're removing as well as path to the OU where mailboxes are stored.

$Mailbox = Get-User -OrganizationalUnit -RecipientTypeDetails UserMailbox    
    $Mailbox | foreach { 
    for ($i=$_.EmailAddresses.Count;$i -ge 0; $i--) 
    if ($_.EmailAddresses[$i].ProxyAddressString  -like "smtp:adatum*" ) 


Quickly Create OU Structure in the new AD domain

Hi folks,

This is a script that will help you creating OU infrastructure in the brand new Active Directory environment (of course it can also be used in creating OUs in existing AD, provided that you use OU names that are not in use). In addition to this it will protect newly created OUs from accidental deletion which is similar to this:

Make sure that you replace DC=contoso,DC=com with the right LDAP path of your domain and feel free to replace names for OUs (next to OU=) with the names that are in your environment

Import-Module ActiveDirectory

$objDomain =[ADSI]"LDAP://DC=contoso,dc=com "
$objOU = $objDomain.Create("organizationalUnit","ou=Clusters")

$objOU = $objDomain.Create("organizationalUnit","ou=Servers")

$objOU = $objDomain.Create("organizationalUnit","ou=Groups")

$objOU = $objDomain.Create("organizationalUnit","ou=Security Groups,ou=Groups")

$objOU = $objDomain.Create("organizationalUnit","ou=Distribution Groups,ou=Groups")

$objOU = $objDomain.Create("organizationalUnit","ou=Services")

$objOU = $objDomain.Create("organizationalUnit","ou=Staff")

$objOU = $objDomain.Create("organizationalUnit","ou=Admins")

$objOU = $objDomain.Create("organizationalUnit","ou=Workstations")

$objOU = $objDomain.Create("organizationalUnit","ou=SQL Servers,ou=Servers")

$objOU = $objDomain.Create("organizationalUnit","ou=Exchange Servers,ou=Servers")

$objOU = $objDomain.Create("organizationalUnit","ou=Lync Servers,ou=Servers")

$objOU = $objDomain.Create("organizationalUnit","ou=CRM Servers,ou=Servers")

$objOU = $objDomain.Create("organizationalUnit","ou=BES Servers,ou=Servers")

Get-ADOrganizationalUnit -Filter 'Name -like "*"' | Set-ADOrganizationalUnit -ProtectedFromAccidentalDeletion $true


Tuesday, November 11, 2014

PowerShell commands to manipulate mailbox folders

Hi folks,

This is a short post in support to those of you you are using PowerShell to manipulate mailbox folders in Exchange 2010/2013.

The main thing to remember is that Get-MailboxFolder command can be run by the mailbox owners as per this Technet article . If you are trying to run the same command as an administrator you are getting the following error:

No RBAC manipulations will help you to overcome this.

Therefore if you are interested in retrieving user mailbox folders you will need to use Get-MailboxFolderStatistics command. By default, it retrieves all the information in the list format, which doesn't always fit the screen. Of course this can be exported to TXT file for further reviews. Additionally you can use certain parameters.

As an example this command I used to retrieve information about folders and their structure within a mailbox:

Get-MailboxFolderStatistics "Farhad Mahmudov" |select Name,Identity

More details about this command can be found here .

Additionally there are commands that allow you to manage mailbox folders permissions like Set-MailboxFolderPermissionRemove-MailboxFolderPermissionAdd-MailboxFolderPermission and Get-MailboxFolderPermission.


Quick Retrieving of Windows Updates installed in Exchange Environment

Hi folks,

I would love to share with you a quick and easy way to retrieve Windows updates installed in your Exchange 2010/2013 estate.

Instead of logging on locally to each server all you need is to execute Get-WmiObject -Class "win32_quickfixengineering" command against each server.

In my case I have simply added to the pipe all the Exchange servers that are domain members (usually Edge Transport servers are not added to the domain and the command should be executed against them separately). So if you don't run Edge servers in your environment there's no need to

So my command to retrieve this information looks like this (please note that I used Sort command to arrange patches based on server name and date of the installation)

Get-ExchangeServer | Where-Object {$_.ServerRole -ne "Edge"} |foreach {Get-WmiObject -Class "win32_quickfixengineering" -ComputerName $_.Name} |sort Source,InstalledOn

Optionally you can export the information to CSV file (screenshots below) to get more comprehensive report or use Select-Object command to limit it to certain parameters you're interested in.


Tuesday, October 21, 2014

Deleting Items from Mailbox Dumpster

Hi folks,

Let me share with you about another task that you usually don't do on day to day basis.

As an intro I would love to share with you link to this article which explains what is Dumpster 2.0 is and how it is used in Exchange 2010 and 2013 environments. In couple words, after being deleted messages are being stored in dumpster for further restore and legal purposes.

Every now and then we need to delete items from dumpster to free up space used by them. Please note that this will create white space in the mailbox which can be reused for the new messages that arrive to the mailbox.

To check how much deleted messages are stored in mailbox you can run command like this:

Get-MailboxStatistics "Farhad Mahmudov" |select *size*

or even a nicer one like this

Get-MailboxStatistics "Farhad Mahmudov" |select DisplayName,TotalDeletedItemSize,TotalItemSize

In any case you will have information of how much data you have in mailbox altogether as well as deleted items size:

To remove data from dumpster you need to run the Search-Mailbox command with parameters as follows and confirm when prompted:

Search-Mailbox -Identity "Farhad Mahmudov" -SearchDumpsterOnly -DeleteContent

This may take some time depending on how much data you have in Recovered Items folder (dumpster) and give you output on how much data has been removed from the mailbox.

Optionally, you can double check this by running Get-MailboxStatistics command.


Friday, October 10, 2014

Reporting ActiveSync Devices' Details

Hi folks,

Looks that I haven't posted on my blog for the good couple months, as I've been off for holiday and then spent all my weekdays evening preparing for certification exams.

This post will be quite short. As IOS 8 became available and many people started updating their iPhone and iPad devices, I have been requested by the client of mine to extract information about the information about the devices that are using ActiveSync to connect to their Exchange mailboxes.

The command Get-ActiveSyncDevice allows you to extract information about ActiveSync devices that are connected to your Exchange 2010 or 2013 environment. Depending of what you need in your report you can select different attributes of the retrieved ActiveSync devices.

In my case final code was pretty easy and straightforward and produced CSV file with the report which went to the CTO's desk.

Get-ActiveSyncDevice -ResultSize unlimited |select UserDisplayName,DeviceType,DeviceId,DeviceOS,DeviceModel,Frie
ndlyName |Export-Csv D:\Software\EAS_Device_Report.csv

And the report looks like this:

Hope it helps you as it helped me.

Thursday, July 31, 2014

Who Shutdown Your Exchange Servers

Hi folks,

Let me share with you a small script that will help to identify when and by whom Exchange servers have been restarted. Since it queries Event Log it can be used for all the other servers, no matter what role they are running.

What the script does checking System log of your Windows box for event 1074 and then you can output details that you need to CSV file.

In my case I'm running this script against Exchange servers which are the part of AD domain (which excludes Edge Transport servers). You can run it on Edge servers locally and therefore script text will slightly change (with no need to create variable that has all Exchange servers.

Therefore my text looks like this:

$ExServers=Get-ExchangeServer | Where-Object {$_.ServerRole -ne "Edge"}

$ExServers | foreach {Get-EventLog -ComputerName $_.Name -LogName system |Where-Object {$_.EventID -eq "1074"}} |Select EventID,MachineName,Message,TimeGenerated,TimeWritten,UserName |Export-Csv D:\Software\Event-1074.csv

Enjoy, hope it helps you as it did help me.

Wednesday, July 30, 2014

Dealing with Failing Exchange 2010 Multi-Role Server Installation

Hi folks,

I would love to share about one more adventure I recently had with one of my clients' messaging platforms. I needed to install additional CAS/HUB servers in order to support hybrid deployment with Office 365 and further migration into the cloud.

What can be simpler that that. I have got 3 VMs running Windows 2008 R2 on which I have installed all the prereqs as listed here.

However, when running step of installing and configuring multi-role server my installation would stumble and fail at the point of installing Exchange management tools. with the error as below:

 Exchange Management Tools                                 FAILED
     The following error was generated when "$error.Clear();  Set-WERRegistryMarkers; " was run: "Provisioning layer initialization failed: '"Scripting Agent initialization failed: "File is not found: 'c:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'.""'".

The investigation lead me that the cause of the problem is Cmdlet Extension Agents which requires ScriptingAgentConfig.xml to be present on  a server. And since it's a brand new server of course this file is not there.

As a result some of Exchange files are installed on the affected server, however server roles are not configured and no changes in Active Directory are done.

To resolve this problem first you will need to run on one of the Exchange 2010 servers in your organization the following command:

Disable-CmdletExtensionAgent -Identity "Scripting Agent"

This will temporarily disable you from running modified or extended Exchange 2010 cmdlets, but it will allow you to successfully complete Exchange 2010 installation.

After disabling Scripting Agent you can re-run Exchange 2010 installation. 

Optionally, you can uninstall failed Exchange 2010 installation as did because I prefer nice and clean installations. In my case I had to run installation couple times as one of uninstallations failed with the error like this one (which is result of a lame installation):

This uninstallation simply removes the files installed earlier.

After that you can run Exchange installation again using your favorite installation method. Just in case details about Exchange installation can be found here for 2013 and here for 2010.

In my case I ran /m:Install /r:CA,HT /InstallWindowsComponents to install Exchange 2010 CAS/HUB server with Windows prereqs and installation went successfully.

After the installation is over you will need to rename file named ScriptingAgentConfig.xml.sample which is installed into C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents. Otherwise Exchange tools will fail starting pointing to the similar error as installation.

You simply right-click the file and remove sample from its name, so it becomes XML file.

Finally,  after the installation of all my multi-role servers was over I have enabled Scripting agent back by running

Enable-CmdletExtensionAgent -Identity "Scripting Agent"

I hope this will be of a great use to you.

Tuesday, July 29, 2014

Slipstreaming Rollup Updates Into Exchange 2010 Installation

Hi folks,

If you are working on a large deployment of Exchange 2010 servers in addition to the latest service pack (SP) you will also need to have latest Rollup Update (RU) installed. Please note that Service Pack installations are the separate builds and there's no need to integrate them into Exchange installation files. So for example to install SP3 you simply need to extract files and run installation running setup.exe (for GUI) or (for unattended).

Back to RU now. Slipstreaming process is fairly easy. All you need to do is to place MSP file for the latest RU into the Update subfolder of your Exchange 2010 installation folders.

After that you run installation command and wait for the end of installation.

And here you go! In my case I have installed Exchange 2010 SP3 with RU5 without any additional effort.

And when you check Exchange version using Exchange Management Console (for some reason in 2010 PowerShell doesn't reveal RU version information) you are getting 14.03.0181.006 which corresponds to RU5 for SP3 (as it is my scenario). This may differ in yours.

Alternatively, you can find it in Windows Updates applet of Control Panel by running see Installed Updates.

Please note that you will need to keep this file structure available to you for any installs, uninstalls or updates of your Exchange build on a server you installed it at.

Finally, in Exchange 2013 the servicing model has been changed and you now have Cumulative Updates (CU) published every quarter and SP published every year. Every CU and SP is now a separate build, so they can be installed using Setup /m:Install on the new servers and Setup /m:Upgrade on the new servers.


Friday, July 11, 2014

Reporting Mount Points for Mailbox Servers

Hi Folks,

Recently I had one more adventure.

I extremely love my customers, especially those who have a good standardization and conventions in their infrastructure. One customer of mine had pretty strict requirements for size of LUNs for database files and transaction logs.

However 2 things happened. Once their backup had got broken which as you know would result of transaction logs not being truncated and disk space usage grow presenting the threat of dismounting of database. This resulted into extending size of LUNs used by transaction logs and further discrepancy in disk size.

So my task was to determine which LUNs don't have standard size so that their size can be squeezed back to a standard one. LUNs were presented to mailbox servers as mount points.

In order to discover their size I decided that I generate a script that would report information about mount points and generate a nice CSV files that can be later processed by Microsoft Excel.

My greatest thanks to the author of  this article as I used it as basis for my script.

First I defined variables so that data is output into CSV in gigabytes instead of bytes. And after that I defined a variable for all mount points which then I have output into CSV file.

And I ended up with script that exports information about absolutely all mount points that I have across my client's Exchange estate:

$TotalGB = @{Name="Capacity(GB)";expression={[math]::round(($_.Capacity/ 1073741824),2)}}
$FreeGB = @{Name="FreeSpace(GB)";expression={[math]::round(($_.FreeSpace / 1073741824),2)}}
$FreePerc = @{Name="Free(%)";expression={[math]::round(((($_.FreeSpace / 1073741824)/($_.Capacity / 1073741824)) * 100),0)}}

$MPoints = Get-MailboxServer |foreach {Get-WmiObject -computer $_.Name win32_volume | Where-object {$_.DriveLetter -eq $null}} 
$Mpoints |Select SystemName,Name, Label, $TotalGB, $FreeGB, $FreePerc |Export-Csv D:\Scripts\Mount_Points_All.csv

I have also done some tweak to make output of a particular mount points I was interested in. I needed information of all mount points which are larger than 100 GB and smaller than 855 GB, so I ended up with the filter {$_.Capacity -gt  107374182400 -and $_.Capacity -lt  918049259520} which is calculated by converting GB into bytes, so the code looked like:

$MPoints |Where-object {$_.Capacity -gt  107374182400 -and $_.Capacity -lt  918049259520} | Select SystemName,Name, Label, $TotalGB, $FreeGB, $FreePerc| sort-object SystemName,Label | Export-Csv D:\Scripts\MountPoints_Oversized.csv

And of course you can come up with different other filters that will allow you to extract the exact information you may need.

In my case I have ended up with the nice Excel file as below:


Tuesday, July 1, 2014

Easy Checking Message Tracking Log Across All Servers

Hi Folks,

This is a quick tip on how to extract information from message tracking logs. Exchange has nice GUI tools like Message Tracking which used ECP, EAC and Tracking Log Explorer. However we can use them against a single server (even it's remote) at a time.

The following commands allow you to extract message tracking information from all Hub Transport servers at the same time. Let's imagine the following scenario: you have Hub Transport server role co-located with CAS on the servers. And also you are trying to find all emails that were sent to a user named John Smith with email address from a user Mary Jones with email address (of course nothing personal, these names are fictional here).

First I created variable called $Hubs which I populate with Hub Transport servers.

If you have separate Hub transport server roles you can use (and no Edge servers will be there simply because they are not present):


If you have Edge servers in your Exchange topology then the following command is for you

$Hubs=Get-ExchangeServer | Where-Object {$_.ServerRole -like "HubTransport"}

And finally in my scenario where I have Edge servers in topology and Hub Transport servers also has CAS role running, I have populated my $Hubs variable this way (since I don't need edge servers there because I was looking only for internal emails):

$Hubs=Get-ExchangeServer | Where-Object {$_.ServerRole -like "ClientAccess, HubTransport"}

Afterwards I have run Get-MessageTrackingLog command against each Hub Transport server in $Hubs variable with specifying

$Hubs |foreach {Get-MessageTrackingLog -server $_.Name -Sender -Recipients}

This command can be even better filtered by using -Start and -End  with dates and times specified (to search limited to certain dates) or any other parameter.You can check all the parameters for Get-MessageTrackingLog command here .

And finally report can be outputted as list (Format-List) to see every single detail for each SMTP transaction, though very hard to read or you can output it to CSV file using Export-CSV command. Something like:

$Hubs |foreach {Get-MessageTrackingLog -server $_.Name -Recipients} |select EventId,Source,Sender,@{N="Recipients";E={$_.Recipients}},MessageSubject,TimeStamp |Export-Csv D:\Software\Output.csv


Playing with RBAC and Roles

Hi folks,

Today let me share my experience with creating and managing role groups. RBAC is a very good security engine in Exchange 2010 and 2013 to manage administrative access by providing to administrative users only rights necessary for them to perform on the daily basis. In couple words a user is limited to a number of PowerShell commands he or she may execute or to certain sections in EMC or EAC he or she may access. I will not go into many details about this as you can read this article nicely provided in Technet.

In my case I was using standard roles created by Exchange 2010 by default with no customization which I was assigning to the newly created role groups.

Below commands are used to manage role groups and assignments.

First of all we create a role group (let's say for Level 1 engineers)

New-RoleGroup Level1

After this we assign role to a group. The following command allows to do it one-by-one

New-ManagementRoleAssignment -SecurityGroup Level1 -Role "Message Tracking"

We can check if rules were successfully assigned to this group:

Get-RoleGroup IPsoft-L1 |fl Roles

The above commands are quite simple and straightforward. Let me share one case when I needed to mimic one role group configuration between lab and production environment (which should be a normal case for any messaging environment and not only messaging).

For the sake of this article let's call this group Level3 as it will be serving admin needs of level 3 IT engineers to whom I don't want to assign Organization Management permissions.

First I have checked all roles assigned to the role group created in the lab:

Get-ManagementRoleAssignment -RoleAssignee Level3 |select Role,RoleAssigneeName

RoleAssigneeName corresponds to a user or a group to which a particular role is assigned. Of course as output of this command I will get the name of the same group assigned to it, but in the next step you will see how I used this to easily recreate this role in my production environment.

I used the same command, however instead of outputting it to screen I have exported results into a CSV file like below:

Get-ManagementRoleAssignment -RoleAssignee Level3 |select Role,RoleAssigneeName |Export-Csv D:\Software\Roles-i3.csv

I have ended up with the output like below:

As you can see I have 2 columns named after attributes I have exported Role,RoleAssigneeName . I will use them in the next command as parameters for -SecurityGroup and -Role.

As you can guess, I had imported username into pipeline and executed ManagementRoleAssignment against each of them with the parameters mentioned above.

Import-Csv .\Roles-i3.csv |foreach {New-ManagementRoleAssignment -SecurityGroup $_.RoleAssigneeName -Role $_.Role}

This simple command is used to add a users into a role group one-by-one:

Add-RoleGroupMember Level3 -Member "Farhad Mahmudov"

If we go back to my previous example of transferring role group between test lab and the production environment, I will show you some magic that I have done in my environment to import mailboxes from lab to production (provided your AD environment in the target environment is not populated as it was in my case)

First, I have extracted group members in my lab with the necessary details and exported detailes necessary for further mailboxes provisioning to CSV file:

Get-RoleGroupMember Level3 |Get-User |select Name,DisplayName,SamAccountName,UserPrincipalName | export-csv D:\Software\Level3.csv

I have transferred CSV file into my production environment and there provisioned mailboxes (please ensure that there are no trailing spaces after name or display name, otherwise you will have some grief of cleaning this out of CSV file). Please note that here I was using standard password for eevery admin which they were required to reset afterwards for security reasons.

$Password = Read-Host "Enter Users Password" -AsSecureString

Import-Csv C:\Scripts\Level3.csv |foreach {New-Mailbox -Name $_.Name -SamAccountName $_.SamAccountName -UserPrincipalName $_.UserPrincipalName -DisplayName $_.DisplayName -Password $Password}

Then I ran this code to simply check if users and mailboxes are created:

Import-Csv .\Level3.csv |foreach {Get-User $_.Name}

As you can guess mailboxes creation was successful. Finally, after that I ran Add-RoleGroupRoleMember command against each user of imported CSV file to add them to this group as members:

Import-Csv .\Level3.csv |foreach {Add-RoleGroupMember Level3 -Member $_.Name}

I hope this will be very beneficial in your daily messaging management.

Monday, June 30, 2014

Managing Distribution Group Administrators by using PowerShell

Hi Folks,

Here I would love share some simple code to tweak distribution groups administrators by using Exchange management shell.

Of course it is easier to do  by using Exchange Management Console or Exchange Admin Center (if you are using Exchange 2013), however sometimes you may need to automate this task or modify admins for more than one distribution list.

Distribution list administrators are controlled by ManagedBy property which in turn corresponds ManagedBy attribute (which can also be accessed and modified by using ADUC and have only 1 value) and to msExchCoManagedByLink (which is accessed via ADSIEDIT or Attributes editor tab of ADUC in advanced view) and can have multiple attributes.

If we modify ManagedBy propery by simply using Set-DistributionGroup with ManagedBy parameter specified, we will simply overwrite current value of this attribute which is not always what we want to achieve, in particular if we want to add more than one group admin to those currently configured.

This requires some PowerShell magic:

First we create a variable which contains the current value of the ManagedBy attribute (let's call it $Mng):

$Mng=(Get-DistributionGroup "My Dist Group").ManagedBy

Secondly we add name of our new admin to this variable:

$mng+="John Smith"

And finally, we use modified $Mng variable in configuring ManagedBy property of the distribution group:

Set-DistributionGroup "My Dist Group" -ManagedBy $Mng

I hope it will be useful to you!

Monday, June 16, 2014

Simple Code for Reporting Mailbox Size for a Single Database

Hi Folks,

This is quite simple PowerShell code that I want to share with you today.

It pipelines mailboxes from a DB (let's call it DB03) and reports their names, total items and total deleted items sizes in megabytes (as for small mailboxes reporting in gigabytes is not always accurate) along with the total items and total deleted items numbers and nicely imports all data to CSV file that can be then processed in MS Excel.

Please note that plain use of TotalItemSize and TotalDeletedItemSize outputs like this:

Therefore I had to convert information about the mailbox size and deleted item size so that it can be then processed in Excel. I had to use the following expressions to achieve this: @{N="MailboxSize MB";E={$_.TotalItemSize.Value.ToMB()}} and @{N="DeletedItem MB";E=$_.TotalDeletedItemSize.Value.ToMB()}}

All this results in the following script:

Get-Mailbox -Database DB03 |Get-MailboxStatistics |select DisplayName,@{N="MailboxSize MB";E={$_.TotalItemSize.Value.ToMB()}},@{N="DeletedItem MB";E=$_.TotalDeletedItemSize.Value.ToMB()}},ItemCount,DeletedItemCount |Export-Csv D:\Scripts\DB03-Mbx-Size.csv

As a result I'm getting a nice output as this one:


Wednesday, June 11, 2014

VSS and Mysteriously Lost Disk Space

Hi folks,

This tricky situation happened to me right after coming back to work place after long and wonderful weekend.

I have been reported by my colleagues that our customer is having issue that with the disk space. Database space was shrinking and for some databases it had reached threshold when it had to be split by 2 parts. As I started investigating this case I had the same though as they did and was ready to prep RFC for database rebuilding and mailbox moves.

However I discovered that free disk space is reported differently for mount point hosting 2 copies. One copy was reporting 48% of disk space while the other one 9%. This confused me especially after I didn't see anything special in the file structure even after hidden files were enabled. Not to mention, that LUN size for both database copies was absolutely the same, as my customer is pretty standardised.

Helpless, I went to my brother Google. And after some time I found this article which became that magic silver bullet to resolve my problem. It had quickly reminded me that we had backups failing for lengthy period of time (as you know since Exchange 2010 there's no more online backup APIs supported, only VSS). And these failed backups would not delete VSS snapshots they would have created.

This is an easy three-step solution what reveals and deletes all of the VSS copies that piled up in the storage.

First we launch the utility. As you understand it's quite obvious. You simply need to type diskshadow and hit Enter.

To type next 2 commands you will need capital letter (press CAPS LOCK or hold Shift button all the time you type them).

First we list if there are any shadow copies that were not removed by backups:


After we discovered that there are any shadow copies sitting on your disk for a long time, which can be easily figured out by the date of each VSS snapshot, we delete all of them by running:


When done you will need to type exit to exit utility and then close Command Prompt.

I hope this article helpful as you solve this tricky and easy, at the same time, problem.


Friday, June 6, 2014

Decomissioning Obsoltete SMTP Domains

Hi folks,

I have recently helped my client to decommission one of their obsolete SMTP domains.

For the sake of this article let's call SMTP domain

My special thanks to the author of this article which helped me to do the trickiest bit e.g. removing aliases from bunch of mailboxes.

My client had quite a few mailboxes which were assigned aliases from which I had to scrap this out. It's worthy to mention that these mailboxes were stored in particular databases so I used -Database parameter when creating variables for the mailboxes. If you have a different scenario, you may want to use a different filter. For example: -ResultSize Unlimited (as in the original article) if you want to scrap it out of all mailboxes in your organization.

By this code I have removed aliases from mailboxes stored in the particular database:

$Mailboxes = Get-Mailbox -Database DB03
$Mailboxes | foreach{
    for ($i=0;$i -lt $_.EmailAddresses.Count; $i++)
        $address = $_.EmailAddresses[$i]
        if ($address.IsPrimaryAddress -eq $false -and $address.SmtpAddress -like "*" )
            Write-host($address.AddressString.ToString() | out-file d:\scripts\addressesRemoved.txt -append)
    Set-Mailbox -Identity $_.Identity -EmailAddresses $_.EmailAddresses}

Another nice feature of this code is out-file d:\scripts\addressesRemoved.txt -append which outputs list of removed aliases thus giving us reporting and control over which domains were removed from the mailboxes.

Ok, the first and the trickiest bit is done. Now we need to prevent organization receiving emails to the To do this we need to remove accepted domain and ensure that accepted domain removal is synced to Edge Transport servers (as in my environment, yours may differ especially if you have a different messaging hygiene solution, still you will need to remove it from there).

The first bit is straightforward and needs really no comments:


In case of edge servers I didn't want to wait till next Edge Sync and forced it. As a lazy person I scripted it into the following code so that it runs against all Edge Transport servers in the organization:

$Edge = Get-ExchangeServer | Where-Object {$_.ServerRole -eq "Edge"}
$Edge | foreach {Start-EdgeSynchronization -Server HUBSERVER -TargetServer $_.Name}

And that's the whole magic.

Wednesday, June 4, 2014

Figting High CPU Usage on Mailbox Servers Caused By Uneven Spread of Mailbox Database Copies

Hi folks,

Sounds like a time to share my recent adventure.

I have recently run into an interesting issue. On 2 of servers that are the part of Database Availability Group (DAG) one was running at the top of CPU. It would never go below 70% which is not healthy. At the same time its pair hosting copies was consuming only 20-30% of CPU.

I had to resolve such injustice and make sure that CPU usage is equal and within norm on both of the pairs. First of all I needed to determine how many of copies each of the pairs has mounted and why are they mounted this way.

This is where my 2 good friends came into play e.g. PowerShell and Microsoft Excel.

First of all I ran against each server hosting the copy to make sure that this number is equal:

(Get-MailboxDatabase -Server MbxSrvHostname).count

An it actually was equal.

Now I wanted to understand how copies are mounted on both servers and why are they mounted there. I ran the following command (please don't do it yourself):

Get-MailboxDatabase -Server ServerName |select Name,Server,Servers,Activation* |export-csv D:\Software\DBCopyN4P4.csv. 

And the result was quite ugly and unreadable.

I had to tweak output for Servers and ActivationPreference parameters as follows to ensure the proper output: @{N="Servers";E={$_.Servers}},@{N="ActivationPreference";E={$_.ActivationPreference}}.

My final command for outputting databases and their copies and preferences turned to be as follows:

Get-MailboxDatabase -Server ServerName |select Name,Server,@{N="Servers";E={$_.Servers}},@{N="ActivationPreference";E={$_.ActivationPreference}} |export-csv D:\Software\DBCopyN4P4.csv

I won't be able to show the output here due to ethics, but I can guarantee that this is working.

Now the time came to redistribute mailbox database copies across both members of the pair. To make my life easier I generated CSV file that contains the list of the database copies and in particular their names which I needed for configuration command:

Get-MailboxDatabase -Server ServerName |Get-MailboxDatabaseCopyStatus |Select Name |Sort Name |Export-Csv D:\Software\DbCopy.csv

I have got output as below

I have done some processing with the file. First, I removed ugly 1st row. Secondly I have added column and added preference numbers for database copies in the following format: 1-2 2-1. This is because copies were sorted in CSV file by name and the copy on the 1st server was always listed before the one on the 2nd.

Something like

Name Preference


After that the time for magic came. I have reconfigured activation of every database copy using CSV file I had created in the previous step. I imported it into pipeline and then executed Set-MailboxDatabaseCopy command against every copy in the file. Something like this:

Import-Csv D:\Software\DbCopyPref.csv |foreach {Set-MailboxDatabaseCopy -Identity $_.Name -ActivationPreference $_.Preference}

After this you can use either Move-ActiveMailboxDatabase command or RedistributeActiveDatabases.ps1 from Exchange scripts to redistribute copies based on their preference. Because I was dealing only with 2 mailbox servers from DAG and didn't want other copies/servers to be affected  I was using Move-ActiveMailboxDatabase against every mailbox database copy. Boring, but safe.

And all of this was worthy of effort. CPU usage on both servers in pair went down to 30-50%.

Hope it will be helpful.

Thursday, May 15, 2014

Flexible Restore of Mailboxes from Recovery Database

Hi Folks,

Looks like its time for another post for in this blog.

Lately I have been working on the restores of mailbox for one VIP user for my client. I have been requested to restore his Calendar from 2 mailbox database restores. Since this user was indeed very important the task was to restore his Calendar into alternate mailbox.

I won't go into details of creating a recovery database as everything is already said in this Technet article

Using Restore-Mailbox command wasn't very helpful since they couldn't find some of the items in the restored Calendar folder and therefore requested me to restore dumpster and Deleted Items as well.

Here when the New-MailboxRestoreRequest command comes into play. Please note that it restores mailbox dumpster (Recoverable Items folder). According to Microsoft folks this command is helpful for all kinds of mailbox restores. Again, you can read this magic article about it and all its keys and parameters.

Please note couple parameters there. First one is TargetRootFolder. You can one of the default root folders or create your own folders beforehand. In my case I created custom folder under root before running restore. It's completely up to you whether you want to use New-MailboxFolder command or simply create it using Outlook.

Secondly, there's another parameter AllowLegacyDN. It allows you to restore data from mailbox to alternate mailbox without ExchangeLegacyDN errors (this parameter is unique for every mailbox).

If you know that mailbox can contain bad items (as it was in my case after I lost 4 hours simply to see that mailbox restore has failed) you can use BadItemLimit  and AcceptLargeDataLoss depending on the number of items you can lose.

And finally in order to distinguish one restore from another I have used Name parameter and gave each restore its name in order to be able to track restores one by one.

Below I will give you examples of 2 commands that I used and based on the Technet article above you can tailor yours based on your needs.

Restore whole mailbox to alternate mailbox (with bad items):

New-MailboxRestoreRequest -SourceDatabase Recovery-DB01 -SourceStoreMailbox "John Smith" -TargetMailbox "Mary Smith" -TargetRootFolder CustomRootFolderName  -AllowLegacyDN Mismatch -Name "Whole Mailbox Restore" -BadItemLimit 150 -AcceptLargeDataLoss

Restore particular folder to alternate mailbox:

New-MailboxRestoreRequest -SourceDatabase Recovery-DB01 -SourceStoreMailbox "John Smith" -TargetMailbox "Mary Smith" -TargetRootFolder CustomRootFolderName -IncludeFolders "Deleted Items" -AllowLegacyDN Mismatch -Name "Restore Deleted Items Aug 2013"

Finally I used New-MailboxExportRequest command in order to import restored data into PST file and handed it to the happy client.


Thursday, May 1, 2014

Mailboxes Reporting on Screen and In Excel

Hi folks,

I'm extremely happy to share with you some more of my experience that I recently had with Microsoft Exchange and scripting for it. This should be working for both Exchange 2010 and 2013.

I have been asked by my customer to automate reporting of the number of all mailboxes by their types. You know managers don't care much for the technical details but only for digits, digits and digits.

The commands I was using can be found anywhere in Technet or any good book on Exchange 2010/2013. However I'm still happy to remind them to you.

(Get-mailbox -resultsize unlimited).count reports total number of all kinds and sorts and breeds of mailboxes in your org. (Get-mailbox -resultsize Unlimited -RecipientTypeDetails RoomMailbox).count reports total number of room mailboxes in the or. You can replace RoomMailbox with EquipmentMailbox, LegacyMailbox, LinkedMailbox, UserMailbox, DiscoveryMailbox, SharedMailbox and many others. Of course all of them for Exchange 2010/2013 you can find here , so I won't spend much time on it

If you have more than 1000 mailboxes in your org don't forget to add -resultsize Unlimited or type your magic number.

The tricky part was how to output all these values to a nicely formatted file. Of course commands like Out-File or even >filename.txt can be used. However I wanted it to look a lot nicer than a simple TXT file. My aim was to produce a CSV file that can be then processed in Excel, should this be needed. So I decided to export every breed of mailbox to its variable and then export variables into a CSV files, which I didn't know how to do. But as my ex-boss would always say "Good IT guy needs 2 things brain and Google". My brain helped me to define what I need, time was for Google to work. So I ended up here . I tried the first response and it did the magic.

As said previously, I wanted the results to be reported on screen for me and in CSV file for the client. So my magic script ended up looking like this. (If you want any of the sections may be commented you. However I keep both which gives me a signal that the commands are actually executing there without error and I'm getting the result).

#Define variables for mailbox types

$TotalMbxText = "Total Mailboxes"
$TotalMbx = (Get-mailbox -resultsize unlimited).count
$RoomMbxText = "Room Mailboxes"
$RoomMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails RoomMailbox).count
$EqpMbxText = "Equipment mailboxes"
$EqpMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails EquipmentMailbox).count
$LgcMbxText = "Legacy mailboxes"
$LgcMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails LegacyMailbox).count
$LnkMbxText = "Linked mailboxes"
$LnkMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails LinkedMailbox).count
$UserMbxText = "User mailboxes"
$UserMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails UserMailbox).count
$DscMbxText = "Discovery mailboxes"
$DscMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails DiscoveryMailbox).count
$SharMbxText = "Shared mailboxes"
$SharMbx = (Get-mailbox -resultsize Unlimited -RecipientTypeDetails SharedMailbox).count

#Output variable values to screen


#Output variables to CSv files


#export to a csv file
$table | Set-Content D:\MessagingMetricsReports\MailboxesByTypeNumber.csv

And the outputs looked like this:

On the screen:

In Excel:

Friday, April 25, 2014

Exchange 2010 rollup update installations for lazy people

Hey folks,

It is a wonderful cloudy Friday morning. So in this short posts I will share with you some of the commands I used to install RU5 for Exchange 2010 SP3.

I used this command for silent RU installation:

msiexec /update Exchange2010-KB2917508-x64-en.msp /qn /L* RU5InstallLog.txt .

First of all., it silently executes RU5 installation on server which allows you to have a cup of tea, rather than pressing Next button every time. Or even better you can use Group Policy in your AD or SCCM to automatically publish RU5 installation.

Pleas note /L* RU5InstallLog.txt parameter. This doesn't keep you blind on how your installation goes. But rather creates a log file which you can check for the details of your installation.

And finally msiexec /update Exchange2010-KB2917508-x64-en.msp /qn /L*v RU5InstallLog.txt creates a verbose log which is necessary if your installation is failing and you need to troubleshoot it as I did in

Enjoy! Happy Friday!

Reporting and Filtering Magic for ActiveSync in a Large Environment

Hi folks,

I was working on the reporting of ActiveSync devices that are used by Exchange users.

Client has provided me a script which wouldn't work crushing with the error as on this screenshot.

I was working based on this article . This perfectly works for a small environment or when Get-CASMailbox command is submitted without -ResultSize Unlimited parameter which is needed in the environment with more than 1000 mailboxes. In their production environment with more than 30K mailboxes this was a disaster and creating a variable for client mailboxes was failing with the same kind of error.

According to some of the articles in the internet this error could be fixed by tweaking web.config file for PowerShell virtual directory on Exchange servers which I didn't really want to do as I prefer not to mess with default config unless really needed and also raising change applying it across all Exchange estate wasn't really attractive to me because of long change management process (and don't forget Microsoft's recommendation that all Exchange servers should be configured in standard fashion across all the estate).

I tried using pipelining results of Get-CASMailbox -ResultSize Unlimited to Where-Object command with the filter. So my command started looking like Get-CASMailbox -ResultSize Unlimited |Where-Object {$_.HasActiveSyncDevicePartnership -eq 'True'} |Get-Mailbox
However this also didn't do magic and the evil "Remote client exceeded allowed maximum" would still appear.

To resolve it I decided to experiment a little bit. First of all it's worthy to know that Get-CASMailbox returns name for a mailbox which can be a good which I used as a value for the  Mailbox parameter of the Get-ActiveSyncDevice. I still used Get-CASMailbox -ResultSize Unlimited |Where-Object {$_.HasActiveSyncDevicePartnership -eq 'True'}. However this time I didn't pipeline it, but rather feed into the $EASUser variable which I used as an input for the Get-ActiveSyncDeviceStatistics command. And since client needed particular information about ActiveSync device, namely when it last talked to server, its type and ID and whether it can be remotely wiped.

So I ended up with the code as below (you can customize second line

$EasUser=Get-CASMailbox -ResultSize Unlimited |Where-Object {$_.HasActiveSyncDevicePartnership-eq 'True'}
$EasUser | foreach {Get-ActiveSyncDeviceStatistics -Mailbox $_.Name | select identity,LastSuccessSync,devicetype,deviceid,isremotewipesupported} |Export-Csv D:\MessagingMetricsReports\ActiveSyncUsageReport.csv

It resulted in the nice Excel Spreadsheet like this which was later sent to the happy customer. However since my production environment has more than 30K mailboxes and some users have more than a single ActiveSync device the file size was about 4 MB, so don't be scared if you get your report that big.

Wednesday, April 23, 2014

Mailbox Database Reporting Headache

Hi folks,

This morning I woke up with the terrible headache. Even sleeping one more hour wasn't very helpful, thus I came to the office.

Another headache was awaiting me. Preparing data for monthly messaging report which has been handed over to me.

One of the reports was the hardest. Preparing report on how mailbox databases are used. In particular client needs a unified report which includes information of the mailbox databases size, white space size and the number of users that actually provisioned to these databases and use all of this space.

I had 2 obstacles. First of all the way Get-MailboxDatabase command outputs information about mailbox database size and white space size like on this screenshot

It is nice but absolutely useless if you want to process data in MS Excel. So in order to output information about size only in digits you need to use expressions like these: @{N="AvailableNewMailboxSpace GB";E={$_.AvailableNewMailboxSpace.ToGB()}} and @{N="DatabaseSize GB";E={$_.DatabaseSize.ToGB()}}  for white space and total database space respectively.

Additionally my client needed the number of mailboxes in the database which is even trickier. After some exercises that even increased my headache I came up with this expression: @{N="Mailboxes Count";E={(get-mailbox -database $}}

And of course I wanted my report to look nicer as databases are arranged in the alphabetical order. Which was pretty easy to achieve by using Sort-Order Name command.

As a result I have got the following code:

$DBs = get-mailboxdatabase -Status |Sort-Object Name
$DBs | Select-Object name,@{N="AvailableNewMailboxSpace GB";E={$_.AvailableNewMailboxSpace.ToGB()}},@{N="DatabaseSize GB";E={$_.DatabaseSize.ToGB()}},@{N="Mailboxes Count";E={(get-mailbox -database $}} | export-csv D:\MessagingMetricsReports\SpaceMbxNumberReport.csv -NoTypeInformation

Which in turn produced a report like this:

And of course the names are sanitized for the sake of ethics.

Now it's time for me to have some aspirin, as my headache is still there. However I have got the script which can produce me the nice report any time I want it.

And of course any other database related attributes can be added to the code if more info is needed in the report.

Tuesday, April 22, 2014

Exchange 2010 Update Rollup Fails with error 1603

Hi folks,

Today when working on a simple task as installing Update Rollup 5 for Exchange 2010 SP3 I ran into a funny issue. On one of my lab servers it was failing with the error 1603.

After some googling i encountered this article . Following it does the magic, however notice couple tricks here. For Exchange 2010 RU to produce verbose logging run command like:

msiexec /update Exchange2010-KB2917508-x64-en.msp /qn /L*v RU5InstallLog.txt

You will need to replace name of MSP file in the command with the appropriate name

Also look in the log file for the appropriate MSP file. It may differ from the one specified in the article (it actually applies for Exchange 2007). In my case it was Exchange2010-KB2866475-x64-en.msp which corresponds to RU2 for Exchange 2010 SP3.

So I ended up downloading RU2 for Exchange 2010 SP3 and dripping it to C:\Windows\installer folder which the article has been suggested. However this wasn't successful as well.

I looked through the log again and discovered that the installer tried to look for c:\Exchange2010-KB2866475-x64-en.msp. I have ended up running command prompt as admin and copy the file there. After which the installation has successfully completed. And, of course, since it is RU, the installation sometimes takes ages. Weird placement for update files, isn't it.

One of the lessons learned. Keep your Exchange install and update files in the folder where you have put them originally as installer will go to those locations whenever you attempt to change something about the server setup.

Thursday, April 17, 2014

Reporting IP Addresses for Exchange Servers

Hi folks,

Let me share another case which made even more sure that PowerShell is really power shell.

My customer was looking to standardize DNS client settings across its Exchange servers. As the first step they wanted me to provide them with the information on how DNS settings are currently configured on Exchange boxes. Since Windows 2008 R2 Powershell is not as a rich as in Windows 2012, I had to do some head scratching and googling.

As a result I ended up having the script which connects to every Exchange server and output them in the nice Excel file format like CSV.

I had to overcome one thing though as when I was that values for such important parameters like IPSubnet or  DefaultIPGateway and most importantly  DNSDomainSuffixSearchOrder were outputted into CSV as ugly. "System.String[]". I overcame this by using expression like this @{Name="IPAddress";Expression={$_.IPAddress}} for each important network configuration parameter and in particular for DNSDomainSuffixSearchOrder which was my target.

Another caveat here is that Edge servers are not members of the AD and hence execution of Get-WmiObject against them will fail if we simply run it, so I had to filter them out from the Get-ExchangeServer command by the {$_.ServerRole -ne "Edge"} filter.

As a result I have ended up with a nice script as this one:

Get-ExchangeServer | Where-Object {$_.ServerRole -ne "Edge"} |foreach {Get-WmiObject -Class Win32_NetworkAdapterConfiguration -Filter IPEnabled=TRUE -ComputerName $_.Name} |Select-Object DNSHostName,Description,@{Name="IPAddress";Expression={$_.IPAddress}},@{Name="IPSubnet";Expression={$_.IPSubnet}},@{Name="DefaultIPGateway";Expression={$_.DefaultIPGateway}},@{Name="DNSServerSearchOrder";Expression={$_.DNSServerSearchOrder}},@{Name="DNSDomainSuffixSearchOrder";Expression={$_.DNSDomainSuffixSearchOrder}} |export-csv D:\scripts\Exchange_IP_Config_Reporting.csv

Which provides me with output as below (IP addresses and hostnames are not included for the sake of professional ethics)

This resulted in a happy customer which can make a decision on how DNS settings are to be configured.

Good Exchange 2013 books

Hi folks,

Looks like it took ages for good Exchange 2013 books to be published after release of this product. There are still no good books for 70-341 and 70-342 exams.

However, in October 2013 MS press has published 2 books in Exchange 2013 Inside Out series. One is written by Tony Redmond and another one by Paul Robichaux.

I have finished first book and highly recommend it, though it misses a subject of DAG autoreseeding feature which can be read in Technet. However, Tony quite clearly explains all the nuts and bolts of a new Mailbox server role with some good practical examples and PowerShell scripts.

Currently I'm in the 2nd chapter of Paul's book that covers new CAS role along with transport and UM. It is also quite a good reading, though I miss Tony's style and tons of script examples there. But it's still a good reading.

These are Amazon links where you can order these books (it's not advertisement, just a friendly advise)

Creating and configuring Receive Connectors in Couple Steps

Hi folks,

As you probably know Exchange server 2007 and later is managing receive connectors on a per server basis, rather than per organization. Thus if you need to create receive connector you need to do the same operation on each hub transport server (mailbox server in Exchange 2013). 

I needed to create a receive connector for an application on all hub servers in my lab environment for tests that listens on the port 465 and accepts traffic from subnet (where application servers reside) with the explanatory name which would distinguish it from the rest of receive connectors.

It doesn't really work for a lazy person like me. So I have played a little bit with PowerShell to simplify this operation and reduce it to 2 steps. 

In my lab setup I have 3 CAS/HUB servers and 1 edge server. Get-TransportServer outputs all transport servers including Edge servers. Therefore I went on and rather used Get-ExchangeServer command by filtering output by CAS/HUB role. I have piped output into the cycle that runs New-ReceiveConnector command against the output. Therefore my final command that worked looks like that:

Get-ExchangeServer |Where-Object {$_.ServerRole -eq "ClientAccess, HubTransport"} |  foreach {New-ReceiveConnector -Name ‘Connector Name’ -Usage 'Custom' -Bindings '' -Fqdn ‘’ -RemoteIPRanges '' -Server $_.Name}

The other part of the task was to configure permissions for the created receive connector to configure authentication for Exchange Users (as a user needs to be authenticated to be able to send emails via this connector) and there was a requirement for IPsec authentication for the traffic sent to this connector. Instead of going around from server to server and ticking multiple boxes I have ended up with one command that collects all the connectors with the name as specified in the previous steps and then configures the appropriate permissions for each of the connector on each server,

Get-ReceiveConnector| where {$_.Name -like ‘Connector Name’ } |Set-ReceiveConnector -PermissionGroups 'ExchangeUsers','ExchangeServers' -AuthMechanism 'Tls','ExternalAuthoritative'

Thus I ended up with 3 connectors on 3 hub transport servers in my lab environment which listen on port 465, accept traffic from the particular subnet and with the necessary permissions and authentication settings. 

And it took me only 5 minutes to create them automatically in running only 2 strings of the code.
My dear friends,

Finally I have found time to start my own blog in the internet. By this blog I want to share my IT skills and interesting tips and tricks that I have discovered while working in IT world.

I am a Senior Windows Engineer in IPsoft. My professional skills lay mainly with Microsoft's technologies such as Active Directory Domain Service, Exchange Server and I'm currently working on building up my Lync skills.

You are more than welcome to visit my blog to read about my developments, lab and production experience and literature that I find good for any IT guy reading.

Sincerely Your,

Farhad Mahmudov