Tuesday, March 27, 2018

Exchange Admin Center can’t handle duplicate display names

If you’re managing a larger organization, or even a smaller one, chances are that some of your coworkers share the same name. From an Exchange perspective this is not supposed to be an issue as the displayName attribute does is not required to be unique. End-users who try to locate the right person in their address book can look at the additional properties such as department or email address to find the correct ‘John Smith’ or ‘Pradeep Gupta’.

The Exchange Admin Center (EAC) in Exchange Online unfortunately does not handle this very well. There’s many places where an admin picks a user from the list, for instance to create a condition or exception in a mail flow rule, but where EAC uses the display name to identify the Exchange object. Which is not a problem, until there happens to be more than one object with that display name.

image

There are multiple recipients matching the identity "Display Name". Please specify a unique value.

In case you’re wondering, the ‘click here for help…’ link goes to a generic error landing page.

Microsoft is aware

I worked with Microsoft Premier Support on this issue and after a lot of communication (“why don’t you use a group instead of a user, put that user in a group”) we were just about to file a DCR, until we were informed that the product group is already aware of this design flaw. According to Premier Support a fix is scheduled but without an ETA.

I can only assume that this will require significate code changes so expect that this will be part of a next major redesign of the EAC. After all we have seen very little updates in this area since the last 5 years.

If you run into this issue, please work with Microsoft Support to gain more visibility on this issue. The PG needs to be aware of that this is a proper bug and every time and admin runs into this issue, it costs a lot of time and effort to work around it.

Work around

That said, there is of course a way to work around the issue. The bug is caused by the way that EAC handles the objects, but this does not apply when using PowerShell directly.

In case of creating or editing a mail flow rule the work around is to select a placeholder object with a unique display name and then edit the properties in Exchange PowerShell with Set-TransportRule.

Repro steps

If you’re interested in this bug, follow these steps to reproduce it on your own tenant:

  • Create two mailboxes and a transport rule

New-Mailbox -Shared -Name displayname1 -DisplayName Shared
New-Mailbox -Shared -Name displayname2 -DisplayName Shared
New-TransportRule "test rule to reproduce issue" -SubjectContainsWords "thisneverhappens" -RejectMessageReasonText "rejected because of the following Mail Flow Rule: test rule to reproduce issue"

  • Add both mailboxes as an exception to the rule

Open de Mail Flow Rule in EAC and add an exception "The sender is this person". In the people picker, select the one of the two test mailboxes and click Add, Ok. Now click Save to reproduce the error.

What about Exchange Server?

My troubleshooting was focused on Exchange Online, but I assume it’s in Exchange Server as well. Please let me know if you can confirm.

Tuesday, February 27, 2018

Office 365 Phone System numbers, how to convert from User to Service?

When porting numbers to Office 365 Phone System, the admin has to specify if the number should be a service number or a user number. The same applies when new numbers are requested. What’s the difference between the two number types, you may ask. The answer is easy.

A user number is assigned to a single user.

A service number is connected to a call queue, a conference bridge or an auto attendant and is capable of handling simultaneous calls.

A common issue for new Office 365 Phone System customers is where they port an entire number block as user numbers, including the organization’s main number. Soon they discover that the main number should’ve been used with a Call Queue or an Auto Attendant but those features require a the number to be configured as a service number instead.image

If you’re searching for information on how to convert the number type you’ll find many outdated articles stating that this is not possible, requires opening a support incident or recommend to port the number away from Microsoft and then port it back.

The situation has now, end of February 2018, much improved. All you have to do is send an email to Microsoft’s Skype for Business Number Porting Team. They need the following information to be able to process the request:

  • The number that needs to be converted
  • The requested number type: user or service
  • When do you want the conversion to take place?

My experience with a number that’s currently not assigned, so no downtime to communicate with the end users, is that it was converted within 30 minutes after confirming the details through email.

To start the process, simply send an email with the required information to ptn@microsoft.com for North America or ptneu@microsoft.com for Europe.

Please let me know in the comments if you’re experience was as smooth as mine.

Friday, February 23, 2018

Disable voicemail on a Skype meeting room accounts

image of a Surface Hub

If you ever had to create user accounts for modern meeting room devices such as Microsoft’s Surface Hub or 3rd party devices such as the Polycom Trio 8800 you may have noticed that process is basically the same. Create a Room Mailbox with an enabled account and a password, then enable the account for Skype for Business with the Enable-CsMeetingRoom cmdlet.

Recently a coworker transferred a Skype call to a meeting room and when the person in the room wasn’t quick enough to respond, the call ended up in the conference room voicemail box. Understandably she requested us to disable voicemail on this account.

After investigation we found that Azure Voicemail was enabled when we added licenses for Office 365 Phone System and Calling Plan. Disabling voicemail is easy. Either sign-in with the username and password of the meeting room and visit the web based advanced settings: https://aka.ms/vmsettings. Then clear the checkbox for Activate Voicemail.

image

Alternatively an admin can disable the voicemail feature in Skype for Business Online PowerShell:

Set-CsMeetingRoom <SIP address> -HostedVoiceMail:$false

Reference:

Check Skype for Business voicemail and options

How to Set up and configure Cloud Voicemail for Skype for Business Online users

Friday, February 16, 2018

HCW does not ‘remember’ mail transport configuration

Centralized mail transport (CMT) is a hybrid mail flow scenario where all outbound email from Exchange Online is routed through on-premises servers first before sending it to the internet. It’s not really common but there are organizations with specific requirements that can be met with centralized mail transport.

As you probably know, when the HCW is run for the first time, the entered configuration is stored on the local hybrid configuration AD object. When an admin starts the HCW again, the HCW will read the configuration form the AD object and use it to pre-populate the wizard with the information that was entered previously.

Last week I was working with a customer with an Exchange 2013 CU19 environment where we built a hybrid configuration. When everyting was setup and tested I re-ran the HCW to add more email domains. In this specific instance I noticed that the checkbox for centralized mail transport was not present:

image

Continuing with the wizard would effectively break mail flow for this customer because the HCW would disable the centralized mail flow feature and route outbound messaged from Exchange Online to the internet directly.

First I verified the actual configuration in EOP and found that is was set up correctly:

image

Next I verified the information stored in the local hybrid configuration AD object and found that CentralizedTransport was listed under Features:

image

After opening a support incident with Microsoft Support the engineer confirmed that this behavior was ‘by design’. I won’t bother you with the argumentation because it made no sense at all. Unfortunately this customer did not have a Premier Support and there were very limited options to escalate the issue.

This is bad and I can think of at least three reasons why. First of all this feature by default is not displayed on the HCW page. The admin has to click on Advanced… for the CMT configuration to appear on the HCW page:

image

This means that it’s almost impossible to spot the problem. Second problem is that this behavior is simply inconsistent, it doesn’t make sense that the HCW caches all information EXCEPT thing single setting.

But third and biggest problem that I have with this issue is the impact. When discussing this with Microsoft the engineer told me that the impact is minimal because outbound email will still be delivered to the receiving mail server. This demonstrates a huge lack of understanding of why organizations choose to use CMT. Having to explain to your legal department why sent emails are missing from the archive is not a fun job.

And that’s one other problem that I have with this experience. The engineer did not investigate the HCW log files, did not agree with the fact that this is an impacting issue and an inconsistency and did not offer any escalation path. This reminded me of how difficult it can be to work with Microsoft Support if you do not have access to higher support tiers or a TAM.

I realize this is a rare scenario, but if you’re using CMT and run into the same issue, please open a service request and escalate if possible. This should be fixed.

Wednesday, October 4, 2017

Project Honolulu and Exchange servers

Recently Microsoft announced a new web-based server management console, it’s called Project Honolulu. Looking at the documentation and screenshots this is going to be a huge step forward from the native sever management consoles that we have seen from Server 2008-2016.

A technical preview of Project Honolulu is already available for download: https://aka.ms/HonoluluDownload Project Honolulu can be used to manage servers with Windows Server 2012, 2012 R2 and Server 2016. However, there is a requirement to install PowerShell v5 on those older servers. From the documentation:

Honolulu requires PowerShell features that are not included in Windows Server 2012 and 2012 R2. If you will manage Windows Server 2012 or 2012 R2 with Honolulu, you will need to install Windows Management Framework (WMF) version 5.0 or higher on those servers.

Unfortunately Microsoft does not support installing newer versions of WMF on servers that have Exchange installed. The Exchange Supportability Matrix says the following about the supported versions of WMF on servers running Exchange:

Version of Windows Management Framework built into the release of Windows Server you're installing Exchange on.

This means that installing PowerShell 5 on Server 2012 or Server 2012 R2 is not supported for servers running Exchange, effectively making it impossible to use Project Honolulu to manage Exchange servers. The only exception is Exchange 2016 running on Server 2016 because this OS was released with PowerShell v5 natively and meets the Project Honolulu requirements.

Monday, May 1, 2017

Exchange Remote PowerShell behind a proxy server

Today I ran into the following issue. I tried to open a Remote PowerShell session to Exchange Online but the module failed to load with the error message:

New-ExoPSSession : Create Powershell Session is failed using OAuth

image

Initially I suspected that the customer’s firewall performed some kind of traffic inspection or recognized and denied the traffic as a blocked ‘application’. I was able to browse the internet from this computer and after all Remote PowerShell is just https traffic.

However, the culprit was the proxy server. More specific, I configured a proxy server in the Internet Explorer settings but did not verify if Remote PowerShell observed the same connection settings. And the answer is no, the console does not use the IE settings but falls back to the winhttp configuration. In this environment the firewall configuration did not allow outbound traffic unless it passes through the proxy.

Telling PowerShell to use the proxy server is easy. Open an elevated prompt and use the following command to configure winhttp to use a proxy server and port:

netsh winhttp set proxy proxy.network.tld:8080

To import the settings that where configured in IE:

netsh winhttp import proxy source =ie

And important, to remove the proxy configuration for winhttp:

netsh winhttp reset proxy

image

Configuring the proxy for winhttp fixed my issue and I was able to connect to Exchange Online with PowerShell successfully.

image

Tuesday, April 11, 2017

Some feedback on the new Exchange PowerShell cmdlets

As someone who has been working pretty much dedicated with Exchange since ~2000 I have witnessed quite a few changes. The single most important one is the move to PowerShell as the primary management interface. The Exchange team was the first within Microsoft to switch for the full 100% to PowerShell and at least two version of Exchange have seen their release slightly delayed because they had to wait for the PowerShell version GA first.

While all other products continued to implement PowerShell as well and used the same set of professional standards there were significant differences to notice between the various implementations. I always preferred the way how the Exchange team implemented PowerShell. Identity is always positional parameter 1, Get cmdlets without a parameter return the first 1k of results or whatever is the default value of ResultSize for that cmdlet and destructive cmdlets (Set, Remove, New) always support the -WhatIf switch. I’m sure that every Exchange admin once tried to run Get-ADuser and discover that the cmdlet doesn’t return any values unless you specify a value for -Filter.

Personally I always preferred the consistent and user friendly implementation in Exchange. Maybe that’s why it bothers me that Microsoft no longer applies the same standard to new cmdlets that are being added in Exchange and Exchange Online.

An example is Get-FoucesedInbox and Set-FoucesedInbox:

image

The first example shows that the -Identity parameter no longer is a the first positional parameter, a positional parameter can be omitted and PowerShell assumes the first value after the cmdlet to be meant for the first positional parameter. This can be a bit annoying because Exchange admins are no longer used to specify the -Identity parameter because they never had to.

What’s more serious is that Set-FoucesedInbox no longer accepts the -WhatIf switch. -WhatIf tells the cmdlet to perform all prerequisite checks but not to make the actual changes. This parameter is essential for admins to check their syntax before running the actual cmdlet.

Some other examples of cmdlets that did not receive the -WhatIf switch are:

Cmdlet

Exchange

Exchange Online

Set-Clutter X X
Set-CompliancePolicySyncTenantInfo X X
Remove-BlockedSenderAddress   X
Remove-CompliancePolicySyncNotification X X
Set-OMEConfiguration   X
New-ReportSchedule   X
Set-ReportSchedule   X

As you may notice they were all added in the last few years.

While I understand that the industry has changed and Microsoft’s priorities are different in the ‘cloud first’ era, I do urge the Exchange Team to keep focus on what made them successful in the first place: excellent quality and always strive to deliver the best. Please do not forget to think of the people that are actually working with the product.

Friday, March 24, 2017

Let’s not run Exchange 2016 Edge Transport on Windows Server 2016…

Another example where old technology meets new technology. Old is in this case the Content Filtering agent on an Edge Transport server, new being Windows Server 2016. In short, the SmartScreen technology that is used in the Content Filtering agent conflicts with SmartScreen implementation in Windows Server 2016 operating system.

At this point this is known to prevent a proper uninstall of Exchange and can cause crashing transport services, this applies to an Exchange 2016 Mailbox server where the admin installed the anti-spam agents too.

From today Microsoft no longer recommends to install the Exchange 2016 Edge Transport role on Windows Server 2016, the same goes for installing the anti-spam agents on the Mailbox role when installed on Windows Server 2016.

What do you need to know about Exchange 2016 on Windows Server 2016:

An interesting note is that the author of the post on the Exchange Team Blog consistently uses the term ‘Edge role’ instead of Edge Transport. This could be one more change in the branding but more likely just a mistake.

Thursday, March 16, 2017

New Exchange Online PowerShell module with MFA support no longer in Preview

In a previous article I wrote about the new PowerShell module with support for Modern Authentication and Multi-factor Authentication. The Preview status was a reason for many organizations to hesitate to take the new module into production.

Yesterday Microsoft released an updated version of the Hybrid Configuration Wizard with MFA support, the HCW now requires installation of the new PS module to support MFA. Microsoft’s ‘hybrid’ PM Timothy Heeney confirmed in the comments section that this also marks the official RTM of the new PowerShell module, the module is no longer in Preview. Good news!

Tuesday, March 7, 2017

Surface Hub devices and the Skype for Business Trust Model

I’m sure that most Lync or Skype for Business admins, users as well, are familiar with the Trust Model. The Trust Model is responsible for the ‘Skype for Business cannot verify that the server is trusted for your sign-in address’ warning. This warning is thrown when the clients tries to create a secure TLS connection with a server and the domain suffix of the server is different from the user’s SIP address, the server can be either a Skype for Business or Exchange server. This warning is very common in organizations that use more than one SIP domain and is often suppressed on managed computers with the TrustModelData registry value.

With a Surface Hub device the issue is a bit more complicated to determine, but very easy to work around. In this article I will explain how.

Let’s consider the following scenario. Contoso is an enterprise organization that uses many different SMTP and SIP domains across their divisions. The AD domain name is contoso.com and this is also the DNS domain suffix for most servers. Skype for Business is hosted with a 3rd party named Fabrikam, their servers have a fqdn with the fabrikam.com suffix.

image

The Northwind Traders division of Contoso has purchased a Microsoft Surface Hub device and created a device account with a SMTP, UPN and SIP address with a nwtraders.com suffix.

The issue

An admin was able to configure the Surface Hub with this computer account, however users are not able to start a meeting.

It’s important to understand that although the device boots successfully, the built-in Skype for Business client is not immediately connecting to Skype for Business (Online) but starting a meeting does trigger this process.

The investigation

Surface Hub devices run on Windows 10 Team edition which does not offer a regular interface that allows to access the file system to collect log files. Instead we need to boot the device, let it run for 5 minutes, then reproduce the issue and tell the Surface Hub to collect the log files.

To do this, connect a USB disk to the device and open the Settings app. Then navigate to Update and Security, Recovery, Collect logs. The log files are now written to the USB disk.

When analyzing the log files, be aware that the Surface Hub’s Skype for Business client is very similar to the Lync 2010 Windows Store app and behaves as a mobile client.

2757 TL_WARN() [2]10B0.1394::02/28/2017-12:38:54.103.00000a8c (NONE,NModel::CTrustModelManager::LookupTrustModel:CTrustModelManager_cpp124)<O_TRC><ADR>0x00000243A9F16EB0</ADR>Trust model for server rp.contoso.com not found. hr=0x80ee0058</O_TRC>
2758 TL_WARN() [2]10B0.1394::02/28/2017-12:38:54.103.00000a8d (NONE,NModel::CTrustModelManager::QueryTrustModel:CTrustModelManager_cpp171)<O_TRC><ADR>0x00000243A9F16EB0</ADR>Server: rp.contoso.com cert=0000000000000000, blockAndWait=0</O_TRC>
2759 TL_INFO() [2]10B0.1394::02/28/2017-12:38:54.103.00000a8e (NONE,NModel::CTrustModelManager::QueryTrustModel:CTrustModelManager_cpp230)<O_TRC><ADR>0x00000243A9F16EB0</ADR>Not able to get SAN from cert. Continue query TrustModel.</O_TRC>

Here we clearly see the issue. The DNS domain suffix of the reverse proxy server is contoso.com and the user’s SIP address suffix is nwtraders.com. This triggers the Trust Model warning and because the Surface Hub interface does not present the familiar warning, it simply prevents the device from connecting with Skype for Business.

The solution

As I mentioned earlier, this is a very common issue for most organizations. The Surface Hub device offers an interface to add domains to the Trusted Domain list. Open the Settings app and navigate to This device, Calling. Here click the Configure domain name and enter a comma separated list of the additional domain names that exist on your Skype for Business and Exchange servers.

image

In this scenario we would need to enter the DNS suffix of the reverse proxy, but that’s not sufficient. While this will allow us to connect to the reverse proxy this will throw another warning in the logs because the DNS suffix of the front-end server is different from the user’s SIP address suffix too. In this example we would need to enter the following:

contoso.com, fabrikam.com

A reboot of the device is required to activate the new settings. If you’re still not able to connect, export and analyze the logs again. There may be additional issues that prevent the device from connecting to Skype for Business.

Summary

Instead of showing a warning popup the Surface Hub simply does not allow to connect when the domain name of a servers is different from the SIP domain. If you know that this scenario applies in your organizations, add the additional domains in the Settings app.

For more information please see: