When it comes to hybrid AD setup, we have to work with whole different types of issues than on-premises AD environments. Azure AD is a managed service by Microsoft, so there is nothing we can do to manage its health. Therefore, most of the hybrid AD issues are related to connectivity, Directory sync or authentication methods (password hash, pass-through authentication, federated). The main component which connects on-premises Active Directory environment with Azure AD is Azure AD Connect. So most of the issues in hybrid environment can also related to Azure AD Connect. In this blog post, we are going to look in to some of the most common Azure AD connect issues and learn how we can recover from those.
Connectivity
Azure AD Connect requires connectivity to Azure AD to do the directory synchronization. Azure AD connect server also need to be able to communicate with on-premises Active Directory Domain Controller. When there is directory synchronization issues, we will see following symptoms.
• New user accounts added in on-premises Active Directory, does not appears in Azure AD or taking long time to appear (more than 30 minutes ).
• After on-premise user change their password, he/she cant authenticate to Azure AD.
• If password-writeback feature is been used, password reset in Azure AD does not work for on-premise users.
• We can see synchronization errors under Azure AD Connect health.
• When there are directory sync issues, Azure AD will also send email notification to directory administrators.
There can be many reasons for connectivity issues. But below i listed down most common reasons for connectivity issues.
• Connection to Azure AD: Azure AD connect server should have stable connection to URLs, IP addresses and port numbers (TCP 80 and 443) listed on https://docs.microsoft.com/en-gb/office365/enterprise/urls-and-ip-address-ranges?redirectSourcePath=%252farticle%252fOffice-365-URLs-and-IP-address-ranges-8548a211-3fe7-47cb-abb1-355ea5aa88a2. You can simply verify the connectivity by using Telnet.
• Connection to On-premises domain controllers: If you have firewall between Azure AD connect server and Domain Controllers, make sure you have following ports are open:
Protocol |
Ports |
DNS |
53 (TCP/UDP) |
Kerberos |
88 (TCP/UDP) |
MS-RPC |
135 (TCP/UDP) |
LDAP |
389 (TCP/UDP) |
SMB |
445 (TCP/UDP) |
LDAP/SSL |
636 (TCP/UDP) |
RPC |
49152 – 65535 (Random high RPC Port)(TCP/UDP) |
• Proxy in use: If Azure AD connect communicate with Azure AD via a proxy, make sure C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config has the proxy configuration in place.
<system.net>
<defaultProxy>
<proxy
usesystemdefault="true"
proxyaddress="http://<proxyserveraddress>:<proxyport>"
bypassonlocal="true"
/>
</defaultProxy>
</system.net>
This should be at the end of the file. If proxy server has been changed, you also can change this by running Azure AD connect configuration wizard.
• DNS issues: In order to make connection to Azure AD as well as on-premises Domain controllers, Azure AD connect server should be able to resolve internal and external host names (FQDN). If the firewall or proxy connectivity is not preventing communication, check if you have health DNS resolution.
Synchronization errors
Apart from connectivity issues, AD object synchronization issues also have major impact on health of the Hybrid AD environment. Let's go ahead and look in to some of the common synchronization issues.
Duplicate attribute
Let's assume we have two users in on-premises AD called John Smith and Jason Smith. Their email address is set to j.smith@rebeladmin.com for both users ( first name initial and last name ). Since proxyaddress attribute is same for both user objects, Azure AD will give Duplicate Attribute error. This can be easily fixed by removing or modifying duplicate attribute value on-premises user.
Data mismatch
In the Azure AD Hybrid environment, when a new object is added or existing object been updated in on-premises Active Directory, it needs to sync back to Azure AD. This is done by Azure AD Connect. During the sync process, two attribute values has been compared to check if it is a new object or existing object for Azure AD. These two attributes are called as sourceAnchor and immutableId. Even it is two attributes, it refer to the same value. When an object comes from Azure AD connect, we refer to this particular object value as sourceAnchor. The same object value in Azure AD, we call as immutableId. The word immutable means cannot change. Similarly, once the immutableId value is defined in Azure AD, it cannot be changed.
When AD sync is initiated from on-premises AD, Azure AD compare sourceAnchor attribute of objects with immutableId attribute of objects. If it matches, we call it as hard match. If Azure AD cant find match it means the object is not existing in Azure AD. In that situation Azure AD will treat it as new object. However, just before Azure AD create new object, It checks if this user object's ProxyAddresses and UserPrincipalName attributes match object already in Azure AD. This is to make sure there is no duplicate objects in Azure AD. This match is called as soft match. Sometime, in hybrid AD environment, there can be objects which doesn't have hard match but does has a soft match. This type of errors are logged in Azure AD as InvalidSoftMatch error. There can be many reasons which can cause a InvalidSoftMatch errors such as:
• There are multiple objects exist in on-premises Active Directory with same ProxyAddresses or UserPrincipalName attributes. In such situation Azure AD can only create one object.
• A new object is added to on-premises Active Directory with same ProxyAddresses or UserPrincipalName attribute but Azure AD already got a object with same values. In such scenario, this new object will not sync to Azure AD.
• An already synced object moved from one Active Directory forest to another. This will change the sourceAnchor attribute of the object.
• Azure AD connect server reinstalled or moved to a different server with new sourceAnchor attribute value.
• Active Directory user account deleted and recreated with the same attributes before Azure AD remove the deleted account. It will also create InvalidSoftMatch error.
We can fix the InvalidSoftMatch errors using following methods,
• Identify the object with conflicting ProxyAddresses or UserPrincipalName attributes. If both of these objects need to sync with Azure AD, change those attributes to unique values.
• Exclude the objects with conflicting ProxyAddresses or UserPrincipalName attributes from the sync scope if not every object need to sync with Azure AD.
• Some of the conflicting object may not be need to exist in Active Directory any more or the conflicting value may not need to associate with object anymore. In such situation we can remove the object or update the attribute value.
All above changes should be done in on-premises Active Directory and let Azure AD Connect sync the changes.
In some scenarios, the objects involved in soft match is not belongs to the same object type.
As an example there can be a user account and a contact with the same ProxyAddresses value. This is called as ObjectTypeMismatch error. This errors can fix using the same methods listed above.
Data validation failure
Azure AD do not accept every data coming from Azure AD connect to be written in to the directory . Azure AD is a managed service, so it has it own restrictions to maintain the data integrity. As an example for UserPrincipalName of an object, Azure AD has following restrictions.
• userPrincipalName must be in email address format with @ sign.
• userPrincipalName only should have maximum of 113 characters. Out of 113, you should only have 64 characters before @ sign and only 48 after @ sign.
• Azure AD do not accept \ % & * + / = ? { } | < > ( ) ; : , [ ] " ' for userPrincipalName
• userPrincipalName cannot have space.
• userPrincipalName should only use routable domain names. It should not have local or internal domain names.
If the incoming object do not comply with about restrictions, it will give Data Validation Failure error.
Large Attribute
Similar to on-premises Active Directory, Azure AD also have size limits for objects, which is defined by schema. If an incoming object exceeds these limits, Azure AD will issue a LargeObject or ExceededAllowedLength error. This happens mainly for following attributes:
• proxyAddresses
• userCertificate
• thumbnailPhoto
• userSMIMECertificate
As an example, the maximum number of proxyAddresses accept by Azure AD is 200.
Existing Admin Role Conflict
If Azure AD administrative role holder has soft match with userPrincipalName attribute, Azure AD will issue a Existing Admin Role Conflict sync error. This error can fix by removing administrative role from the conflicting object. This will resume the replication and later we can create separate account for administrative tasks.
In this section, I have only listed down most common issues related to Azure AD connect. Apart from that authentication issues, single-sign-on issues, design issues also can make direct impact on the health of Azure AD hybrid environment.
Azure AD connect has built in tool for troubleshooting to cover most of these scenarios. Based on the symptoms, the tool will run pre-built tests and provide insights which help to find the root cause.