Email Phishing Protection Guide – Part 12: Discover Exactly Who is Attacking Your Office 365 User Identities

The Email Phishing Protection Guide is a multi-part blog series written to walk you through the setup of many security focused features you may already own in Microsoft Windows, Microsoft Office 365, and Microsoft Azure. By implementing some or all of these items, an organization will increase their security posture against phishing email attacks designed to steal user identities. This guide is written for system administrators with skills ranging from beginner to expert.

Email Phishing Protection Guide:

Introduction: Email Phishing Protection Guide – Enhancing Your Organization’s Security Posture

Part 1: Customize the Office 365 Logon Portal

Part 2: Training Users with the Office 365 Attack Simulator

Part 3: Deploy Multi Factor Authentication (MFA)

Part 4: Deploy Windows Hello

Part 5: Define Country and Region Logon Restrictions for Office 365 and Azure Services

Part 6: Deploy Outlook Plug-in to Report Suspicious Emails

Part 7: Deploy ATP Anti-Phishing Policies

Part 8: Deploy ATP Safe Link Policies

Part 9: Deploy ATP Safe Attachment Policies

Part 10: Deploy and Enforce Smart Screen for Microsoft Edge, Microsoft Internet Explorer and Google Chrome

Part 11: Monitor Phishing and SPAM Attacks in Office 365

Part 12: Discover Who is Attacking Your Office 365 User Identities

Part 13: Update Your User Identity Password Strategy

Part 14: Prevent Brute Force and Spray Attacks in Office 365

Part 15: Implement the Microsoft Azure AD Password Protection Service (for On-Premises too!)

Part 16: Disable Office 365 Legacy Email Authentication Protocols

Part 17: Control Application Consent Registrations in Microsoft Office 365 and Microsoft Azure

Part 18: Increase Security with Microsoft Secure Score

Part 19: Email Phishing Protection Security Checklist

Part 20: Recommended Security and Anti-Phishing Training from Microsoft Ignite 2018

Part 12: Discover Exactly Who is Attacking Your Office 365 User Identities
Within this blog series we have setup over ten locks that an attacker using phishing emails must overcome – reducing the likelihood of a successful attack. While Part 11 of this series highlights many of the reports available to review detected SPAM, Phishing and other types of detected malicious activity, another great nugget of security information is the availability to see just how often attackers are trying to logon as your users. And, if they were successful or not. Whether you are a small five person organization, medium 500, or large 50,000 user enterprise, the information available in this blog is both eye opening and sometimes alarming for technical and security teams.

Using reporting available in the Azure Active Directory Premium P1 subscription or a 30 day trial for P2, you will quickly gain valuable insight into threats against your organization. This blog will explain how to:

• See which of your user accounts are the most heavily attacked in terms of who is trying to guess their usernames and passwords. Some user accounts, such as those for a leadership team, are often more attacked than others.
• Identify if a user account has been breached and take quick action.
• Download detailed information in a Microsoft Excel or CSV file format for data sorting.
• Sort information about where these attacks are coming from based on country location.

Let’s move forward with how to get access to this data and review the results…

Setup License to Access User Identity Attack Information (Including a 30 Day Trial)

Azure Active Directory is available in four editions (Free, Basic, Premium P1 and Premium P2). To see a comparison of each edition, review a table available in this link. Detailed information about each edition is available within the document What is Azure Active Directory?

The information we want to review in this blog is available with a license for Azure Active Directory Premium P1 or P2. If you do not have this as part of your subscription, I have included step by step information below about how to setup the free trial license to see this information.

Let’s first see what type of Azure Active Directory License your organization has using the steps below.

1. Logon to portal.azure.com as a global administrator
2. In the index of features on the left, select Azure Active Directory
3. By default, Overview will be highlighted
4. Which license do you have?
   1. In the Overview section, if you see the screen below with the text “To see sign-in data, your organization needs Azure AD Premium P1 or P2. Start free trial.” then proceed to the next section to enable the 30 day trial license.

      

   2. In the Overview section, if you see the screen below with sign-in event information for your organization then you have the license needed to view the data.

      

Setup Free Trial of Azure Active Directory Premium P2

If your organization does not currently have Azure Active Directory Premium P1 or P2 to view logon information, use the steps below to setup a free 30 day trial. Otherwise, continue on to the next section to view the information.

Although Azure Active Directory P1 is needed to view the information, the trial is for P2 which is why we are setting up the subscription for P2.

1. In the screen above (step 4a), click on the link to Start a free Trial.
2. You now see two trial options available. One for Enterprise Mobility and Security E5 (a future blog) and second option for Azure AD Premium P2. Click on the Free Trial in Azure AD Premium P2.

   

3. In the next screen, information about the trial is displayed. After reviewing the information, click Activate.

   

4. Within several seconds, a notification of a successful activation for this trial is displayed in the upper right corner.
5. With the trial subscription now activated, in the same Azure Portal screen click on the Azure Active Directory option in the index on the left side of the screen. You will now see sign-in information for your Azure AD users in Office 365.

View Office 365 Sign-In (and Attempted Sign-In) Information

1. When logged in to the Azure Portal at portal.azure.com, click on Azure Active Directory in the left index area.
2. Now displayed is a summary graph of Office 365 User Sign-In information. Click directly on the summary graph to view detailed information.

   

3. We now start to see some interesting details such as the user sign-in, what application was used to sign-in, was a conditional access policy applied (future blog) and was Multi-Factor Authentication (MFA) required? In my demo tenant data below, notice my global administrator account required MFA but a user named Megan Bowen did not.
4. Depending on your organization size, there may be a great deal of logon activity to scroll through. From this screen, you can review Successful and Failed logon events by choosing the options in Sign-In Status.
   
5. In this view, we are still not seeing all the information available. Let’s make this a bit more interesting and gather detailed information to review the points highlighted in the beginning of this blog.

Download Detailed Sign-In Events

1. Within the same Azure Portal view as above, click on the Download link

   

2. The detailed information downloads in a CSV format by default.
3. Open the file in Microsoft Excel. I usually start by sorting entries at the top of each column. Do this by highlighting a cell in row one where the column data heading is. Then select Sort &Filter on the Excel ribbon in the upper right. Then select Filter in the drop down options.

   

4. With a drop down filter now added for each column heading, we can start to review data logged for suspicious logon activity. Below, I set the stage a bit more for the data shown from this fictitious company.

   Scenario: As we begin to review the sign-in information, consider the scenario of your own tenant. Are you a small business who operates in one specific region, a State, a Province, etc. Perhaps you are a mid-size company or even an enterprise company where you have traveling sales people. For your organization, you will most likely know your users and where they typically operate from. Even more importantly, you will know where your organization does not operate and therefore be able to identify suspicious logon activities from regions of the world where your users would never travel to.

   The data presented in this blog is fictitious from a demonstration tenant in Office 365, but represents user logon names, applications, locations, etc. data you may see in your logs.

   The data presented here has been modified a bit for our scenario to represent a small-mid size company operating in the Maryland area of the United States.

   Note: The IP addresses in this list have been obscured, but know that you will get exact source IP addresses listed in your data. I also removed a few columns not needed for this type of data review.

5. At a first glance at this data, I see what looks like normal logon information. The sign-in information from cities in Maryland are normal, the usernames look right, there are successful logon events, etc. We even see that some users have Multi Factor Authentication (MFA) enabled while others do not.

   

6. While the information displayed so far appears normal, let’s see where other logons are coming from. In my column F, I’ll select the filter drop down option to display the location values in this long list. Now this starts to get interesting.

   Remember my demo tenant scenario of being a small-mid size organization in the Maryland state area. I know my users do not travel as part of their work outside of the region and while it is possible an employee or two may be traveling the world on vacation, it is not likely.

   So looking at the now sorted data, I now see attempted logon attempts from a long list of cities and countries where I shouldn’t be seeing any activity from.

   

7. I want to research more information about these logons and will therefore go through the list of locations and deselect all of the Maryland and surrounding areas found in the list above, leaving me with mostly international locations. Now this really starts to get interesting in the list below. The frequency and widespread logon attempts from attackers attempting to access your network is now in full view.
8. In addition to understanding where logon attempts are coming from to access your organization’s resources, there is additional information to review:

a.   I also want to see the Failures listed in the Sign-in Status column. In the screen capture below where we are sorted on locations other than where our organization operates, also sort on the Sign-In Status column to see where Success is located. From here, be sure to look for any Success status listed for suspicious locations. If seen, you need to assume you have a breach in your organization and take appropriate action to protect your organization – starting with the user account that appears to be breached in this report.

b.   A scan of the data will also reveal who your most attacked users are. Is it someone from your leadership team with financial approval, is it a webmaster or admin email account that has been published on the Internet, or even an info@domain.com address listed on your website? This data should be used to generate a report to the leadership team of your organization with a recommendation of actions to take (such as enable MFA for all users).

This is Great Information, But What Can You Do Now?

As I stated early in this blog, this information is both interesting and may be alarming to find out just who and how often your organization is experiencing suspicious logon attempts. This information is a great way to see where logons are being attempted from as well as to see if they are successful.

Remember, an attacker has all the time in the world to guess a username and password. It only takes one successful match for a breach to take place. You can do several tasks outlined in this blog series to make it more difficult for an attacker to attempt logons to your organization such as blocking logons from locations other than where your users normally operate. Also, you can enable MFA on all user accounts so that even if an attacker is successful with a username and password combination, they will not be able to gain access to the environment.

Part 3: Deploy Multi Factor Authentication (MFA)

Part 5: Define Country and Region Logon Restrictions for Office 365 and Azure Services

This data is being presented as part of a 30 day trial license to Azure Active Directory Premium P2 for some readers. If on the trial, consider subscribing to this license to periodically review this data and further protect all of your users. The data collected in this report using the P2 license is just one of many features available with this subscription as described in this site.

Summary

This blog series highlights many items available in Office 365 and Microsoft Azure that can be implemented to help increase the security posture of most organizations. This blog explains how to use logon information from Microsoft Azure Active Directory to view where attempts to logon to your environment are coming from.

Remember, the best posture you can take to protect the security of your organization is to assume breach. Using data such as this will enable you to help find where attackers have breached your organization and how to take action to prevent it in the future.