NDUS Office 365 - May be unable to log in or applications (Mail, Teams, OneDrive) may not be showing up.

Resolved
Resolved

Final status: After extended monitoring and receiving confirmation from previously affected users, Microsoft validated that adding further processing throughput successfully resolved the issue.

Scope of impact: Impact was specific to some users who were served through the affected infrastructure. The majority of reported impact stemmed from users located in North and South America.

Preliminary root cause: High resource utilization on the infrastructure which supports the affected services resulted in the end-user impact.

Next steps:

  • Microsoft is continuing to investigate the underlying cause of the high resource utilization which led to the impact.
  • Microsoft is monitoring service telemetry to ensure impact doesn't recur.
Updated

Current status: Microsoft has completed the full reversion of the service update that occurred on Monday, April 17, 2023, and are monitoring the environment to see if this resolves the issue. Additionally, they continue the preparations for adding further processing throughput within the environment in an attempt to provide relief. This is approximately 50 percent complete, and expect this process to complete in full in the next hour.

Next update will be at 4:00 p.m. CDT

20793555

Updated

User impact: Some users may be unable to view or access Microsoft 365 apps or services

More info: Impacted services may include, but are not limited to:

  • Microsoft 365 Online apps - Users may be unable to access Microsoft 365 web apps, such as Excel Online. Additionally, the search bar may not appear in any Office Online service.
  • Microsoft Teams - Users may be unable to access the Microsoft Teams admin center.
  • SharePoint Online - Users may be unable to view the settings gear, search bar and waffle.
  • Microsoft Planner - Users may be unable to access Microsoft 365 web apps through Microsoft Planner.
  • Yammer - The search bar is missing from the User Interface.
  • Outlook on the web - Users may experience slowness or latency when accessing or using the service. Microsoft Project for the Web - Users may be unable to view the waffle menu, settings, and help content.

Whilst the Microsoft 365 apps may not render, users can still access the applications directly through the URL. Some examples of these include: Microsoft 365 Admin Center - admin.microsoft.com Outlook - outlook.office.com Microsoft Teams - teams.microsoft.com Word Online - microsoft365.com/launch/word Excel Online - microsoft365.com/launch/excel

Current status: Microsoft is focusing their investigation on high CPU utilization utilization that is occurring on the components which facilitate the back-end API calls for the navigation headers and features. Microsoft is changing some specific configurations which were part of a service update that occurred on Monday, April 17, 2023, to see if this provides relief whilst they continue to investigate the source of the high CPU utilization.

Scope of impact: Impact is specific to some users who are served through the affected infrastructure. The majority of reported impact is coming from users located in North and South America.

Next update will be at 2:00 p.m. CDT.

20793555

Updated

Update from Microsoft

User impact: Some users may be unable to view or access Microsoft 365 apps or services

More info: Impacted services may include but are not limited to:

  • Microsoft 365 Online apps - Users may be unable to access Microsoft 365 web apps, such as Excel Online. Additionally, the search bar may not appear in any Office Online service.
  • Microsoft Teams - Users may be unable to access the Microsoft Teams admin center.
  • SharePoint Online - Users may be unable to view the settings gear, search bar and waffle.
  • Microsoft Planner - Users may be unable to access Microsoft 365 web apps through Microsoft Planner.
  • Yammer - The search bar is missing from the User Interface.

Whilst the Microsoft 365 apps may not render, users can still access the applications directly through the URL. Some examples of these include: Microsoft 365 Admin Center - portal.office.com Outlook - outlook.office.com Microsoft Teams - teams.microsoft.com Word Online - microsoft365.com/launch/word Excel Online - microsoft365.com/launch/excel

Current status: We determined that a section of caching infrastructure is performing below acceptable performance thresholds, causing calls to gather user licensing information to bypass the cache and go directly to Azure Active Directory infrastructure resulting in high resource utilization, resulting in throttling and impact. We’re analyzing forensics within the caching infrastructure to identify the cause of the issue.

Scope of impact: Impact is specific to some users who are served through the affected infrastructure.om Microsoft:

Next update will be at 12:00 p.m. CDT (noon)

20793555

Identified

User impact: Users may be unable to view or access web apps in Microsoft 365.

More info: Users may also be unable to access Microsoft 365 web apps through Microsoft Planner. Additionally, users may be unable to access the Microsoft Teams admin center.

Current status: Analysis of diagnostic data has identified an unusually high number of timeout exceptions within Microsoft's Office 365 caching and Azure Active Directory (AAD) infrastructure. Microsoft is working to isolate the cause of these exceptions whilst identifying steps to remediate impact.

Scope of impact: Impact is specific to some users who were served through the affected infrastructure attempting to view or access web apps in Microsoft 365.

Next update will be at 11:00 a.m. CDT

20793555

Investigating

This is just affecting the web access, Outlook is still working.

People may be unable to log into Office 365 via the web, or when they do the Office 365 applications do not show up.

NDUS technicians are working to resolve reported issues. Updates will be provided as they become available.

Began at:

Affected components
  • Fabric
  • Office 365
    • Outlook
    • SharePoint / OneDrive
    • Teams
    • Teams Voice