Rackspace Hosted Exchange Outage Due to Security Incident

Posted by

Rackspace hosted Exchange suffered a catastrophic blackout beginning December 2, 2022 and is still ongoing as of 12:37 AM December fourth. At first referred to as connectivity and login problems, the guidance was ultimately upgraded to announce that they were dealing with a security occurrence.

Rackspace Hosted Exchange Issues

The Rackspace system decreased in the morning hours of December 2, 2022. At first there was no word from Rackspace about what the problem was, much less an ETA of when it would be fixed.

Customers on Buy Twitter Verification reported that Rackspace was not responding to support emails.

A Rackspace client independently messaged me over social networks on Friday to relate their experience:

“All hosted Exchange customers down over the past 16 hours.

Uncertain how many business that is, but it’s considerable.

They’re serving a 554 long delay bounce so individuals emailing in aren’t knowledgeable about the bounce for numerous hours.”

The official Rackspace status page provided a running update of the blackout however the initial posts had no info aside from there was a failure and it was being investigated.

The first official update was on December second at 2:49 AM:

“We are investigating a concern that is affecting our Hosted Exchange environments. More details will be posted as they become available.”

Thirteen minutes later Rackspace began calling it a “connectivity concern.”

“We are examining reports of connectivity problems to our Exchange environments.

Users may experience an error upon accessing the Outlook Web App (Webmail) and syncing their e-mail client(s).”

By 6:36 AM the Rackspace updates explained the ongoing issue as “connectivity and login concerns” then later on that afternoon at 1:54 PM Rackspace revealed they were still in the “examination phase” of the interruption, still trying to determine what went wrong.

And they were still calling it “connectivity and login concerns” in their Cloud Office environments at 4:51 PM that afternoon.

Rackspace Recommends Moving to Microsoft 365

4 hours later on Rackspace described the situation as a “substantial failure”and began providing their clients totally free Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround up until they comprehended the problem and might bring the system back online.

The main assistance specified:

“We experienced a substantial failure in our Hosted Exchange environment. We proactively shut down the environment to prevent any further issues while we continue work to bring back service. As we continue to resolve the origin of the concern, we have an alternate solution that will re-activate your ability to send and receive emails.

At no cost to you, we will be offering you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 up until further notice.”

Rackspace Hosted Exchange Security Event

It was not up until nearly 24 hours later on at 1:57 AM on December 3rd that Rackspace formally announced that their hosted Exchange service was suffering from a security incident.

The statement further exposed that the Rackspace professionals had actually powered down and disconnected the Exchange environment.

Rackspace published:

“After additional analysis, we have determined that this is a security occurrence.

The recognized effect is isolated to a portion of our Hosted Exchange platform. We are taking necessary actions to examine and secure our environments.”

Twelve hours later on that afternoon they upgraded the status page with more info that their security team and outside specialists were still dealing with fixing the failure.

Was Rackspace Service Affected by a Vulnerability?

Rackspace has actually not launched details of the security occasion.

A security occasion typically involves a vulnerability and there are two extreme vulnerabilities presently in the wile that were covered in November 2022.

These are the 2 most present vulnerabilities:

  • CVE-2022-41040
    Microsoft Exchange Server Server-Side Request Forgery (SSRF) Vulnerability
    A Server Side Demand Forgery (SSRF) attack allows a hacker to check out and change information on the server.
  • CVE-2022-41082
    Microsoft Exchange Server Remote Code Execution Vulnerability
    A Remote Code Execution Vulnerability is one in which an attacker is able to run harmful code on a server.

An advisory published in October 2022 described the effect of the vulnerabilities:

“An authenticated remote attacker can carry out SSRF attacks to escalate privileges and perform arbtirary PowerShell code on vulnerable Microsoft Exchange servers.

As the attack is targeted versus Microsoft Exchange Mail box server, the opponent can possibly get to other resources by means of lateral motion into Exchange and Active Directory environments.”

The Rackspace blackout updates have not shown what the specific problem was, only that it was a security occurrence.

The most current status upgrade since December 4th stated that the service is still down and clients are motivated to migrate to the Microsoft 365 service.

Rackspace published the following on December 4, 2022 at 12:37 AM:

“We continue to make progress in addressing the incident. The accessibility of your service and security of your data is of high value.

We have actually committed extensive internal resources and engaged world-class external knowledge in our efforts to reduce unfavorable effects to clients.”

It’s possible that the above kept in mind vulnerabilities relate to the security incident affecting the Rackspace Hosted Exchange service.

There has been no announcement of whether consumer information has been jeopardized. This occasion is still continuous.

Included image by Best SMM Panel/Orn Rin