Skip to main content

Google Workspace Outage

Yesterday, on December 14th, all services associated with Google Workspace (AKA GSuite - or for those who aren't familiar with it, what is essentially Google's paid "business" services) went offline for roughly an hour from 7AM to 8AM Eastern time.

Users typically first encountered the error when attempting to send email or after receiving an error indicating that their account could "not be found" when attempting to login to Google services. 


Google's App Status dashboard

Other impacted services include Youtube and the Google Nest home security service. 

Google's official statement to the press described the cause in extremely general terms:

“Services requiring users to log in experienced high error rates during this period,” a Google spokesperson said. “The authentication system issue was resolved at (7:32 a.m. EST). All services are now restored. We apologize to everyone affected, and we will conduct a thorough follow-up review to ensure this problem cannot recur in the future.”

This outage follows similar major failures with Microsoft's Office services and Windows Updates as well as service interruptions with Amazon's EC2 and related cloud products.

Comments

Popular posts from this blog

Fixing Event ID 10154 - The WinRM service failed to create the following SPN

The Problem The configuration of the system when this error was encountered is as follows: A. Windows Server 2008 R2 Redundant Domain Controllers - we will call these DC1.joshwieder.com and DC2.joshwieder.com B. Windows Server 2003 Web Server with Windows Remote Management enabled / part of the Active directory deployment - we will call this WEB.joshwieder.com C. For the sake of our example, let's say I have configured an OU named "Web Servers" on those domain controllers Whenever the Windows 2003 Web server reboots, or WinRM.exe service on the Windows 2003 Web server restarts, the following error was logged into the Event Viewer: Event ID: 10154 Source: Microsoft-Windows-WinRM Version: 6.1 Symbolic Name: LOG_WSMAN_SPN_CREATION Message: The WinRM service failed to create the following SPN: %1. Additional Data The error received was 8344: Insufficient access rights to perform the operation. User Action The SPN can be created by an administrator using sets

Email server using amavisd-new fails with (!)DENIED ACCESS from IP 1.2.3.4, policy bank ''

I have used ClamAV and Spamassassin for many years. I've had a less experience with Amavis (now amavisd-new), but I've decided to give it a try with a new mail server deployment I've been working on. As a reference for my install, I relied on the documentation provided by Amavis for integration with Postfix  as well as a somewhat-outdated but still-relevant walkthrough published by CentOS . Prior to integration with amavisd, Postfix worked fine. Similarly, I had no issues with Spamassassin on its own. But once I finished my install of amavisd-new, things quickly went wrong. Attempting to send messages to accounts hosted on my email server resulted in the following chaing of errors in my maillog: Jan 13 18:17:34 hostname amavis[31578]: Net::Server: 2016/01/13-18:17:34 CONNECT TCP Peer: "[192.168.1.1]:40209" Local: "[127.0.0.1]:10024" Jan 13 18:17:34 hostname amavis[31578]: loaded base policy bank Jan 13 18:17:34 hostname amavis[31578]: lookup_ip_

Wikileaks Global Intelligence File Dump is Loaded With Malicious Software

Click here for the second post on this topic, which includes more detailed technical information. Hector Monsegur, formerly sabu of Lulzsec, has offered his point of view on this post. Get his opinion by reading my third post on the topic. In my fourth post on this topic, I explain how malware is not limited to the Stratfor leak torrent - curated links throughout the Wikileaks.Org website allow users to download individual infected files . This series of posts is beginning to receive coverage from several newspapers around the world. German speakers should check out the story in Neue Zürcher Zeitung / New Zurich Times . For English speakers, I recommend The Register from the UK for an excellent summary of these findings . Beginning in  February 27, 2012 , the controversial news organization Wikileaks has been publishing a large and growing trove of emails from the private intelligence firm  Strategic Forecasting, Inc (more widely known as  Stratfor). The leak publication bega