We've implemented a few additional configuration changes that have allowed us to return response times to within their normal ranges. We're still working on the underlying instability in our database cluster and will continue to do so until we're able to address the root cause. We feel the changes we've made today will help mitigate further instability while we do so. We apologize for the delays and thank you again for your patience!
Posted May 19, 2016 - 17:20 PDT
Update
Things are looking better! The adjustments we made earlier in the day have decreased 502 and 504 responses dramatically. Response times are still above normal, but are much closer to normal levels after this change. We will continue monitoring and update you as new information becomes available.
Posted May 19, 2016 - 14:46 PDT
Update
While response times for both our Data Collection API and Data Analysis API continue to be elevated, we have made configuration changes that have relieved the pressure we were seeing earlier in the day and should soften some of the spikes. We are continuing to work towards a full resolution of this issue.
Posted May 19, 2016 - 12:59 PDT
Monitoring
The response times for our Data Collection API are increasing again, and are once again in a degraded state. This was expected as we entered peak hours again today and we are still working to resolve the issues, but response times will continue to be elevated for a bit. We are keeping an eye on things, and will continue to update you if things get worse or as we work towards resolving the issue.
Posted May 19, 2016 - 09:30 PDT
This incident affected: Stream API and Compute API.