Impact Statement: Starting at 10:32 UTC on 16 Nov 2022, you have been identified as a customer using Speech Service in West Europe who may experience 5xx errors when trying to use Text-to-speech (TTS). Both Neural voices and Custom Neural voices are affected.
Current Status: Our investigation points to a networking issue as the potential root cause. This caused a subset of nodes to become unhealthy, resulting in the 500 level errors mentioned above. We have performed recovery actions to restore the affected nodes functionality and at this stage customers may start experiencing signs of recovery. We will continue monitoring error rates closely to ensure mitigation, while working on identifying the full root cause. The next update will be provided in 2 hours, or as events warrant.
Posted Nov 16, 2022 - 16:22 UTC
Update
We received word from our third party provider.
Impact Statement: Starting at 10:32 UTC on 16 Nov 2022, you have been identified as a customer using Speech Service in West Europe who may experience 5xx errors when trying to use Text-to-speech (TTS). Both Neural voices and Custom Neural voices are affected.
Current Status: We continue our investigation of the underlying root cause and exploring potential mitigation options. Customers using Custom Neural Voices might be seeing signs of recovery. The next update will be provided in 2 hours, or as events warrant.
We only use Neural voices
Posted Nov 16, 2022 - 14:25 UTC
Update
We got the following update from Azure
Impact Statement: Starting at 10:32 UTC on 16 Nov 2022 you have been identified as a customer using Text-to-speech (TTS) in West Europe who may experience 5xx errors when trying to use the service. Both Neural voices and Custom Neural voices are affected.
Current Status: We are aware of this issue and are actively investigating. The next update will be provided in 60 minutes, or as events warrant.
Posted Nov 16, 2022 - 13:44 UTC
Update
We are continuing to wait for a response from our third party API provider. We apologise for any inconvenience. Our other voices are working fine.
Posted Nov 16, 2022 - 13:28 UTC
Identified
We've identified some of the issues and are speaking with Microsoft to resolve this. This affects other companies than just us btw according to twitter.
Posted Nov 16, 2022 - 12:30 UTC
Update
We are continuing to investigate
Posted Nov 16, 2022 - 11:54 UTC
Investigating
It appears that all of our azure voices are down. This affects various voices. Other voices are operational