[Status] [Minor] SMS: Problems with SMS (Open)

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold
Update #0: 2024-02-19 09:13:21 GMT

_Started: 2024-02-19 09:00:00_
We're investigating problems with inbound/outbound SMS.

_Update 19 Feb 2024 09:13:21_ One of our SMS carriers is experiencing
connectivity problems within AWS (Amazon Web Services) causing this
problem. _Update expected: 2024-02-19 09:30:00_


URL: https://aastatus.net/apost.cgi?incident=42627

--
AAISP Status Feed
URL: https://aastatus.net/

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #1: 2024-02-19 09:19:00 GMT

| _Started: 2024-02-19 00:36:00_

We're investigating problems with inbound/outbound SMS.

_Update 19 Feb 2024 09:13:21_ One of our SMS carriers is experiencing
connectivity problems within AWS (Amazon Web Services) causing this
| problem.
>

> _Update 19 Feb 2024 09:19:00_ The connectivity problems started at
> 00:36, and was escalated within hosting provider at 07:06. This is
> still being worked on by our carrier and their hosting provider (AWS) _Update

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #2: 2024-02-19 10:31:30 GMT

_Started: 2024-02-19 00:36:00_
We're investigating problems with inbound/outbound SMS.

_Update 19 Feb 2024 09:13:21_ One of our SMS carriers is experiencing
connectivity problems within AWS (Amazon Web Services) causing this
problem.

_Update 19 Feb 2024 09:19:00_ The connectivity problems started at
00:36, and was escalated within hosting provider at 07:06. This is
| still being worked on by our carrier and their hosting provider (AWS)
|

> _Update 19 Feb 2024 10:31:30_ From our carrier: This issue is still
> being worked on by our vendor's Sr. Network and IT experts to recover
> the service located in the AWS. As soon as we receive further updates,
> we will inform you accordingly. _Update expected: 2024-02-19 11:00:00_

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #3: 2024-02-19 11:30:30 GMT

_Started: 2024-02-19 00:36:00_
We're investigating problems with inbound/outbound SMS.

_Update 19 Feb 2024 09:13:21_ One of our SMS carriers is experiencing
connectivity problems within AWS (Amazon Web Services) causing this
problem.

_Update 19 Feb 2024 09:19:00_ The connectivity problems started at
00:36, and was escalated within hosting provider at 07:06. This is
still being worked on by our carrier and their hosting provider (AWS)

_Update 19 Feb 2024 10:31:30_ From our carrier: This issue is still
being worked on by our vendor's Sr. Network and IT experts to recover
the service located in the AWS. As soon as we receive further updates,
| we will inform you accordingly.
>

> _Update 19 Feb 2024 11:30:30_ From our carrier: This connectivity issue
> is still being worked on and we are focused on providing a solution as
> soon as possible. Our vendor support together with their upper
> management teams are actively working on restoring the service. _Update
> expected: 2024-02-19 12:30:00_

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #4: 2024-02-19 12:39:54 GMT

_Started: 2024-02-19 00:36:00_
We're investigating problems with inbound/outbound SMS.

_Update 19 Feb 2024 09:13:21_ One of our SMS carriers is experiencing
connectivity problems within AWS (Amazon Web Services) causing this
problem.

_Update 19 Feb 2024 09:19:00_ The connectivity problems started at
00:36, and was escalated within hosting provider at 07:06. This is
still being worked on by our carrier and their hosting provider (AWS)

_Update 19 Feb 2024 10:31:30_ From our carrier: This issue is still
being worked on by our vendor's Sr. Network and IT experts to recover
the service located in the AWS. As soon as we receive further updates,
we will inform you accordingly.

_Update 19 Feb 2024 11:30:30_ From our carrier: This connectivity issue
is still being worked on and we are focused on providing a solution as
soon as possible. Our vendor support together with their upper
| management teams are actively working on restoring the service.
|

> _Update 19 Feb 2024 12:39:54_ From our carrier: We are currently
> waiting for our vendor's teams to implement the fix for this
> connectivity issue. The issue on the AWS infrastructure has now been
> fixed and our vendor's Sr. Engineering team is working on bringing back
> live the FortiGate firewall controlling all traffic. _Update expected:
> 2024-02-19 13:30:00_

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #5: 2024-02-19 12:42:19 GMT

_Started: 2024-02-19 00:36:00_

| We're investigating problems with inbound/outbound SMS. (Outbound SMS
> will work, but our SMS API will take a while to respond due to timeouts
> with our main carrier before it fails over to the secondary carrier)

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #6: 2024-02-19 13:52:07 GMT

_Started: 2024-02-19 00:36:00_
We're investigating problems with inbound/outbound SMS. (Outbound SMS
will work, but our SMS API will take a while to respond due to timeouts
with our main carrier before it fails over to the secondary carrier)

_Update 19 Feb 2024 09:13:21_ One of our SMS carriers is experiencing
connectivity problems within AWS (Amazon Web Services) causing this
problem.

_Update 19 Feb 2024 09:19:00_ The connectivity problems started at
00:36, and was escalated within hosting provider at 07:06. This is
still being worked on by our carrier and their hosting provider (AWS)

_Update 19 Feb 2024 10:31:30_ From our carrier: This issue is still
being worked on by our vendor's Sr. Network and IT experts to recover
the service located in the AWS. As soon as we receive further updates,
we will inform you accordingly.

_Update 19 Feb 2024 11:30:30_ From our carrier: This connectivity issue
is still being worked on and we are focused on providing a solution as
soon as possible. Our vendor support together with their upper
management teams are actively working on restoring the service.

_Update 19 Feb 2024 12:39:54_ From our carrier: We are currently
waiting for our vendor's teams to implement the fix for this
connectivity issue. The issue on the AWS infrastructure has now been
fixed and our vendor's Sr. Engineering team is working on bringing back
| live the FortiGate firewall controlling all traffic.
|

> _Update 19 Feb 2024 13:51:56_ From our carrier: We have positive
> feedback from our vendor's Sr. teams along with the FortiGate team
> regarding the Inbound traffic towards HTTP accounts which now appears
> to be delivering. Our teams are still working on bringing back the
> entirety of the service and as soon as further updates are available,
> we will let you know. _Update expected: 2024-02-19 14:30:00_

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #7: 2024-02-19 15:02:54 GMT

> _Update 19 Feb 2024 15:00:04_ _Inbound SMSs are now being received_ -
> including one that were sent earlier in the day.
>
> _Update 19 Feb 2024 15:02:54_ From our carrier: Our vendor's teams
> along with FortGate teams are still working on restoring all SMSC
> connectivities back up and running. _Update expected: 2024-02-19
> 15:30:00_

[Andrews & Arnold]

Posted at 2024-02-19 09:11:38 GMT by Andrews & Arnold

Update #8: 2024-02-19 16:19:41 GMT

> _Resolution_ Service was restored at around 3PM. We'll post further
> updates from our carrier as we get them. During this time outgoing SMSs
> were working via our secondary carrier, but our API would have taken a
> longer than usual amount of time to accepts messages. Incoming messages
> sent today would have been delayed and received after 3PM.