Posted at 2023-11-14 16:50:33 GMT by Andrews & Arnold
Update #14: 2024-01-12 17:12:26 GMT
> _Update 12 Jan 2024 17:09:37_
>
> An update of where we are (Friday 12th January).
>
> Some customers have had interruption to their service this week as we
> have seen a number of crashes on both Z.Witless and Y.Witless.
>
> Today we replaced the hardware of Z.Witless.
>
> Our developers have been working on investigating each crash we have.
> We have been saying in recent updates that progress had been made on
> the crashes we have seen, and this week we applied the software update
> to two of our three 'Witless' LNSs. In our test lab we have never seen
> this updated software crash during 3 weeks of testing. However, we have
> had crashes this week since applying the updated software.
>
> Usually with a crash, our developers are sent a crashlog with details
> specifying exactly where in the code the crash happened. However, the
> crashes that have been affecting us are different in that the hardware
> locks up and restarts - with this type of crash we have less forensic
> to work with which is making getting to the bottom of the problem that
> much harder.
>
> We are still working hard to resolve this. We various avenues of
> investigation to take, and during the next week we will be planning
> more overnight work as well as datacentre trips.
>
> We know how disruptive this has been for those customers affected, and
> we are doing all we can to work towards a stable service for everyone.
>
> _Update expected: 2024-01-15 13:00:00_