City power has been restored and all power distrubution units are all once again fully operational.
Incident start: December 7th, 2021 @ 08:10 CET Incident end: December 7th, 2021 @ 14:00 CET
Our technical staff remained onsite during the entire incident to observe and immediately engage, should any further issues have arised due to the city power distribution issue.
We sincerely apologize for the service interruption and the problem may have been causing to the customers in the time interval of 08:10 to 08:55 CET were affected by the SAN storage entering standby mode during the primary power distribution failure and redundancy from diesel and UPS generators.
All affected customers were contacted by phone during the interruption.
Yours faithfully, CloudNordic Support & Helpdesk services
--- Datacenter levering af A-feed (bystrøm) er fuldt ud genetableret og operationelt igen.
Start: 7. December 2021 @ 08:10 CET Slut: 7. December 2021 @ 14:00 CET
Vores teknikere forblev på lokationen under hele seancen, så de kunne skride ind og tage action, skulle yderligere probelmer været opstået imens dieselgenerator og UPS systemer leverede den redundante strøm.
Vi beklager eventuelle gener denne strømafbrydelse måtte have forårsaget, hvilke kunder vi har været telefonisk i kontakt med i tidsrummet 08:10 - 08:55 da det ene SAN gik i standby mode og skulle have ekstra strøm feed fra diesel og UPS strøm distributionen.
De bedste hilsner fra CloudNordic Support & Helpdesk services
Posted Dec 07, 2021 - 15:18 CET
Update
Update; 07-12-2021 @10.00 CET
All systems are fully operational, but power distribution is still supplied via the diesel and UPS power distrbution unit, as the entire datacenter area is affected by city power issues.
However, the power issue which earlier unfortunately affected the Storage Array Network (SAN) systems secondary power supply has been resolved.
The PSU issues with the SAN system going into standby mode was resolved at 08:55 CET.
For some reason that failover power supply for SAN power was available but forced the SAN system into standby mode, when the diesel and UPS power supply kicked in.
It is now being investigated and mitigated by our technical staff, as we currently have placed the secondary part of the primary power supply directly together with third (failover) in the diesel and UPS power distrubution unit.
For the ongoing city power issue, we have received the following operational information from the ISP and datacenter provider;
Incident start: December 7th, 2021 @ 08:10 CET Incident end: December 7th, 2021 @ 11:00 CET (expected)
Our technical staff is still onsite to observe and immediately engage, should any further issues arise due to the ongoing city power distribution issue.
We sincerely apologize for the service interruption and the problem may have been causing.
Yours faithfully, CloudNordic Support & Helpdesk services
---
Vores Internet Service Provider (ISP) og datacenteret er desværre stadig ramt af strømafbrydelse, men alle vores systemer kører på UPS samt dieselgenerator strøm med fuld tilgængelighed.
Det Storage Array Network (SAN) som midlertidigt blev ramt af strømafbrydelsen og er af en eller anden årsag ikke var kørende på den redundante strøm er igen i luften, ved at vi flyttede den sekundære primære strømforsyning til diesel og USP strøm, så den sidder sammen med den tredje strømforsynings enhed i samme PDU (power distribution unit).
Kun enkelte POP3/IMAP løsninger blev ramt af dette issue og vi har stadig en tekniker på stedet som observerer alle system imens strømafbrydelsen er igang.
Vi fik løst dette problem klokken 08:55 CET, hvorefter de enkelte påvirkede POP3 og IMAP løsninger igen var i luften.
Vores ISP og datacenter leverandør har meldt følgende om strømafbrydelsen;
Start: 7. December 2021 @ 08:10 CET Slut: 7. December 2021 @ 11:00 CET (forventet)
Vi har været i telefonisk kontakt med kunder på nævnte SAN i tidsrummet hvor det ikke var tilgængeligt og i standby mode.
De bedste hilsner fra CloudNordic Support & Helpdesk services
Posted Dec 07, 2021 - 09:59 CET
Monitoring
Our Service Provider are having power issues, which are unfortunately affecting the Storage Array Network (SAN) system. For some reason that SAN is not available on the secondary diesel and UPS power supply, which is being investigated and mitigated by our technical staff.
Our Exchange mail, OWA and OpenXchange AppSuite solutions are responding without any problems..
Affected systems: POP3/IMAP mail services
Incident start: December 7th, 2021 @ 08:10 CET Inciden end: December 7th, 2021 @ 09:00 CET (expected)
We sincerely apologize for the service interruption and the problem be causing.
Yours faithfully, CloudNordic Support & Helpdesk services
---
Vores Internet Service Provider (ISP) er ramt af strømafbrydelse og vores systemer kører på UPS samt dieselgenerator strøm.
Desværre er vores Storage Array Network (SAN) ramt af strømafbrydelsen og er af en eller anden årsag ikke kørende på redudant strøm selvom det er sat op og tidligere testet funktionelt.
Kun enkelte POP3/IMAP løsninger er ramt af dette issue og vi har en tekniker på stedet til at udbedre fejlen.
Start: 7. December 2021 @ 08:10 CET Slut: 7. December 2021 @ 09:00 CET (forventet)
Vi beklager på det denne strøm afbrydelse af enkelte services og de eventuelle gener afbrydelsen måtte have medført.
De bedste hilsner fra CloudNordic Support & Helpdesk services
Posted Dec 07, 2021 - 08:28 CET
This incident affected: CloudNordic services (OX AppSuite + Dovecot Pro).