Kraken, the fifth-largest cryptocurrency exchange, has resumed service after scheduled maintenance that was supposed to take two hours but instead took two days.
The unexpectedly long downtime agitated many users, since not only trading but also withdrawals were unavailable for the period, during which coin prices gyrated on bearish news coming out of South Korea.
As a penance of sorts, the exchange, based in San Francisco, is waiving trading fees for all its clients through the end of the month, it said in a blog post published Saturday. However, the waiver does not apply to non-margin trades.
In the blog post, the company reassured clients that their funds were kept safe during the incident and expressed contrition, saying:
"We apologize for the uncertainty our downtime has caused. The scheduled downtime was to replace our old trading engine with a brand new trading engine – an improvement that customers have long asked for and that we have long been working hard on."
'Possibly longer'
It all started when Kraken gave customers a heads up on Wednesday that the next day it would be offline for a systems upgrade. The company estimated then that this upgrade would take "about 2 hours ... possibly longer."
As scheduled, the downtime began Thursday at 05:00 UTC, according to a status page on Kraken's website. But the rest did not go according to plan.
"We got a late start and the upgrade is moving slower than anticipated," Kraken said on the page at 07:34 UTC, about a half hour after the the work was initially expected to be done.
A few more hours went by, with a couple updates from Kraken. Then, at 15:27 UTC, it looked like there was a light at the end of tunnel, with Kraken telling users:
"We are in the final stage of installing the upgrade now. We are getting close but hard to give an exact ETA since it depends on how the final testing goes."
No such luck. Three and a half hours later, Kraken said it now had to "resolve a number of issues that came up during final testing" and that it could not give a hard estimate for when that would be done.
The pattern of periodic updates, but no resumption of service, continued. "Yes, this is our new record for downtime since we launched in 2013," one update confessed. "No, we're not proud of it."
All the while, users were unable to access their funds. Understandably, many of them took to social media to voice their complaints.
#kraken #krakendown longest silence between update messages. Last one was 6h ago. Anybody still in there??
— Jakub Karczynski (@KubaKarczynski) January 12, 2018
'Elusive bug'
At 20:15 UTC Friday, Kraken said that "an elusive bug" was still holding up the relaunch of the site. In its next message, Saturday morning at 09:32, it said service would resume "shortly" but all orders had been canceled.
Finally, at 11:30 Saturday, Kraken's systems were back online, and by 15:47 trading had resumed.
In its subsequent mea culpa blog post, Kraken gave a general explanation for the delays:
"Our system is replicated in development, where all tests passed. However when the new engine was moved to the production environment, regression tests picked up a condition that proved quite difficult to track down."
A more detailed breakdown of what happened, and how Kraken intends to prevent such snafus from reoccurring, will come in the next few days, the company said.
In the meantime, it said it would be monitoring the performance of the new trading engine and cautioned that "the site will come down if needed, which may occur with little to no notice."
Source: https://www.coindesk.com/kraken-exchange-is-back-online-after-troublesome-system-upgrade/