{"id":29095,"date":"2017-02-03T18:01:18","date_gmt":"2017-02-03T10:01:18","guid":{"rendered":"https:\/\/www.hongkiat.com\/blog\/?p=29095"},"modified":"2017-02-03T17:30:41","modified_gmt":"2017-02-03T09:30:41","slug":"gitlab-down-for-a-day","status":"publish","type":"post","link":"https:\/\/www.hongkiat.com\/blog\/gitlab-down-for-a-day\/","title":{"rendered":"Minor Mistake, Major Catastrophe – GitLab Goes Offline For a Day"},"content":{"rendered":"

GitLab<\/a>, a startup that is an alternative to the highly popular GitHub<\/a>, has had a rather rough day recently as a human error caused the whole website to be down for a full day<\/strong>.<\/p>\n

\n\t\t\t\t\tRead Also:<\/strong>\u00a0\n\t\t\t\t\t10 Useful Github Features to Know<\/a>\n\t\t\t\t<\/p>\n

GitLab\u2019s problem first began when the website was experiencing load time and stability problems<\/strong>. The website was brought offline in order to rectify the issues. However, during the maintenance, someone unwittingly made the mistake of accidentally deleting a directory containing 300GB of live production data<\/strong>.<\/p>\n

Fortunately for GitLab, the contents that were deleted only affected issues and merged requests<\/strong>. The repositories and wikis were not affected by this mistake in any way, meaning that the damage caused by this mistake was not as severe<\/strong> as it could have been<\/p>\n

\"gitlab<\/figure>\n

With the directory deleted, the GitLab personnel immediately turn towards its backup. Unfortunately for them, GitLab ran into yet another problem<\/strong>. According to the Google Doc file <\/a>that was constantly being updated as GitLab struggles to get back online, all 5 backup\/replication techniques that the website deployed failed to work reliably<\/strong>, and in some cases were never set up in the first place<\/strong>.<\/p>\n

Retrieving these lost files from the cloud<\/strong> is not an option for GitLab either as the startup decided late last year to dump the cloud in order to build and operate its own Ceph clusters instead<\/a><\/strong>. Needless to say, GitLab is now reconsidering their stance on that particular topic.<\/p>\n

\n

@TheRegister<\/a> @gitlab<\/a> will be working on making the application more performant and exploring alternative cloud hosting providers.<\/p>\n

\u2014 Connor Shea (@connorjshea) February 1, 2017<\/a><\/p><\/blockquote>\n