Restore backup fails (#8880) · Issues · gpt / large_projects / gitlabhq1

Restore backup fails

Created by: odesport

Summary

Restore backup from an other server with 7.8.1 version both fails. Old server works with MySQL and new one with PotgreSQL.

Steps to reproduce

Old server

service gitlab stop sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production sudo -u git -H mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p sudo -u git -H python mysql-postgresql-converter/db_converter.py gitlabhq_production.mysql db/database.sql Copy new database.sql to the tar backup file

New server

copy tar file in /home/git/gitlab/tmp/backups Extract tar file and copy its content in /home/git/gitlab/tmp/backups because Restore fails :

Unpacking backup ... done
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - backup_information.yml
/home/git/gitlab/lib/backup/manager.rb:112:in `unpack'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:28:in `block (3 levels) in <top (required)>'
Tasks: TOP => gitlab:backup:restore

Launch restore :

sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production --trace

Expected behavior

Backup is restored.

Observed behavior

Restore fails.

Relevant logs and/or screenshots

The trace :

** Invoke gitlab:backup:restore (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab:backup:restore
Unpacking backup ... done
** Invoke gitlab:backup:db:restore (first_time)
** Invoke environment 
** Execute gitlab:backup:db:restore
Restoring database ... 
Restoring PostgreSQL database gitlabhq_production ... ** Invoke gitlab:db:drop_all_tables (first_time)
** Invoke environment 
** Execute gitlab:db:drop_all_tables
** Invoke gitlab:db:drop_all_postgres_sequences (first_time)
** Invoke environment 
** Execute gitlab:db:drop_all_postgres_sequences
START TRANSACTION
SET
SET
SET CONSTRAINTS
psql:/home/git/gitlab/tmp/backups/db/database.sql:20: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « application_settings_pkey » pour la table « application_settings »
CREATE TABLE
INSERT 0 1
psql:/home/git/gitlab/tmp/backups/db/database.sql:34: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « broadcast_messages_pkey » pour la table « broadcast_messages »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:43: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « deploy_keys_projects_pkey » pour la table « deploy_keys_projects »
CREATE TABLE
INSERT 0 1
psql:/home/git/gitlab/tmp/backups/db/database.sql:54: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « emails_pkey » pour la table « emails »
psql:/home/git/gitlab/tmp/backups/db/database.sql:54: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « emails_email_key » pour la table « emails »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:68: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « events_pkey » pour la table « events »
CREATE TABLE
INSERT 0 1206
INSERT 0 1146
INSERT 0 1165
INSERT 0 841
psql:/home/git/gitlab/tmp/backups/db/database.sql:82: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « forked_project_links_pkey » pour la table « forked_project_links »
psql:/home/git/gitlab/tmp/backups/db/database.sql:82: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « forked_project_links_forked_to_project_id_key » pour la table « forked_project_links »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:92: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « identities_pkey » pour la table « identities »
CREATE TABLE
INSERT 0 12
psql:/home/git/gitlab/tmp/backups/db/database.sql:111: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « issues_pkey » pour la table « issues »
psql:/home/git/gitlab/tmp/backups/db/database.sql:111: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « issues_project_id_iid_key » pour la table « issues »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:123: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « keys_pkey » pour la table « keys »
CREATE TABLE
INSERT 0 28
psql:/home/git/gitlab/tmp/backups/db/database.sql:134: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « label_links_pkey » pour la table « label_links »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:144: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « labels_pkey » pour la table « labels »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:157: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « members_pkey » pour la table « members »
CREATE TABLE
INSERT 0 57
psql:/home/git/gitlab/tmp/backups/db/database.sql:170: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « merge_request_diffs_pkey » pour la table « merge_request_diffs »
psql:/home/git/gitlab/tmp/backups/db/database.sql:170: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « merge_request_diffs_merge_request_id_key » pour la table « merge_request_diffs »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:192: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « merge_requests_pkey » pour la table « merge_requests »
psql:/home/git/gitlab/tmp/backups/db/database.sql:192: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « merge_requests_target_project_id_iid_key » pour la table « merge_requests »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:206: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « milestones_pkey » pour la table « milestones »
psql:/home/git/gitlab/tmp/backups/db/database.sql:206: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « milestones_project_id_iid_key » pour la table « milestones »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:219: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « namespaces_pkey » pour la table « namespaces »
CREATE TABLE
INSERT 0 13
psql:/home/git/gitlab/tmp/backups/db/database.sql:237: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « notes_pkey » pour la table « notes »
CREATE TABLE
INSERT 0 1
psql:/home/git/gitlab/tmp/backups/db/database.sql:252: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « oauth_access_grants_pkey » pour la table « oauth_access_grants »
psql:/home/git/gitlab/tmp/backups/db/database.sql:252: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « oauth_access_grants_token_key » pour la table « oauth_access_grants »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:267: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « oauth_access_tokens_pkey » pour la table « oauth_access_tokens »
psql:/home/git/gitlab/tmp/backups/db/database.sql:267: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « oauth_access_tokens_token_key » pour la table « oauth_access_tokens »
psql:/home/git/gitlab/tmp/backups/db/database.sql:267: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « oauth_access_tokens_refresh_token_key » pour la table « oauth_access_tokens »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:282: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « oauth_applications_pkey » pour la table « oauth_applications »
psql:/home/git/gitlab/tmp/backups/db/database.sql:282: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « oauth_applications_uid_key » pour la table « oauth_applications »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:311: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « projects_pkey » pour la table « projects »
CREATE TABLE
INSERT 0 24
psql:/home/git/gitlab/tmp/backups/db/database.sql:322: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « protected_branches_pkey » pour la table « protected_branches »
CREATE TABLE
INSERT 0 1
psql:/home/git/gitlab/tmp/backups/db/database.sql:328: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « schema_migrations_version_key » pour la table « schema_migrations »
CREATE TABLE
INSERT 0 215
psql:/home/git/gitlab/tmp/backups/db/database.sql:342: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « services_pkey » pour la table « services »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:357: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « snippets_pkey » pour la table « snippets »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:369: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « taggings_pkey » pour la table « taggings »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:375: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « tags_pkey » pour la table « tags »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:428: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « users_pkey » pour la table « users »
psql:/home/git/gitlab/tmp/backups/db/database.sql:428: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « users_email_key » pour la table « users »
psql:/home/git/gitlab/tmp/backups/db/database.sql:428: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « users_reset_password_token_key » pour la table « users »
psql:/home/git/gitlab/tmp/backups/db/database.sql:428: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « users_authentication_token_key » pour la table « users »
psql:/home/git/gitlab/tmp/backups/db/database.sql:428: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « users_confirmation_token_key » pour la table « users »
CREATE TABLE
INSERT 0 12
psql:/home/git/gitlab/tmp/backups/db/database.sql:439: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « users_star_projects_pkey » pour la table « users_star_projects »
psql:/home/git/gitlab/tmp/backups/db/database.sql:439: NOTICE:  CREATE TABLE / UNIQUE créera un index implicite « users_star_projects_user_id_project_id_key » pour la table « users_star_projects »
CREATE TABLE
psql:/home/git/gitlab/tmp/backups/db/database.sql:454: NOTICE:  CREATE TABLE / PRIMARY KEY créera un index implicite « web_hooks_pkey » pour la table « web_hooks »
CREATE TABLE
COMMIT
START TRANSACTION
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE SEQUENCE
 setval 
--------
      1
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
      1
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
   4961
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
     18
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
     55
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
     69
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
     20
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
      1
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
     49
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
      1
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------
     21
(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
CREATE SEQUENCE
 setval 
--------

(1 ligne)

ALTER TABLE
COMMIT
[DONE]
done
** Invoke gitlab:backup:repo:restore (first_time)
** Invoke environment 
** Execute gitlab:backup:repo:restore
Restoring repositories ...
 * odesport/fichiers-rudder ... [FAILED]
failed: git clone --bare /home/git/gitlab/tmp/backups/repositories/odesport/fichiers-rudder.bundle /home/git/repositories/odesport/fichiers-rudder.git
Restore failed

Output of checks

Results of GitLab Application Check

sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true

Checking Environment ...

Git configured for git user? ... yes

Checking Environment ... Finished

Checking GitLab Shell ...

GitLab Shell version >= 2.5.4 ? ... OK (2.5.4)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
Satellites access is drwxr-x---? ... yes
hooks directories in repos are links: ... 
3/3 ... repository is empty
3/9 ... repository is empty
11/11 ... repository is empty
3/13 ... repository is empty
3/15 ... repository is empty
9/17 ... repository is empty
3/19 ... repository is empty
5/20 ... repository is empty
3/21 ... repository is empty
3/22 ... repository is empty
3/24 ... repository is empty
3/26 ... repository is empty
3/27 ... repository is empty
16/28 ... repository is empty
3/30 ... repository is empty
16/31 ... repository is empty
16/32 ... repository is empty
16/36 ... repository is empty
16/39 ... repository is empty
20/41 ... repository is empty
16/42 ... repository is empty
16/43 ... repository is empty
17/46 ... repository is empty
16/49 ... repository is empty
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files: 
        /home/git/repositories/: OK
        /home/git/.ssh/authorized_keys: OK
Test redis-cli executable: redis-cli 2.4.14
Send ping to redis server: PONG
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ... 
3/3 ... yes
3/9 ... yes
11/11 ... yes
3/13 ... yes
3/15 ... yes
9/17 ... yes
3/19 ... yes
5/20 ... yes
3/21 ... yes
3/22 ... yes
3/24 ... yes
3/26 ... yes
3/27 ... yes
16/28 ... yes
3/30 ... yes
16/31 ... yes
16/32 ... yes
16/36 ... yes
16/39 ... yes
20/41 ... yes
16/42 ... yes
16/43 ... yes
17/46 ... yes
16/49 ... yes
Projects have satellites? ... 
3/3 ... can't create, repository is empty
3/9 ... can't create, repository is empty
11/11 ... can't create, repository is empty
3/13 ... can't create, repository is empty
3/15 ... can't create, repository is empty
9/17 ... can't create, repository is empty
3/19 ... can't create, repository is empty
5/20 ... can't create, repository is empty
3/21 ... can't create, repository is empty
3/22 ... can't create, repository is empty
3/24 ... can't create, repository is empty
3/26 ... can't create, repository is empty
3/27 ... can't create, repository is empty
16/28 ... can't create, repository is empty
3/30 ... can't create, repository is empty
16/31 ... can't create, repository is empty
16/32 ... can't create, repository is empty
16/36 ... can't create, repository is empty
16/39 ... can't create, repository is empty
20/41 ... can't create, repository is empty
16/42 ... can't create, repository is empty
16/43 ... can't create, repository is empty
17/46 ... can't create, repository is empty
16/49 ... can't create, repository is empty
Redis version >= 2.0.0? ... yes
Ruby version >= 2.0.0 ? ... yes (2.1.5)
Your git bin path is "/usr/bin/git"
Git version >= 1.7.10 ? ... yes (1.7.10)

Checking GitLab ... Finished

Version of GitLab 7.8 stable
Last commit SHA-1 of the GitLab version GitLab 7.8.1 e2d785c3
Setup :

System information
System:         Debian 7.8
Current User:   git
Using RVM:      no
Ruby Version:   2.1.5p273
Gem Version:    2.2.2
Bundler Version:1.8.3
Rake Version:   10.3.2
Sidekiq Version:3.3.0

GitLab information
Version:        7.8.1
Revision:       e2d785c
Directory:      /home/git/gitlab
DB Adapter:     postgresql
Using LDAP:     yes
Using Omniauth: no

GitLab Shell
Version:        2.5.4
Repositories:   /home/git/repositories/
Hooks:          /home/git/gitlab-shell/hooks/
Git:            /usr/bin/git