After upgrade 6.1->6.2 / 6.1->6.3 all repositories are empty
Created by: Wayneoween
Summary: I upgraded from 6.1 to 6.2/6.3 and all my repositories were assumed to be empty afterwards. Steps to reproduce: I don't really know. Expected behavior: That all repositories work as before. Observed behavior: At first I noticed that I can't see any push events, but in the files view everything was there. I then upgraded my 6.1 installation to 6.2 where I found that every repository was empty. I restored the database and downgraded to 6.1 again where everything was fine except for the broken sidekiq (I guess) thing. I tried upgrading directly to 6.3 with the same results. When I try to backup the repos now, every one of them are skipped, because they are presumed empty. If I push something, it works. A clone also works fine.
Relevant logs:
System information
System: Ubuntu 12.04
Current User: git
Using RVM: no
Ruby Version: 1.9.3p327
Gem Version: 1.8.23
Bundler Version:1.2.4
Rake Version: 10.1.0
GitLab information
Version: 6.3.1
Revision: 779b4dc
Directory: /home/git/gitlab
DB Adapter: mysql2
URL: https://example.com
HTTP Clone URL: https://example.com/some-project.git
SSH Clone URL: [email protected]:some-project.git
Using LDAP: no
Using Omniauth: no
GitLab Shell
Version: 1.7.9
Repositories: /home/git/repositories/
Hooks: /home/git/gitlab-shell/hooks/
Git: /usr/bin/gitand:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
Checking Environment ...
Git configured for git user? ... yes
Has python2? ... yes
python2 is supported version? ... yes
Checking Environment ... Finished
Checking GitLab Shell ...
GitLab Shell version >= 1.7.9 ? ... OK (1.7.9)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
update hook up-to-date? ... yes
update hooks in repos are links: ...
<all repos> ... repository is empty
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files:
/home/git/repositories: OK
/home/git/.ssh/authorized_keys: OK
/usr/bin/redis-cli: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking GitLab ...
Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ...
<all repos have namespaces>
Projects have satellites? ...
<all repos have satellites>
Redis version >= 2.0.0? ... yes
Your git bin path is "/usr/bin/git"
Git version >= 1.7.10 ? ... yes (1.8.5)
Checking GitLab ... Finishedsidekiq.log is showing this behaviour on a push:
2013-12-10T11:09:27Z 3547 TID-1hkoki PostReceive JID- INFO: start
2013-12-10T11:09:27Z 3547 TID-1hkoki PostReceive JID- INFO: fail: 0.336 sec
2013-12-10T11:09:27Z 3547 TID-1hkoki WARN: {"class"=>"PostReceive", "args"=>["/home/git/repositories/foo/bar.git", "874c02b93a368d43c16d97635ec202d9057efa94", "b065de13b3014d3dbc72e90ab6af94493a1ad257", "refs/heads/master", "key-1"]}
2013-12-10T11:09:27Z 3547 TID-1hkoki WARN: undefined method `commits_between' for nil:NilClass
2013-12-10T11:09:27Z 3547 TID-1hkoki WARN: /home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/gitlab_git-3.0.0.rc2/lib/gitlab_git/commit.rb:89:in `between'
/home/git/gitlab/app/models/repository.rb:45:in `commits_between'
/home/git/gitlab/app/services/git_push_service.rb:21:in `execute'
/home/git/gitlab/app/workers/post_receive.rb:32:in `perform'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/processor.rb:48:in `block (3 levels) in process'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:115:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:115:in `block in invoke'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/server/active_record.rb:6:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:117:in `block in invoke'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/server/retry_jobs.rb:62:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:117:in `block in invoke'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/logging.rb:22:in `with_context'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/server/logging.rb:7:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:117:in `block in invoke'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:120:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/middleware/chain.rb:120:in `invoke'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/processor.rb:47:in `block (2 levels) in process'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/processor.rb:102:in `stats'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/sidekiq-2.14.0/lib/sidekiq/processor.rb:46:in `block in process'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/calls.rb:25:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/calls.rb:25:in `public_send'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/calls.rb:25:in `dispatch'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/calls.rb:67:in `dispatch'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/future.rb:15:in `block in new'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/internal_pool.rb:59:in `call'
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/celluloid-0.14.1/lib/celluloid/internal_pool.rb:59:in `block in create'What can I do to diagnose / fix the problem?