GitLab

Created by: stefanhoth

Summary: I had a failing sidekiq that basically stopped creating repositories etc. but it was hard to track down because the UI showed no error - at all.

Steps to reproduce:

1. Make sidekiq unusable, e.g. by adding an error in the gemfile (after upgrading to Ruby 2.0 my bundler was outdated)
2. create a new repository through the web interface

Expected behavior: If sidekiq fails creating the repository a) it should be shown in the UI as a warning "Repository is not yet set up", b) rake gitlab:check should mention that the repository paths that exist in the db are not present

Best if both happens.

Observed behavior: The repositories aren't created on disk if sidekiq fails (d'oh). Git push fails with "fatal: '/home/git/repositories/[user]/[repo].git' does not appear to be a git repository" The only thing which pointed my in the direction was sidekiq.log which I checked after all other reasons failed.

Relevant logs: (sidekiq.log)

Message: 2014-01-15T21:06:29Z 30725 TID-t6rf8 GitlabShellWorker JID-ce0a258013e1a8d130a9fc40 INFO: done: 0.553 sec
/usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.1.3/lib/bundler/dsl.rb:12:in `rescue in evaluate': There was an error in your Gemfile, and Bundler cannot continue. (Bundler::GemfileError)
# [stacktrace]

Results of GitLab Application Check:

# [deleted stuff]
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
# [deleted yes-checks]

# [other repos]
Projects have satellites? ...
# [other repos]
myteam / testrepo ... can't create, repository is empty
# [more stuff]

Environment:

sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

System information
System:         Debian 6.0.8
Current User:   git
Using RVM:      no
Ruby Version:   2.0.0p353
Gem Version:    2.0.14
Bundler Version:1.5.2
Rake Version:   10.1.0

GitLab information
Version:        6.4.3
Revision:       42131d0
Directory:      /home/git/gitlab
DB Adapter:     mysql2
URL:            #hidden
HTTP Clone URL: #hidden
SSH Clone URL:  #hidden
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: google_oauth2

GitLab Shell
Version:        1.8.0
Repositories:   /home/git/repositories/
Hooks:          /home/git/gitlab-shell/hooks/
Git:            /usr/bin/git

Possible fixes:

• Add a intermediate status to the creation of a repository and display the status, i.e. the missing directory, in the UI as long as it's the case.
• Additionally add a warning to gitlab:check that a repository that's in the DB is not existing on disk (yet).
• Also it might be a good thing to watch the sidekiq tasks and give a warning (maybe via mail to the admin?) that the task failed.