Skip to content

GitLab

  • Menu
    • Projects Groups Snippets
      Help
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • G gitlabhq1
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 21
    • Issues 21
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 12
    • Merge requests 12
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • gpt
  • large_projects
  • gitlabhq1
  • Issues
  • #5103

Closed
Open
Created 11 years ago by Administrator@rootOwner
  • New issue

  • Report abuse

  • New issue

  • Report abuse

Init.d script reports Sidekiq Job Dispatcher as Not Running in v6.1

Closed

Init.d script reports Sidekiq Job Dispatcher as Not Running in v6.1

Created by: demisx

After upgrading to 6.1-stable, the gitlab init.d script, besides taking much time to start, reports sidekiq not running:

# service gitlab start 
Starting the GitLab Unicorn web server...
Starting the GitLab Sidekiq event dispatcher...
The GitLab Unicorn webserver with pid 27448 is running.
The GitLab Sidekiq job dispatcher is not running.

However, it appears to be running and GitLab functions properly. Here is my output of ps after the start:

# ps -ef | grep sidekiq | grep -v grep                                                                                                                                                                        
git      27484     1  7 11:20 pts/1    00:00:44 sidekiq 2.14.0 gitlab [0 of 25 busy]        

And the environment:

$ bundle exec rake gitlab:env:info   

System information
System:     Ubuntu 12.04
Current User:   git
Using RVM:  yes
RVM Version:    1.22.3
Ruby Version:   1.9.3p429
Gem Version:    1.8.25
Bundler Version:1.3.5
Rake Version:   10.1.0

GitLab information
Version:    6.1.0.rc1
Revision:   16b6040
Directory:  /home/git/gitlab
DB Adapter: mysql2
URL:        http://git.colddata.com
HTTP Clone URL: http://git.colddata.com/some-project.git
SSH Clone URL:  git@git.colddata.com:some-project.git
Using LDAP: no
Using Omniauth: no

GitLab Shell
Version:    1.7.0
Repositories:   /home/git/repositories/
Hooks:      /home/git/gitlab-shell/hooks/
Git:        /usr/bin/git

Linked issues
0


  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: XadillaX

    I have this problem too.

    By Administrator on 2013-09-17T06:53:59 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: dyadyaMisha

    Help!

    Last login: Mon Sep 23 09:30:47 2013
    misha@GitLab:~$ cd ..
    misha@GitLab:/home$ cd git/gitlab
    misha@GitLab:/home/git/gitlab$ sudo service gitlab stop
    [sudo] password for misha:
    Stopping the GitLab Unicorn web server...
    Stopping Sidekiq job dispatcher............................
    Sidekiq shut down gracefully.
    GitLab is not running.
    misha@GitLab:/home/git/gitlab$ sudo service gitlab start
    Starting the GitLab Unicorn web server...
    Starting the GitLab Sidekiq event dispatcher...
    The GitLab Unicorn webserver with pid 2019 is running.
    The GitLab Sidekiq job dispatcher is not running.
    misha@GitLab:/home/git/gitlab$ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
    Checking Environment ... 
    
    Git configured for git user? ... yes
    Has python2? ... yes
    python2 is supported version? ... yes 
    Checking Environment ... Finished 
    Checking GitLab Shell ... 
    
    GitLab Shell version >= 1.7.0 ? ... OK (1.7.1)
    Repo base directory exists? ... yes
    Repo base directory is a symlink? ... no
    Repo base owned by git:git? ... yes
    Repo base access is drwxrws---? ... yes
    update hook up-to-date? ... yes
    update hooks in repos are links: ...
    ANHK / ServerExtension ... repository is empty
    DocsVision / soglasovanie ... repository is empty
    Roma / test1 ... repository is empty
    
    Checking GitLab Shell ... Finished
    
    Checking Sidekiq ...
    Running? ... yes
    
    Checking Sidekiq ... Finished
    
    Checking GitLab ...
    Database config exists? ... yes
    Database is SQLite ... no
    All migrations up? ... yes
    GitLab config exists? ... yes
    GitLab config outdated? ... no
    Log directory writable? ... yes
    Tmp directory writable? ... yes
    Init script exists? ... yes
    Init script up-to-date? ... yes
    Projects have satellites? ...
    A / ServerExtension ... can't create, repository is empty
    D / soglasovanie ... can't create, repository is empty
    Roma / test1 ... can't create, repository is empty
    Redis version >= 2.0.0? ... yes
    Your git bin path is "/usr/bin/git"
    Git version >= 1.7.10 ? ... yes (1.8.4)
    Checking GitLab ... Finished 
    
    misha@GitLab:/home/git/gitlab$ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
    
    System information
    System:        Ubuntu 12.04
    Current User:  git
    Using RVM:      no
    Ruby Version:  2.0.0p247
    Gem Version:    2.0.3
    Bundler Version:1.3.5
    Rake Version:  10.1.0 
    
    GitLab information
    Version:        6.1.0.rc1
    Revision:      16b6040
    Directory:      /home/git/gitlab
    DB Adapter:    mysql2
    URL:            http://10.0.0.153
    HTTP Clone URL: http://10.0.0.153/some-project.git
    SSH Clone URL:  git@10.0.0.153:some-project.git
    Using LDAP:    no
    Using Omniauth: no 
    
    GitLab Shell
    Version:        1.7.1
    Repositories:  /home/git/repositories/
    Hooks:          /home/git/gitlab-shell/hooks/
    Git:            /usr/bin/git
    misha@GitLab:/home/git/gitlab$

    By Administrator on 2013-09-23T01:07:42 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: TOMeeeek

    I have same problem :-(

    By Administrator on 2013-09-25T06:31:32 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: Razer6

    +1

    By Administrator on 2013-09-25T06:55:58 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: julien-hedoux

    I have exactly the same problem : root@website:/# /etc/init.d/gitlab restart Stopping the GitLab Unicorn web server... Stopping Sidekiq job dispatcher..........................Sidekiq shut down gracefully.

    GitLab is not running. Starting the GitLab Unicorn web server... Starting the GitLab Sidekiq event dispatcher... The GitLab Unicorn webserver with pid 11878 is running. The GitLab Sidekiq job dispatcher is not running.

    By Administrator on 2013-09-25T17:12:45 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: oscx

    +1

    By Administrator on 2013-09-25T22:31:06 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: grim8634

    Yup, me too :(

    By Administrator on 2013-09-27T23:39:37 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: ghost

    I have the same problem, any ideas? Thanks

    By Administrator on 2013-09-29T10:05:18 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: julien-hedoux

    I had rollbacked the install at 6.0 version in my side. I know that this version works.

    By Administrator on 2013-09-29T10:49:26 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: rdanbrook

    Had the same problem, but it seems to have gone away after stopping and restarting gitlab.

    By Administrator on 2013-09-30T13:34:43 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: oscx

    rdanbrook: Thanks for this comment. However, I can not confirm the issue to be resolved after process stop/restart. It's still pending. 6.0 started without troubles. Next sub version notifies about sidekiq issue on start.

    By Administrator on 2013-09-30T13:47:10 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: dyadyaMisha

    Will use version 6.0 :-(

    By Administrator on 2013-10-01T04:15:44 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: minhdanh

    I'm having same problem. Restarting gitlab did not help.

    By Administrator on 2013-10-04T09:58:13 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: cedricfung

    The same issue here.

    sudo service gitlab start
    
    Starting the GitLab Unicorn web server...
    Starting the GitLab Sidekiq event dispatcher...
    The GitLab Unicorn webserver with pid 4255 is running.
    The GitLab Sidekiq job dispatcher is not running.

    The check

    sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
    
    
    Checking Sidekiq ...
    
    Running? ... yes
    Number of Sidekiq processes ... 2
      Try fixing it:
      sudo service gitlab stop
      sudo pkill -u git -f sidekiq
      sleep 10 && sudo pkill -9 -u git -f sidekiq
      sudo service gitlab start
      Please fix the error above and rerun the checks.
    
    Checking Sidekiq ... Finished

    The sidekiq is actually running, 1 process

    ps -ef | grep sidekiq | grep -v grep
    git       4273     1  7 23:43 pts/0    00:00:18 sidekiq 2.14.0 gitlab [0 of 25 busy]

    By Administrator on 2013-10-05T03:49:52 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: plamen-nikolov

    +1 The same issue :(

    By Administrator on 2013-10-06T19:28:03 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: ghost

    Same issue here sidekiq is running but when starting GitLab service responds with: The GitLab Sidekiq job dispatcher is not running.

    By Administrator on 2013-10-07T08:24:22 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: ladyka

    I have this problem too. URI http://dev.quadrate64.ru/ Screen from terminal http://img585.imageshack.us/img585/3213/sbc9.png ( You can see the different colors )

    By Administrator on 2013-10-08T11:49:28 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: kryachkov

    Same issue

    By Administrator on 2013-10-11T07:18:49 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: JorisOnghena

    Same problem as vecio. Reports 2 sidekiq processes but only one showing.

    By Administrator on 2013-10-16T05:53:01 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: dosire

    Thanks for reporting people. This is fixed with https://github.com/gitlabhq/gitlabhq/commit/f3d77a7661a9933bd194ed6634f8383ac9fd0448

    By Administrator on 2013-10-16T12:32:36 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: dyadyaMisha

    Thanks :)

    By Administrator on 2013-10-17T01:09:47 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: dosire

    @dyadyaMisha You're welcome!

    By Administrator on 2013-10-17T05:17:33 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: jacobvosmaer

    There are two separate issues here. @vecio mentions that rake gitlab:check erroneously reports that there are two sidekiq processes while there is only one. This issue with GitLab 6.2 was fixed by f3d77a76 and 73d19bb3.

    The original issue reported by @colddata is about the init script in GitLab 6.1 saying Sidekiq is not running while in fact it is. I wonder if this has to do with this timeout: https://github.com/gitlabhq/gitlabhq/blob/master/lib/support/init.d/gitlab#L140 . What do you think @rovanion?

    By Administrator on 2013-10-18T07:37:55 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: Rovanion

    Yes that timing was set on my machine which although it is very slow, it only has an AMD Sempron 145, may not be the slowest. The best thing would be if we could have Sidekiq tell us when it's started, but that's not likely to happen. We could make a loop which checks if there exists a pidfile there. I'll thinker and make a pull request.

    By Administrator on 2013-10-18T08:18:19 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: jacobvosmaer

    @Rovanion I suspect that 97f124b3 may be the fix for this.

    In 6-1-stable, the rake sidekiq:start task exits while Sidekiq itself is still booting in the background: note the nohup ... & in https://github.com/gitlabhq/gitlabhq/blob/6-1-stable/lib/tasks/sidekiq.rake#L9.

    I will make a Merge Request to get 97f124b3 backported into 6-1-stable.

    By Administrator on 2013-10-18T09:07:59 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: Rovanion

    @jacobvosmaer So that makes RAILS_ENV=$RAILS_ENV bundle exec rake sidekiq:start not return until the pid is written? If that's the case the sleep time could be removed.

    By Administrator on 2013-10-18T09:14:35 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: jacobvosmaer

    @Rovanion I think we should keep the sleep 2 because the PID only gets written by Sidekiq after daemonization: https://github.com/mperham/sidekiq/blob/v2.14.0/lib/sidekiq/cli.rb#L39

    The fix I proposed just pushes the exit time of rake sidekiq:start closer to Sidekiq's write_pid.

    By Administrator on 2013-10-18T09:37:09 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: Rovanion

    I began typing a solution which would wait for the pid to be written, it's basically:

    while [ ! -f $pid_path ] do sleep 100ms;

    Perhaps a break after a set amount of time to declare failure would be good, but other than that I can't think of any. A patch doing this should probably be done either way because there will still be some machine where the 2 second timeout isn't enough. It also speeds up the script where the whole 2 seconds aren't needed.

    By Administrator on 2013-10-19T09:37:20 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: jacobvosmaer

    @Rovanion I agree. It would be nicer to have the polling loop with a timeout (10 seconds?) as you described. Could you make a PR for that?

    By Administrator on 2013-10-21T08:41:45 (imported from GitLab project)

  • Administrator
    Administrator @root · 11 years ago
    Owner

    Created by: Rovanion

    @jacobvosmaer These changes are now included in the pull request I got here: https://github.com/gitlabhq/gitlabhq/pull/5326

    By Administrator on 2013-10-21T21:24:55 (imported from GitLab project)

  • You're only seeing other activity in the feed. To add a comment, switch to one of the following options.
Please register or sign in to reply
0 Assignees
None
Assign to
Milestone
No milestone
None
None
Time tracking
Due date
None
None
0
Labels
None
Assign labels
  • No matching results
  • Manage project labels
Confidentiality
Not confidential

You are going to turn on confidentiality. Only team members with at least Reporter access will be able to see and leave comments on the issue.

Lock issue
Unlocked
participants
Reference:

Menu

Projects Groups Snippets
Help