Assigning Certain Issues Fail

[Mike Rawdon]

We are running Gitlab 8.9.6 assigning most issues works fine.  But a small number of issues in one of our projects cannot be assigned.

Using the Gitlab website the spinner never goes away and I get the following error in my JS http://gitlab/group/project/issues/466.json net::ERR_CONTENT_LENGTH_MISMATCH

curl -X GET -H "PRIVATE-TOKEN: *****" http://gitlab/api/v3/projects/14/issues/888

{"id":888,"iid":466,"project_id":14,"title":...

$ curl -X PUT -H "PRIVATE-TOKEN: *****" http://gitlab/api/v3/projects/14/issues/888?assignee_id=4

{"message":"404 Not found"}

The log from the website shows the following

==> /var/log/gitlab/gitlab-workhorse/current <==

2016-08-09_12:19:57.14059 2016/08/09 12:19:57 ErrorPage: serving predefined error page: 500

2016-08-09_12:19:57.14072 gitlab @ - - [2016-08-09 12:19:56.852076848 +0000 UTC] "PUT /group/project/issues/466.json HTTP/1.1" 500 0 "http://gitlab/group/project/issues/466" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36" 0.288612

==> /var/log/gitlab/nginx/gitlab_error.log <==

2016/08/09 12:19:57 [error] 31935#0: *62356 upstream prematurely closed connection while reading upstream, client: 10.1.0.165, server: localhost, request: "PUT /group/project/issues/466.json HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/socket:/group/project/issues/466.json", host: "gitlab", referrer: "http://gitlab/group/project/issues/466"

==> /var/log/gitlab/gitlab-rails/production.log <==

Started GET "/autocomplete/users.json?search=&per_page=20&active=true&project_id=14&current_user=true&author_id=10" for 127.0.0.1 at 2016-08-09 12:19:57 +0000

Processing by AutocompleteController#users as JSON

  Parameters: {"search"=>"", "per_page"=>"20", "active"=>"true", "project_id"=>"14", "current_user"=>"true", "author_id"=>"10"}

Completed 200 OK in 55ms (Views: 9.0ms | ActiveRecord: 3.2ms)

==> /var/log/gitlab/gitlab-workhorse/current <==

2016-08-09_12:19:57.20719 gitlab @ - - [2016-08-09 12:19:57.148387793 +0000 UTC] "GET /autocomplete/users.json?search=&per_page=20&active=true&project_id=14&current_user=true&author_id=10 HTTP/1.1" 200 2339 "http://gitlab/group/project/issues/466" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36" 0.058747

==> /var/log/gitlab/nginx/gitlab_access.log <==

10.1.0.165 - - [09/Aug/2016:12:19:57 +0000] "GET /autocomplete/users.json?search=&per_page=20&active=true&project_id=14&current_user=true&author_id=10 HTTP/1.1" 200 722 "http://gitlab/group/project/issues/466" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"

10.1.0.165 - - [09/Aug/2016:12:19:57 +0000] "PUT /group/project/issues/466.json HTTP/1.1" 500 0 "http://gitlab/group/project/issues/466" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"

From the API I see

==> /var/log/gitlab/gitlab-rails/production.log <==

Started PUT "/api/v3/projects/14/issues/888?assignee_id=4" for 127.0.0.1 at 2016-08-09 12:18:57 +0000

==> /var/log/gitlab/gitlab-workhorse/current <==

2016-08-09_12:18:57.42088 gitlab @ - - [2016-08-09 12:18:57.386632413 +0000 UTC] "PUT /api/v3/projects/14/issues/888?assignee_id=4 HTTP/1.1" 404 27 "" "curl/7.44.0" 0.034141

==> /var/log/gitlab/nginx/gitlab_access.log <==

10.1.0.165 - - [09/Aug/2016:12:19:57 +0000] "PUT /mitrend/wheatley/issues/466.json HTTP/1.1" 500 0 "http://gitlab:2789/mitrend/wheatley/issues/466" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"

From the API I see the following

==> /var/log/gitlab/gitlab-rails/production.log <==

Started PUT "/api/v3/projects/14/issues/888?assignee_id=4" for 127.0.0.1 at 2016-08-09 12:18:57 +0000

==> /var/log/gitlab/gitlab-workhorse/current <==

2016-08-09_12:18:57.42088 gitlab:2789 @ - - [2016-08-09 12:18:57.386632413 +0000 UTC] "PUT /api/v3/projects/14/issues/888?assignee_id=4 HTTP/1.1" 404 27 "" "curl/7.44.0" 0.034141

==> /var/log/gitlab/nginx/gitlab_access.log <==

10.1.0.165 - - [09/Aug/2016:12:18:57 +0000] "PUT /api/v3/projects/14/issues/888?assignee_id=4 HTTP/1.1" 404 27 "-" "curl/7.44.0"

[Drew Blessing]

Hi Mike,

I have never heard of this error before, but a quick search turned up https://gitlab.com/gitlab-org/gitlab-ce/issues/14113. According to this there may be a useful log entry if you follow the logs before trying to set the assignee on the problematic issue. Maybe you'll see a permission issue or some other problem. If you find anything in the logs, please let me know. 

[Mike Rawdon]

I had an issue where I was able to recreate this on again.  Running gitlab-ctl tail showed the errors below

==> /var/log/gitlab/gitlab-workhorse/current <==

2016-09-14_17:04:25.52396 gitlab @ - - [2016-09-14 17:04:25.301705828 +0000 UTC] "GET /api/v3/projects/14/labels?page=1&per_page=100 HTTP/1.1" 200 2948 "" "Go-http-client/1.1" 0.221943

==> /var/log/gitlab/gitlab-rails/production.log <==

Started PUT "/group/project_name/issues/735.json" for 10.1.0.165 at 2016-09-14 17:04:26 +0000

Processing by Projects::IssuesController#update as JSON

  Parameters: {"issue"=>{"assignee_id"=>"16"}, "namespace_id"=>"group", "project_id"=>"project_name", "id"=>"735"}

ActiveRecord::RecordNotFound (Couldn't find User with 'id'=0):

  app/models/concerns/issuable.rb:87:in `update_assignee_cache_counts'

  app/services/issuable_base_service.rb:149:in `block in update_issuable'

  app/services/issuable_base_service.rb:148:in `update_issuable'

  app/services/issuable_base_service.rb:163:in `update'

  app/services/issues/update_service.rb:4:in `execute'

  app/controllers/projects/issues_controller.rb:105:in `update'

  lib/gitlab/request_profiler/middleware.rb:15:in `call'

  lib/gitlab/middleware/go.rb:16:in `call'

Completed 500 Internal Server Error in 86ms (ActiveRecord: 35.4ms)

ActionView::MissingTemplate (Missing template mnt/md0/opt/gitlab/embedded/service/gitlab-rails/public/404 with {:locale=>[:en], :formats=>[:json], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :haml]}. Searched in:

  * "/mnt/md0/opt/gitlab/embedded/service/gitlab-rails/app/views"

  * "/mnt/md0/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/kaminari-0.17.0/app/views"

  * "/mnt/md0/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/doorkeeper-4.2.0/app/views"

  * "/mnt/md0/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/devise-4.1.1/app/views"

  * "/mnt/md0/var/opt/gitlab/gitlab-rails/working"

  * "/"

):

  app/controllers/application_controller.rb:142:in `render_404'

  app/controllers/application_controller.rb:36:in `block in <class:ApplicationController>'

  lib/gitlab/request_profiler/middleware.rb:15:in `call'

  lib/gitlab/middleware/go.rb:16:in `call'

==> /var/log/gitlab/gitlab-workhorse/current <==

2016-09-14_17:04:26.49727 2016/09/14 17:04:26 ErrorPage: serving predefined error page: 500

2016-09-14_17:04:26.49850 gitlab @ - - [2016-09-14 17:04:26.365425788 +0000 UTC] "PUT /group/project_name/issues/735.json HTTP/1.1" 500 0 "http://gitlab/group/project_name/issues/735" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 0.132124

==> /var/log/gitlab/nginx/gitlab_error.log <==

2016/09/14 17:04:26 [error] 30125#0: *39 upstream prematurely closed connection while reading upstream, client: 10.1.0.165, server: localhost, request: "PUT /group/project_name/issues/735.json HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/socket:/group/project_name/issues/735.json", host: "gitlab", referrer: "http://gitlab/group/project_name/issues/735"

[Drew Blessing]

This appears to be an error that occurs when a current/previous assignee no longer exists. There must have been a specific set of circumstances when the user with ID 0 was deleted that did not destroy these type of relationships. Now you see strange issues like this. You may have to change the assignee in the database so it's no longer ID 0, which corresponds to the non-existent user. 

[Mike Rawdon]

The issue was caused by 

https://gitlab.com/gitlab-org/gitlab-ce/issues/22664

Mike

On Wed, Sep 21, 2016 at 1:07 AM Drew Blessing <dr...@blessing.io> wrote:

This appears to be an error that occurs when a current/previous assignee no longer exists. There must have been a specific set of circumstances when the user with ID 0 was deleted that did not destroy these type of relationships. Now you see strange issues like this. You may have to change the assignee in the database so it's no longer ID 0, which corresponds to the non-existent user. 

--
You received this message because you are subscribed to a topic in the Google Groups "GitLab" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gitlabhq/GAA3xG5xYMU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gitlabhq+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gitlabhq/73360347-54b0-41a4-878a-cef12294f32c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.