Batch import stuck in processing

[Evan Laszlo]

I've set up a development instance of Bibapp, and have managed to work out most errors. I've tried to export an XML file formatted for Refworks, from Endnote, and have imported it into Bibapp. The job has been stuck in processing since yesterday afternoon. Is there a way I can stop it and perhaps try again? I've looked at most previous threads on here, but have not been successful so far.

I'd appreciate any advice.

[stirn...@gmail.com]

Evan,

There's a rake task for that :)

rake jobs:clear

Specifically, from the command line, you'll probably want to run

bundle exec rake jobs:clear RAILS_ENV=development

That should clear existing jobs from the delayed_job queue in your development database.

You can see all the available tasks with bundle exec rake -T

Restarting BibApp should always restart delayed_job - that's the queue worker that handles background jobs. However, once restarted it will pick up the job from the database and try again. So, you'll need to remove that job from the database using the rake task.

If it doesn't work for you, let us know.

Jason

On Mar 13, 2014, at 1:39 PM, Evan Laszlo <laszl...@gmail.com> wrote:

I've set up a development instance of Bibapp, and have managed to work out most errors. I've tried to export an XML file formatted for Refworks, from Endnote, and have imported it into Bibapp. The job has been stuck in processing since yesterday afternoon. Is there a way I can stop it and perhaps try again? I've looked at most previous threads on here, but have not been successful so far.

I'd appreciate any advice.

--
You received this message because you are subscribed to the Google Groups "bibapp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bibapp+un...@googlegroups.com.
To post to this group, send email to bib...@googlegroups.com.
Visit this group at http://groups.google.com/group/bibapp.
For more options, visit https://groups.google.com/d/optout.

[Evan Laszlo]

Oddly, that's not working, even though I've tried that very command multiple times. It seems to run after I issue the command, but the jobs don't disappear from the IMPORTS interface. They status is "processing" and if I click on one of them I get "This import is in processing. You'll receive an email when the batch is complete".

[root@bib controllers]# bundle exec rake jobs:clear RAILS_ENV=development
Your Gemfile lists the gem database_cleaner (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
Your Gemfile lists the gem cucumber-rails (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
(in /home/bibapp/BibApp)
WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.7.6
WARNING: Cucumber-rails required outside of env.rb.  The rest of loading is being deferred until env.rb is called.
  To avoid this warning, move 'gem 'cucumber-rails', :require => false' under only group :test in your Gemfile.
  If already in the :test group, be sure you are specifying ':require => false'.
[deprecated] I18n.enforce_available_locales will default to true in the future. If you really want to skip validation of your locale you can set I18n.enforce_available_locales = false to avoid this message.
[root@bib controllers]#

[Evan Laszlo]

What's odd is that when I run:

bundle exec rake jobs:clear RAILS_ENV=development

it is clearing the jobs from the bibapp_development.delayed_jobs table, yet they still show under the IMPORTS window.

On Thursday, March 13, 2014 2:04:51 PM UTC-7, Jason Stirnaman wrote:

[Jason Stirnaman]

Evan,
That Import row will still get stuck with status "processing", but it should be removed from the delayed_job queue.

You can attempt to re-import and watch your delayed-job.log to find out what's going wrong.

You can inspect, requeue, or remove that import job through Rails console, 

bundle exec rails c development

Some examples:

Read your raw import file:

> Import.first.read_import_file

Process your import and watch it fail from the console:

> Import.find([your_import_id]).batch_import

Requeue your import:

> Import.find([your_import_id]).queue_import

Or, delete your import record entirely:

> Import.destroy([your_import_id])

I don't think there's anything in the UI for #destroy.

Jason

[fur...@gmail.com]

Evan,

Have you made sure that your app is set up to send email correctly?  Did you configure smtp.yml?

In the past, some of my imports have gotten stuck because they get hung up at the notification email stage.

[Evan Laszlo]

The development instance is not sending emails correctly. I read in another thread that the development environment does not send emails, but I don't know if that's true. That's why I had not bothered with it.

The server where Bibapp runs from has postfix/sendmail configured to send through a campus relay. I've tested that successfully and it works.

I've also configured the smtp.yml with the relay information, but for some reason emails are not sent.

Seems like I have two problems, it sounds. one being the emails that are not being sent, and the other being the imports that are not processing correctly.

On Thu, Mar 13, 2014 at 7:23 PM, fur...@gmail.com <fur...@gmail.com> wrote:

Hi Evan,

I don't believe that the rake task will actually clear out the job listing on the imports page.

One other thing to double check is if you have setup your instance to correctly send mail.  Did you configure smtp.yml?

From past experience, when the import does not finish and I don't receive the email indicating that it's done there's been a problem with the app sending mail.

-John

On Thursday, March 13, 2014 6:50:29 PM UTC-4, Evan Laszlo wrote:

What's odd is that when I run:
bundle exec rake jobs:clear RAILS_ENV=development

it is clearing the jobs from the bibapp_development.delayed_jobs table, yet they still show under the IMPORTS window.

On Thursday, March 13, 2014 2:04:51 PM UTC-7, Jason Stirnaman wrote:

Evan,

There's a rake task for that :)

rake jobs:clear

Specifically, from the command line, you'll probably want to run

bundle exec rake jobs:clear RAILS_ENV=development

That should clear existing jobs from the delayed_job queue in your development database.

You can see all the available tasks with bundle exec rake -T

Restarting BibApp should always restart delayed_job - that's the queue worker that handles background jobs. However, once restarted it will pick up the job from the database and try again. So, you'll need to remove that job from the database using the rake task.

If it doesn't work for you, let us know.

Jason

On Mar 13, 2014, at 1:39 PM, Evan Laszlo <laszl...@gmail.com> wrote:

I've set up a development instance of Bibapp, and have managed to work out most errors. I've tried to export an XML file formatted for Refworks, from Endnote, and have imported it into Bibapp. The job has been stuck in processing since yesterday afternoon. Is there a way I can stop it and perhaps try again? I've looked at most previous threads on here, but have not been successful so far.

I'd appreciate any advice.

--
You received this message because you are subscribed to the Google Groups "bibapp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bibapp+un...@googlegroups.com.
To post to this group, send email to bib...@googlegroups.com.
Visit this group at http://groups.google.com/group/bibapp.
For more options, visit https://groups.google.com/d/optout.

[Jason Stirnaman]

Development environment doesn't send emails by default. To change that, look at config/environments/development.rb

Re: imports,

That Import row will still get stuck with status "processing", but it should be removed from the delayed_job queue.
You can attempt to re-import and watch your delayed-job.log to find out what's going wrong.

You can inspect, requeue, or remove that import job through Rails console, 
bundle exec rails c development

Some examples:

Read your raw import file:
> Import.first.read_import_file

Process your import and watch it fail from the console:
> Import.find([your_import_id]).batch_import

Requeue your import:
> Import.find([your_import_id]).queue_import

Or, delete your import record entirely:
> Import.destroy([your_import_id])

I don't think there's anything in the UI for #destroy.

Jason