Records do not show up

[edwardiglesias]

I've been trying to import records using bothe the catalog importer
and the III harvester. In both cases the records seem to be coming in
but then do not get published. Using the sample record set from
remaining relevant I use

The source prefix:
ii

example: bb (must be two characters, a-z and 0-9 accepted)

The source ID field:

['001'][0]->data

The records appear

Reading the file and parsing 336 records. Please be patient.

1. The Children Of Noisy Village ii
2. Wishbone: Salty Dog ii
3. The Prime Of Miss Jean Brodie ii
4. Silas Marner ii
5. The Year Of Living Dangerously ii
6. Bill Nye The Science Guy : Dinosaurs - Those Big Boneheads ii

etc... Ending with

Processing complete.

1008 of 336 records harvested. with 1008 warnings and 0 errors.

Continue to the next step to publish those harvested catalog entries.

At which point only this appears

Scriblio Catalog Importer

That's all folks. kthnxbye.

Any ideas?

[don....@gmail.com]

Hey Edward:

I think your problem might be with

The source ID field:

['001'][0]->data

Do your MARC records consistently have data in that field (336 of 1008
records)?

You might want to look for 035 (System control number) to see if
there's data there.

Alternately when you export your MARC records you could include a
unique identifier and code it to a specific MARC field/indicator?

dm

[Edward Iglesias]

I tried replacing


['001'][0]->data

with

['907'][0]->subfields['a']

with the same result.  I double checked and there is data in both those fields

=001  ocm10777692
=907  \\$a.b10004592$bmulti$c-

Perhaps my syntax is off.  They show up on the screen

Reading the file and parsing 475 records. Please be patient.

1. Chaucer ii.b10004592
2. Chaucer And Dante : A Revaluation ii.b10060339
3. Editing Chaucer : The Great Tradition ii.b10061174

and in the directory

localhost:/usr/local/apache2/htdocs/wp/wp-content/uploads/2008/04# ls

chaucerout.import

I get warnings

475 of 475 records harvested. with 475 warnings and 0 errors.

Like this

1. Invalid record length "01269"
2. Invalid record length "01298"
3. Invalid record length "01379"

 But when I go to "publish harvested records" they vanish.

E

--
Edward Iglesias

[Casey Bisson]

Edward,

Published records are not shown on the front page of your site[1]. Example:

http://library.plymouth.edu/

That front page is inside Scriblio, but you have to go to this URL to see the new books:

http://library.plymouth.edu/browse

The /browse/ URL is the default, so it'll probably work on your site. You can also do a search on your site, and the records should appear there. Here's an example from my site:

http://library.plymouth.edu/search/history

I should note that all of this requires that you have permalinks setup and a working .htaccess file.

http://codex.wordpress.org/Glossary#Permalink

http://codex.wordpress.org/Glossary#.htaccess

http://about.scriblio.net/wiki/htaccess

Finally, you should also be able to see the records if you go to your WordPress dashboard, click on the Manage tab, then select Author: cataloger from the filter list, and click Filter.

I might be way off base with those answers, but don't hesitate to tell me so,

--Casey

[1]: Scriblio is a CMS, not just a catalog. The front page can be used to post news or feature interesting works in the collection. Alternatively, you can change the code to post new titles to the front page.

[Edward Iglesias]

Thanks Casey, but that was the first thing we tried.  They simply do not show up as posts either in browse or by just looking.  I have permalinks set to numeric and .htaccess set to 777.

E

--
Edward Iglesias

[Casey Bisson]

Can you look at your posts mysql table to see what's there?

[Dennis Whiteman]

Casey and Edward...

My problem has been the same and I think I have a clue...

When I've tried to import into my existing test Wordpress install,
only one new table is created in addition to the standard wordpress
tables -- wp_bsuite3_search. I just did a virgin Wordpress install to
see if that would work and these additional tables were added --
wp_bsuite3_hits, wp_bsuite3_refs_terms, wp_bsuite3_search,
wp_scrib_harvest. So for whatever reason, some tables were not being
created by the plug-ins in my Wordpress 2.5 install, but were added to
my 2.3.1 install. I only used 2.3.1 because that's something I already
had setup for testing; I'll definitely deploy on 2.5.

That said, only the last record of my 5 MARC record is actually
imported, but I think that's a different problem and at least I have
some idea of how to set up the tables to convert my existing data
over. I'm still a little confused about the source-id stuff so that
could be the reason only one of my records is being imported (zm and
[852][0]->subfields['p']).

I followed the Screencast this time when creating a new Wordpress
install, rather than attempting to import into my existing Wordpress
development install.

Dennis

> >> On Wed, Apr 16, 2008 at 9:40 AM, don.mo...@gmail.com <don.mo...@gmail.com

[Casey Bisson]

Dennis,

Thanks for these details. I'm looking into it.

--Casey

[brownthebook]

Hi all,

I just came back to Scriblio today and did a fresh install of the
latest svn version of Wordpress and Scriblio.

I followed the screencast, and I have what seems like exactly the same
problem as Edward.

My rewrites are working fine at last after some tweaking of Apache,
and as I'd previously loaded some sample MARC records into Scriblio at
my first attempt many months ago, I thought I'd done the hard bit ;-)

I get the invalid record warnings and my records are also nowhere to
be seen when I return the browse url.

I've tried loading Casey's sample records and a recordset of my own
and I get the same results with both.

On a related issue, what is the [0] in the statement below? Is it to
indicate that Scriblio should pickup the first 035 field, or is it
connected with incidicators in the marc record?

['035'][0]->subfields['a']

It feels like I'm getting close - will follow this thread to see what
develops

Andrew

[Casey Bisson]

As Dennis reported, there appears to be a problem with table creation.
I'm working on it now and I'll update SVN as soon as I get a fix.

The invalid record length warnings are a bug in PHP MARC[1], the code
library Scriblio uses to read the MARC files. Dan Scott recommends
File_MARC[2], which has the advantage of being actively maintained,
but I've not had a chance to look into it. The short answer is that
you can usually ignore record length warnings.

[1]: http://sourceforge.net/projects/phpmarc/
[2]: http://pear.php.net/package/File_MARC/

The syntax of the source ID field comes from the way arrays are
addressed in PHP and the structure of the array that PHP MARC creates
to represent the MARC record. In the example below, [0] means the
first 035 field, just in case there's more than one.

--Casey

On Apr 18, 2008, at 10:09 AM, brownthebook wrote:

> I get the invalid record warnings and my records are also nowhere to
> be seen when I return the browse url.

> On a related issue, what is the [0] in the statement below? Is it to

[Casey Bisson]

After some experimentation, I think I narrowed this down to a problem
with the plugin activation hook for the catalog importer plugin. I
haven't figured out what the problem is, but I've added a work around
that will make sure the wp_scrib_harvest table is created each time
you use the importer.

Can anybody who's been struggling with this please update via SVN and
confirm that this solves the problem?

[Edward Iglesias]

I did an


svn update                  
Fetching external item into 'plugins/akismet'
External at revision 42035.

At revision 7760.

This did not change anything.  These are the tables in the database scriblio.  I do not see wp_scrib_harvest.


mysql> show tables;
+-----------------------+
| Tables_in_scriblio    |
+-----------------------+
| wp_bsuite3_hits       |
| wp_bsuite3_refs_terms |
| wp_bsuite3_search     |
| wp_comments           |
| wp_links              |
| wp_options            |
| wp_postmeta           |
| wp_posts              |
| wp_term_relationships |
| wp_term_taxonomy      |
| wp_terms              |
| wp_usermeta           |
| wp_users              |
+-----------------------+
13 rows in set (0.00 sec)

--
Edward Iglesias

[Casey Bisson]

I wish there was a way to tell SVN to fetch new sources for everything
in a directory (please save me from my ignorance if there is), but I
think you'll have to do an 'svn up' inside the scriblio directory of
your plugins directory. You should get some messages about revision 58
and it should download at least the new importer.php, if not some
other files.

Then, after the SVN update, you'll need to try using the plugin before
the new table will show up.

--Casey

[Casey Bisson]

Dennis, I spent a while rummaging around in the code to remember which
tables are created when/by what plugin. Here's the list:

bSuite:
creates the wp_bsuite3_search table at activation

bSuite bStat:
creates the wp_bsuite3_hits and wp_bsuite3_refs_terms tables at
activation

Scriblio:
creates the wp_scrib_suggest when you click the "rebuild search
suggest table" button.

Scriblio catalog importer:
creates the wp_scrib_harvest table when you first use the importer
(though it's supposed to create it when activated).

Thanks for pointing out that a missing table might be the issue,

--Casey

On Apr 17, 2008, at 11:47 AM, Dennis Whiteman wrote:

[Dennis Whiteman]

Thanks Casey. I'm grading my student's final projects this week, but
I'll jump back into this as soon as I come up for air...

Dennis

[Casey Bisson]

Jonathan,

I'm grasping at straws, but can you confirm that the activate()
function (which creates the tables) is called inside greet() in your
importer.php?
function greet() {
$this->activate();
the greet() function is called when you go to Dashboard->Manage-
>Import->Scriblio Catalog Importer

On Apr 21, 2008, at 4:44 PM, Jonathan wrote:
> New to Scriblio - pretty keen on getting it set up and running. I'm
> running into an issue with the importing, though... I'm using the
> Scriblio III Catalog Importer on a test set of records from our III
> OPAC. It processes without any warnings/errors: 146 records
> harvested. with 0 warnings and 0 errors.
>
> But when I click on 'Publish Harvested Records,' nothing happens but
> the 'That's all folks. kthnxbye' message. Browsing to the /browse
> directory confirms that nothing was imported.
>
> Like another thread here, I don't have a wp_scrib_harvest table.
>
> Using current version of the scriblio plugin via svn (4/21 4:30pm)
>
> Any ideas? I followed the screencast and everything else works as
> advertised, but I may have missed something critical?
>
> -Jonathan

[Jonathan]

Hi Casey,

Yes, the activate() function is inside greet().

I got the import function to work a few minutes ago by manually
creating the wp_scrib_harvest table in my db. After I did that, the
import worked like a charm. Hope that helps diagnose.

Thanks,
Jonathan.

[Casey Bisson]

Well, it confirms that the overall problem is related to the missing
table, but I'm still a bit confused about why that one table isn't
getting created.

Thanks for your persistence, glad it's finally working for you.

[Edward Iglesias]

Casey,

I got it to work after doing a fresh install.  I was able to upload the test file and it worked well though not all of the covers seem to show up.

Thanks for all your help.

Edward

--
Edward Iglesias

[brownthebook]

Casey,

Thanks for the reply - I did the svn update and your sample MARC file
now loads just fine.

I still receive the invalid record length errors - let me know if
you'd like more info on that.

I'm now moving on to loading some of our own records and will report
back.

Thanks for a great development!

Andrew


On Apr 22, 4:03 pm, "Edward Iglesias" <edwardigles...@gmail.com>
wrote:

> Edward Iglesias- Hide quoted text -
>
> - Show quoted text -

[Eunice Riesberg]

Edward,

How far back did you go on the fresh install? I think I've got the
same problem.

Eunice

At 09:03 AM 4/22/2008, you wrote:
>Casey,
>
>I got it to work after doing a fresh install. I was able to upload
>the test file and it worked well though not all of the covers seem to show up.
>
>Thanks for all your help.
>
>Edward
>
>On Tue, Apr 22, 2008 at 10:55 AM, Jonathan
><<mailto:jyou...@gmail.com>jyou...@gmail.com> wrote:
>
>Hi Casey,
>
>Yes, the activate() function is inside greet().
>
>I got the import function to work a few minutes ago by manually
>creating the wp_scrib_harvest table in my db. After I did that, the
>import worked like a charm. Hope that helps diagnose.
>
>Thanks,
>Jonathan.
>
>On Apr 22, 9:38 am, Casey Bisson

Eunice Riesberg
Northeast Iowa Library Service Area
415 Commercial St.
Waterloo, IA 50701
319-233-1200, 800-772-2023
319-233-1964 (fax)
ries...@neilsa.org
http://www.neilsa.org

[Edward Iglesias]

Scorched earth.  I made a new directory and did a fresh svn install of wordpress all all of its friends.

E

--
Edward Iglesias

[davidkane]

Casey,

In reply to the note on the Scriblio site: (http://about.scriblio.net/
wiki/installing-scriblio)

I do have wp_scrib_harvest in my DB.

I'll do a clean install and try again.

David.

[David Kane]

Did a clean install and got all the latest versions of the archives etc.

The table wp_scrib_harvest did not automatically appear

I was able to harvest from the catalogue (about 50 records) but these did not get published.

The reason that it was on the last one was because it was an upgrade from WP2.3.x I had an implementation of Scriblio on that, which I got rid of. With this clean install the database table does not get created.  Even when it was there though, It did not get written to.  So maybe that's something.

David.

2008/5/25 davidkane <david...@gmail.com>:

--
David Kane
Systems Librarian
Waterford Institute of Technology
http://library.wit.ie/
T: ++353.51302838
M: ++353.876693212

[David Kane]

I set:

ini_set("display_errors","2");
ERROR_REPORTING(E_ALL);

in my admin.php and ran the publish imported records script again ( /wp-admin/admin.php?import=scribimporter&step=3 )

Nothing shows up.

Perhaps there is something else I can override in the php.ini?

Saved settings in Scriblio and Bsuite plugins but the table did not appear.

David.

2008/5/25 David Kane <david...@gmail.com>:

[Casey Bisson]

David,

Are you installing via SVN? The version now in SVN (but not in the tar file) is a little more persistent about trying to create the table (it appears the normal plugin activation hook may not be triggering reliably in WP 2.5.x).

You can see the code to create the table in the activate() function here:

http://svn.scriblio.net/plugin/trunk/importer.php

If nothing else, you could create the table manually.

More information about SVN, including a log of changes to the code is here:

http://about.scriblio.net/wiki/svn

The SVN log looks quiet, but don't let it deceive you. I've been working on some WP2.5.1 optimizations and enhancements, and HKUST just shared their enhanced code with me. There's a lot in the pipe.

--Casey