How do I write general DigitalNZ queries.

[Robert Barnes]

I am trying to use the DigitalNZ API to retrieve data and integrate it with other data held in a SQL database.  My plan is that my ASP.NET web site will create a query for the DigitalNZ API, then use the information returned in combination with information it its own database.

My first test query was: -
    http://api.digitalnz.org/v3/records.xml?api_key=xxxxxxxx&text=Freda+Cook&fields=verbose
This returned XML from which I can get everything I want about the first 20 records.  The query tells me that there are 744 records that I might be interested in. It was relatively easy to put the XML into an XMLDocument and extract the information that I wanted into a table in my server-side ASP.NET code.

I then tried to explore the query language to find out what I could do with it. I need the fields id, title, and subject, and source_url would also be useful, but I don't need other fields in the returned data (although I do want to include date in the query - more of this later), so I replaced "fields=verbose" with "fields=id,title,source_url,subject".  The query now returned only title and subject, but id and source_url were not included. (I tried both source_url and source-url, but it made no difference).

If I could write a SQL query I'd write
    Set @Name = 'Freda Cook'
    Set @Year1 = 1896
    Set @Year2 = 1990
    SELECT id, title, subject, source_url FROM DigitalNZAPI WHERE text=@Name AND Date>=@Year1 AND Date <=@Year2

How do I write the equivalent query through the DigitalNZ API?  At the moment the only way I can see to do what I want is to return everything (Fields=verbose,) and throw away the fields that I don't want, and the records that I don't want because they have incorrect dates).

Thank you, Robert Barnes.

[Chris McDowall]

Hi Robert,

Unfortunately, not all fields are currently available through the "fields" API parameter. This is a bug that we will fix but, in the meantime, I suggest you continue using "&fields=verbose" and ignoring fields you don't want.

To get more data I recommend you use the "per_page" and "page" parameters to work through fields. The "per_page" field enables you to specify up to 100 results back in your query. The "page" parameter enables you to move through these results 

http://api.digitalnz.org/v3/records.xml?api_key=xxxxxxxx&text=Freda+Cook&fields=verbose&per_page=100&page=1

http://api.digitalnz.org/v3/records.xml?api_key=xxxxxxxx&text=Freda+Cook&fields=verbose&per_page=100&page=2

http://api.digitalnz.org/v3/records.xml?api_key=xxxxxxxx&text=Freda+Cook&fields=verbose&per_page=100&page=3

... and so on. Does that make sense?

I presented a search workshop at the National Digital Forum a couple of weeks ago around the DigitalNZ API that goes into depth on this issues and includes code samples. I will get the slides and a blogpost up shortly and respond to this thread when it is done.

Chris McDowall

Manager, DigitalNZ systems

[Robert Barnes]

Thanks Chris, that's what I was already doing as my "Interim" solution. Actually the problem is not as bad as I had thought: when I searched for "Freda Cook" with the quotes and without the + there were only 8 results returned, not the 744 that included records of Freda ??? climbing Mt Cook as well as records of the person of interest. But my program dies if I simply ask for "Cook". I have programmed it to get Page 2, 3, ... n if there are more than 20 results, but I'll put a limit of 100 on it so that both ends can cope.

I have another question, butI'll start another thread about this.

Regards, Robert.