Web API

From ISFDB
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Introduction

At this time the ISFDB supports the following web APIs:

  • getpub.cgi which takes an ISBN and returns publication data for all matching publications
  • getpub_by_ID.cgi which takes an External ID, e.g. an ASIN, and returns publication data for matching publications
  • getpub_by_internal_ID.cgi which takes an ISFDB publication number and returns its publication data
  • submission.cgi which is used to submit data to the ISFDB

There are currently no methods for obtaining author, title, series, or awards information from the ISFDB.

License Keys

Retrieving data from the ISFDB using the Web API doesn't require a license key.

Submitting data to the ISFDB via the Web API requires a valid user ID and a license key. When logged into the ISFDB with your user ID, a license key can be generated by visiting https://www.isfdb.org/cgi-bin/edit/keygen.cgi and clicking on [Generate New Key]. For data submissions this key must appear within a LicenseKey XML tag following the Submitter tag. In addition, the submitting user's name must be added to the whitelist of authorized submitters maintained via a bureaucrat-only menu option. Post on the ISFDB Moderator Noticeboard to request access.

Publication Lookups

At this time the following ways of retrieving publication data from the ISFDB database are supported:

  • by ISBN (getpub.cgi)
  • by External ID (getpub_by_ID.cgi)
  • by internal ISFDB record number/ID (getpub_by_internal_ID.cgi)

The XML data returned by these APIs is identical. A license key is not required to use them. A valid query returns an XML payload which includes the following:

  • the Records tag which indicates the number of records found
  • zero or more Publication records containing the metadata for matching ISFDB publication record(s)

Example: 

<?xml version="1.0" encoding="iso-8859-1" ?>
<ISFDB>
 <Records>1</Records>
 <Publications>
   <Publication>
     <Record>325837</Record>
     <Title>The Winchester Horror</Title>
     <Authors>
       <Author>William F. Nolan</Author>
     </Authors>
     <Year>1998-12-22</Year>
     <Isbn>1881475530</Isbn>
     <Publisher>Cemetery Dance Publications</Publisher>
     <PubSeries>Cemetery Dance Novella</PubSeries>
     <PubSeriesNum>6</PubSeriesNum>
     <Price>$30.00</Price>
     <Pages>111</Pages>
     <Binding>hc</Binding>
     <Type>CHAPBOOK</Type>
     <Tag>THWNCHSTRH1998</Tag>
     <CoverArtists>
       <Artist>Eric Powell</Artist>
     </CoverArtists>
     <Note>Hardcover Limited Edition of 450 signed and numbered copies bound in full-cloth and Smyth sewn</Note>
     <External_IDs>
       <External_ID>
          <IDtype>1</IDtype>
          <IDtypeName>ASIN</IDtypeName>
          <IDvalue>B01G1K1RV8</IDvalue>
       </External_ID>
     </External_IDs>
     <Webpages>
        <Webpage>https://babel.hathitrust.org/cgi/pt?id=nnc1.cr60139269</Webpage>
     </Webpages>
     <TransTitles>
        <TransTitle>The Winchester Horror</TransTitle>
     </TransTitles>
   </Publication>
 </Publications>
</ISFDB>

getpub.cgi

The getpub.cgi API takes an ISBN as its argument. If more than one publication is associated with the requested ISBN, multiple publication records will be returned. If an ISBN-10 is submitted, the API will retrieve and return the data for the requested ISBN as well as for the equivalent ISBN-13 (and vice versa.) The URL for getpub.cgi should appear as follows: https://www.isfdb.org/cgi-bin/rest/getpub.cgi?0439176832

getpub_by_ID.cgi

The getpub_by_ID.cgi API takes two arguments. The first argument is the External ID type -- see the leftmost column on the Advanced Publication Search by External Identifier page for a list of currently supported External ID types. The second argument is the External ID value.

If more than one publication is associated with the requested External ID, multiple publication records will be returned. The URL for getpub_by_ID.cgi should appear as follows: https://www.isfdb.org/cgi-bin/rest/getpub_by_ID.cgi?ASIN+B0764JW7DK

getpub_by_internal_ID.cgi

The getpub_by_internal_ID.cgi API takes one argument, which must be the internal number/ID of the requested publication. The URL for getpub_by_internal_ID.cgi should appear as follows: https://www.isfdb.org/cgi-bin/rest/getpub_by_internal_ID.cgi?100 .

Note that, since internal publication IDs are unique within the ISFDB database, this API always returns one record, but it uses the same XML structure as what is used by publication-specific APIs which can return multiple records.

Example python script utilizing getpub.cgi

Although publication lookup APIs can be invoked via a browser, a more practical application is to invoke a URL via a programming interface, parse the result with an XML parser, and then do something with the resulting data. The following example shows the Python code needed to obtain the XML data using getpub.cgi; it does not show an example of parsing the XML payload: 

import httplib

host = "www.isfdb.org"

def GetXml(isbn):
      webservice = httplib.HTTP(host)
      command = '/cgi-bin/rest/getpub.cgi?%s' % isbn
      webservice.putrequest("GET", command)
      webservice.putheader("Host", host)
      webservice.putheader("User-Agent", "Wget/1.9+cvs-stable (Red Hat modified)")
      webservice.endheaders()
      errcode, errmsg, headers = webservice.getreply()
      if errcode != 200:
              resp = webservice.getfile()
              print "Error:", errmsg
              print "Resp:", resp.read()
              resp.close()
              return ''
      else:
              resp = webservice.getfile()
              raw = resp.read()
              resp.close()
              index = raw.find('<?xml')
              return raw[index:]

Error Conditions

If no matching publication records are found, publication lookup APIs will return an XML structure with the number of records set to zero: 

<?xml version="1.0" encoding="iso-8859-1" ?>
<ISFDB>
  <Records>0</Records>
  <Publications>
  </Publications>
</ISFDB>

If the submitted query contains invalid parameters, the invoked Web API will return a short error message explaining the nature of the error.

submission.cgi

The submission.cgi application allows a remote user to create ISFDB submissions programmatically. Note that the use of submission.cgi requires a valid License Key. See Web_API#License_Keys for instructions on obtaining a License Key.

Any of the data payloads defined in the Data Submission Formats article are accepted. The format of the XML data is identical to that used for local submissions, with the exception that the user's license key must be inserted after the Submitter tag. The user's registered login name should be contained within the Submitter tag, and the user's current license key should be contained with the LicenseKey tag. The URL for submission.cgi should appear as follows:

https://www.isfdb.org/cgi-bin/rest/submission.cgi

Submissions are placed in the moderator queue as with an ISFDB edit, and are not integrated into the database until a moderator approves it. As such it is important that you not flood the moderator queue with hundreds of submissions. In general you should only post no more than 20 submissions per session, until they are processed out by a moderator. If you want a submission to be automatically put on hold on behalf of a particular moderator, add <Holder>moderator_user_name</Holder> after <IsfdbSubmission>.

An example submission XML payload follows, with the LicenseKey x'd out: 

<?xml version="1.0" encoding="iso-8859-1" ?>
<IsfdbSubmission>
 <Holder>Ahasuerus</Holder>
 <NewPub>
   <Submitter>Dissembler</Submitter>
   <LicenseKey>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</LicenseKey>
   <Subject>Lady of the Glen: A Novel of 17th-Century Scotland and the Massacre of Glencoe</Subject>
   <Isbn>1575661292</Isbn>
   <Title>Lady of the Glen: A Novel of 17th-Century Scotland and the Massacre of Glencoe</Title>
   <Year>1997-01-01</Year>
   <Price>$14.95</Price>
   <Language>English</Language>
   <Publisher>Kensington</Publisher>
   <PubSeries>Time Romances</PubSeries>
   <PubSeriesNum>17</PubSeriesNum>
   <Pages>432</Pages>
   <Authors>
     <Author>Jennifer Roberson</Author>
   </Authors>
   <Binding>tp</Binding>
   <PubType>NOVEL</PubType>
   <Image>https://ecx.images-amazon.com/images/I/519xedh9nrL.jpg</Image>
   <Note>Data from Amazon.com as of 2008-05-08</Note>
 </NewPub>
<IsfdbSubmission>

Example python script utilizing submission.cgi

The submission.cgi application uses the http POST method, so it can not be invoked via a web browser. Once a proper XML payload has been constructed as described above, a connection needs to be made to the ISFDB web server, and the appropriate http header needs to be sent, including content type and length information. This example below shows how this can be implemented in python: 

import httplib

host = "www.isfdb.org"

def SendXml(xmlmsg):
      webservice = httplib.HTTP(host)
      webservice.putrequest("POST", "/cgi-bin/rest/submission.cgi")
      webservice.putheader("Host", host)
      webservice.putheader("Content-type", "text/xml; charset=\"iso-8859-1\"")
      webservice.putheader("Content-length", "%d" % len(xmlmsg))
      webservice.endheaders()
      webservice.send(xmlmsg)

      errcode, errmsg, headers = webservice.getreply()
      if errcode != 200:
              resp = webservice.getfile()
              print "Error:", errmsg
      else:
              resp = webservice.getfile()
      resp.close()

Here is a "real world" Python 3 script that uses the API, involving modern libraries such as Requests (for HTTP) and Untangle (for XML parsing).

Successful Completion

If submission.cgi successfully creates a submission, it will return an HTTP 200 status code. It will also return an XML structure with a "Status" value of "OK" and the "SubmissionID" value with the ID of the created submission record, e.g.: 

<?xml version="1.0" encoding="iso-8859-1" ?>
<ISFDB>
  <Status>OK</Status>
  <SubmissionID>123456</SubmissionID>
</ISFDB>

where "123456" is the ID of the submission that was created.

Note that the Web API only performs high level validation of the XML payload to ensure that it contains a valid XML structure. It does not perform the checks for potential issues that the manual submission creation process performs and it doesn't return any of the warnings that post-submission pages display.

Error Conditions

In the event of an error condition, submission.cgi will return an appropriate HTTP status code and an XML structure with a "Status" value of "FAIL". The cause of the error will be found within the "Error" tag, e.g.: 

<?xml version="1.0" encoding="iso-8859-1" ?>
<ISFDB>
  <Status>FAIL</Status>
  <Error>Bad XML data</Error>
</ISFDB>

The following HTTP status codes and errors are currently supported:

  • 400:
    • Malformed XML data
    • XML parsing failed
  • 401:
    • No Submitter Field
  • 403:
    • This user is not authorized to create submissions via the ISFDB Web API. Post on the ISFDB Moderator Noticeboard if you need access.
    • Invalid submitter
    • No LicenseKey Field
    • Invalid License Key
  • 422:
    • Invalid holding moderator
    • Specified holder is not a moderator
    • Unknown Submission Type
Retrieved from "https://isfdb.org/wiki/index.php?title=Web_API&oldid=666196"