User:Fixer

Fixer is a robot account, which is used to create submissions via the new Web API. It is currently owned by User:Ahasuerus. Fixer 13:08, 30 May 2008 (UTC)

Fixer Queues as of 2014-02-07
Legend: n-p - new (unprioritized) paper books, n-e - new ebooks, 1-p - queue 1 (high priority) paper books, 1-e - queue 1 ebooks

Mon      2014                   2013                   2012 n-p n-e  1-p  1-e|   n-p  n-e   1-p  1-e |   n-p  n-e   1-p   1-e| NON  3    0    0    0|    33    0     4    0|    357   0      0     0| Jan  0  418   73  228|     0   56     7    1|    703   3     37     0| Feb  0  107   69   51|     1   50     2    0|      0   1    166     0| Mar  0    0  322  183|     0   65     5    0|   1398   4      5     0| Apr 667 174   39    7|     0   54     9    0|   1193   6      4     0| May 616 183   14    8|     0   66     4    1|   1244   6      4     0| Jun 465 119   12    8|     0   84     7    1|   1119   5      1     0| Jul 476  96   21   21|     0  517    17    2|    975   8      3     0| Aug 385  64    8    6|     0 1181    13    6|   1249   6      7     0| Sep 308  26    5   16|     1 1260    46    0|   1430  16      8     0| Oct 145  11    3    0|     1 1751   175    6|      0  23    392     0| Nov 58    6    2    0|   672 1603     4    1|      0  12    220     0| Dec 24    2    2    0|     5 1698   131    6|      0  11    207     0|

Note: Queues 2 and 3 contain lower priority ISBNs and are not shown here.

How Fixer Works as of 2010
User:Ahasuerus is in the process of improving Fixer, so things are rather fluid, but here is how things currently work:


 * 1) Fixer queries Amazon.com and Amazon UK (Amazon CA is under construction) for a particular subject, author and/or time period. There are various (often obscure and/or complex) Amazon-imposed limits on what you can query for and how many items are returned, so it's not as simple as it sounds.
 * 2) Amazon sends some data for the books that match the query back to Fixer. Note that the data that Amazon sends back is not always the same as the data displayed on Amazon Web pages, e.g. Fixer doesn't have access to cover artists.
 * 3) If the ISBN of the incoming book has been submitted to ISFDB in the past -- or previously suspended/rejected -- then the book is ignored. Otherwise Fixer adds the book's data to the main "queue".
 * 4) Once all books matching the query have been returned by Amazon (which takes anywhere from minutes to hours depending on the query), Fixer looks for any ISBNs that were returned by only one of the two Amazon stores and queries the other store. This happens frequently when a book is released first in one country and then the other.
 * 5) At this point all data for all eligible books is in the main queue. Typically, each ISBN will have two records, one from Amazon.com and one from Amazon UK.
 * 6) Next, Fixer tries to determine which store's data to use. For example, if Amazon.com says that the publisher is "Baen" and Amazon UK says that the publisher is "Unknown", then chances are that Fixer should use the US record to create the ISFDB submission. Ideally, this would be an automated process, but there are many cases when both records look plausible, e.g. the respective prices may be $17.99 and £11.99, and it requires manual intervention. Hopefully, the automated identification process will get better over time as Fixer learns which publishers are "US only" and which ones are "UK only". Any publishers that are active on both side of the Atlantic, e.g. Orbit, will still need to be handled manually.
 * 7) Next, Fixer decides whether the ISBN should be automatically suspended. This is currently done for any ISBNs starting with "2" or "3" (France and Germany), any audio, CD and MP3 books, and, lately, for any books published by the better known vanity publishers. We will get to all of them -- eventually -- but for now I didn't want the submission queue to grow unmanageably long.
 * 8) Next, a manual process is run. I review all books in the queue and decide whether it should be submitted, suspended or rejected. Unfortunately, there are quite a few cases where the logic is just too complex for Fixer to handle, e.g. TokyoPop publishes both manga and written versions of certain series and there is no easy way of teaching Fixer which one is which. Besides, Amazon's data is often quite dirty, which makes the number of potential permutations too high for a poor overworked robot to handle reliably.
 * 9) If I decide to submit the book, Fixer marks the ISBN as "submitted", builds the submission and sends it to ISFDB.

In the past Fixer worked with less human supervision, which made it easier for me to run, but also resulted in more bad submissions and more work for moderators. As Fixer gets better at making some of these decisions, it should become a faster and smoother process both for me and for the approving moderators.

Next steps:


 * Amazon CA -- in progress.
 * Australian sources -- some 20,000 records have been captured, but I still need to parse them before Fixer can create submissions.
 * Library catalogs -- a number of major catalogs have been captured, but the data will require a lot of massaging before it is ready to be submitted. Suffice it to say that there are well over a thousand meaningful fields in the most popular standard for catalog records. Although not all of them are required for our purposes, quite a few need to be included in Moderator Notes since they may facilitate the decision making process at approval time. (And then there are catalogs with non-standard formats or no formatting at all, but that's a whole different headache.)


 * Ahasuerus 20:46, 30 March 2010 (UTC)

Publishers processed

 * Pyr - done (5)
 * Roc - done (50)
 * Baen - done (70)
 * Ace - done (80)
 * Tor - done (450?)
 * Del Rey - done (396)
 * Tandem - done (but not Tandem-somethingelse)
 * Gollancz - Amazon UK only

Amazon and friends

 * Check books published by "Telos" in 2009 and 2010
 * Capture Amazon CA
 * 2 sources for AU data
 * Combine US/UK/CA/AU records and upgrade the logic to submit the resulting composite records intelligently
 * Upgrade notes/modnotes
 * Add support for "creators", e.g. Role="Editor" and Role="Illustrator"
 * Create a request for the response group that includes illustrators
 * Listmania
 * Redo Authors
 * Reformat Subjects, RejectReasons and SuspendReasons

Other

 * Correct _S* and _A* URLs
 * Move non-genre authors to Biblioholics
 * Grab LOCIS, Melvyl, the British Library, etc using subject headings
 * Scan Locus Online's ISBNs
 * Merge duplicate titles
 * Add EDITOR Titles to Magazines that are missing them

Done

 * Books/magazines with ISBNs published prior to 1966 are now auto-suspended
 * Authors with no spaces between the initials, e.g. "H.G. Wells", now have a space added (i.e. "H. G. Wells") at submission creation time
 * Amazon US records are now marked as submitted when their related Amazon UK record is submitted and v.v.
 * If there is more than one pre-existing book length Title (with the same Authors) on file, auto-merge is no longer attempted
 * Books marked as "westerns" are now automatically rejected
 * Books scheduled to appear in 2010 and October-December 2009 are now automatically suspended
 * Books marked as "Abandoned" by Amazon UK are now automatically suspended
 * Books with "Manga" in the title are now automatically suspended
 * Merged the two eligibility checks so 999 and 555 ISBNs are always suspended
 * Fixer now uses ISBN-13 for books published in 2008 and later
 * Made the author field mandatory. It now uses "uncredited" instead of leaving it blank.
 * Amazon-provided formats (e.g. "Large Print") are now added to Notes. "Bargain Price" is ignored.
 * Added a big warning when the book is marked "Import" by Amazon.com
 * Rejected all maps, "*.exe"s, calendars and NTSC
 * Fixer will now use "MP3 Audio" and "CD" as the binding if no other binding information is available
 * Fixed apostrophe/quotes
 * Removed "General" and "General AAS" browse nodes from the mod notes
 * Changed the logic to automatically merge Amazon's accent-less records for "China Miéville" and similar authors
 * Implemented automatic merging with pre-existing titles

Known UK Publishers

 * Aldine
 * Allen & Unwin
 * Allen Lane
 * Allison & Busby
 * Armada
 * Armada Lions
 * Arrow
 * BBC
 * Badger
 * Beccon Publications
 * Big Finish
 * Bodley Head
 * Boxtree
 * Brown Watson
 * Cape
 * Cassell
 * Century Hutchinson
 * Chatto & Windus
 * Collins
 * Corgi
 * Coronet
 * Dobson
 * Eyre & Spottiswoode
 * Eyre Methuen
 * Faber
 * Fontana
 * Four Square
 * Futura
 * George Allen & Unwin
 * Gollancz
 * Grafton
 * Granada
 * Hamlyn
 * Hart-Davis
 * Heinemann
 * Hodder & Stoughton
 * Hodder Headline
 * Howard and Wyndham
 * Hutchinson
 * J. M. Dent
 * John Spencer
 * Jonathan Cape
 * Legend
 * Lions
 * Magnet
 * Mammoth
 * Mandarin
 * Mayflower
 * Mayflower-Dell
 * Methuen
 * Michael Joseph
 * Millennium
 * NEL
 * New English Library
 * Orbit
 * Orion
 * Paladin
 * Pan
 * Panther
 * Panther Granada
 * Peacock
 * Penguin
 * Picador
 * Piccolo
 * Puffin
 * Quartet Books
 * Rupert Hart-Davis
 * Scion
 * Sidgwick & Jackson
 * Sphere
 * Star
 * Tandem
 * Target
 * Telos
 * The Science Fiction Foundation
 * The Women's Press
 * Titan
 * Triad
 * Triad Grafton
 * Triad Granada
 * Triad Panther
 * Unwin Hyman
 * VGSF
 * Venture SF
 * Virago
 * Virgin
 * Vista
 * Voyager
 * W. H. Allen Star
 * W. H. Allen
 * Weidenfeld & Nicolson
 * William Kimber

You have to be careful with some: e.g. "Pan" alone is probably British, but "Pan Macmillan" is global.