User:Fixer
From ISFDB
Fixer is a robot account, which is used to create submissions via the new Web API. It is currently owned by User:Ahasuerus. Fixer 13:08, 30 May 2008 (UTC)
Contents |
How Fixer Works as of 2010
User:Ahasuerus is in the process of improving Fixer, so things are rather fluid, but here is how things currently work:
- Fixer queries Amazon.com and Amazon UK (Amazon CA is under construction) for a particular subject, author and/or time period. There are various (often obscure and/or complex) Amazon-imposed limits on what you can query for and how many items are returned, so it's not as simple as it sounds.
- Amazon sends some data for the books that match the query back to Fixer. Note that the data that Amazon sends back is not always the same as the data displayed on Amazon Web pages, e.g. Fixer doesn't have access to cover artists.
- If the ISBN of the incoming book has been submitted to ISFDB in the past -- or previously suspended/rejected -- then the book is ignored. Otherwise Fixer adds the book's data to the main "queue".
- Once all books matching the query have been returned by Amazon (which takes anywhere from minutes to hours depending on the query), Fixer looks for any ISBNs that were returned by only one of the two Amazon stores and queries the other store. This happens frequently when a book is released first in one country and then the other.
- At this point all data for all eligible books is in the main queue. Typically, each ISBN will have two records, one from Amazon.com and one from Amazon UK.
- Next, Fixer tries to determine which store's data to use. For example, if Amazon.com says that the publisher is "Baen" and Amazon UK says that the publisher is "Unknown", then chances are that Fixer should use the US record to create the ISFDB submission. Ideally, this would be an automated process, but there are many cases when both records look plausible, e.g. the respective prices may be $17.99 and £11.99, and it requires manual intervention. Hopefully, the automated identification process will get better over time as Fixer learns which publishers are "US only" and which ones are "UK only". Any publishers that are active on both side of the Atlantic, e.g. Orbit, will still need to be handled manually.
- Next, Fixer decides whether the ISBN should be automatically suspended. This is currently done for any ISBNs starting with "2" or "3" (France and Germany), any audio, CD and MP3 books, and, lately, for any books published by the better known vanity publishers. We will get to all of them -- eventually -- but for now I didn't want the submission queue to grow unmanageably long.
- Next, a manual process is run. I review all books in the queue and decide whether it should be submitted, suspended or rejected. Unfortunately, there are quite a few cases where the logic is just too complex for Fixer to handle, e.g. TokyoPop publishes both manga and written versions of certain series and there is no easy way of teaching Fixer which one is which. Besides, Amazon's data is often quite dirty, which makes the number of potential permutations too high for a poor overworked robot to handle reliably.
- If I decide to submit the book, Fixer marks the ISBN as "submitted", builds the submission and sends it to ISFDB.
In the past Fixer worked with less human supervision, which made it easier for me to run, but also resulted in more bad submissions and more work for moderators. As Fixer gets better at making some of these decisions, it should become a faster and smoother process both for me and for the approving moderators.
Next steps:
- Amazon CA -- in progress.
- Australian sources -- some 20,000 records have been captured, but I still need to parse them before Fixer can create submissions.
- Library catalogs -- a number of major catalogs have been captured, but the data will require a lot of massaging before it is ready to be submitted. Suffice it to say that there are well over a thousand meaningful fields in the most popular standard for catalog records. Although not all of them are required for our purposes, quite a few need to be included in Moderator Notes since they may facilitate the decision making process at approval time. (And then there are catalogs with non-standard formats or no formatting at all, but that's a whole different headache.)
- Ahasuerus 20:46, 30 March 2010 (UTC)
Publishers processed
- Pyr - done (5)
- Roc - done (50)
- Baen - done (70)
- Ace - done (80)
- Tor - done (450?)
- Del Rey - done (396)
- Tandem - done (but not Tandem-somethingelse)
- Gollancz - Amazon UK only
To Do
Amazon and friends
- Check books published by "Telos" in 2009 and 2010
- Capture Amazon CA
- 2 sources for AU data
- Combine US/UK/CA/AU records and upgrade the logic to submit the resulting composite records intelligently
- Upgrade notes/modnotes
- Add support for "creators", e.g. Role="Editor" and Role="Illustrator"
- Create a request for the response group that includes illustrators
- Listmania
- Redo Authors
- Reformat Subjects, RejectReasons and SuspendReasons
Other
- Correct _S* and _A* URLs
- Move non-genre authors to Biblioholics
- Grab LOCIS, Melvyl, the British Library, etc using subject headings
- Scan Locus Online's ISBNs
- Merge duplicate titles
- Add EDITOR Titles to Magazines that are missing them
Done
- Books/magazines with ISBNs published prior to 1966 are now auto-suspended
- Authors with no spaces between the initials, e.g. "H.G. Wells", now have a space added (i.e. "H. G. Wells") at submission creation time
- Amazon US records are now marked as submitted when their related Amazon UK record is submitted and v.v.
- If there is more than one pre-existing book length Title (with the same Authors) on file, auto-merge is no longer attempted
- Books marked as "westerns" are now automatically rejected
- Books scheduled to appear in 2010 and October-December 2009 are now automatically suspended
- Books marked as "Abandoned" by Amazon UK are now automatically suspended
- Books with "Manga" in the title are now automatically suspended
- Merged the two eligibility checks so 999 and 555 ISBNs are always suspended
- Fixer now uses ISBN-13 for books published in 2008 and later
- Made the author field mandatory. It now uses "uncredited" instead of leaving it blank.
- Amazon-provided formats (e.g. "Large Print") are now added to Notes. "Bargain Price" is ignored.
- Added a big warning when the book is marked "Import" by Amazon.com
- Rejected all maps, "*.exe"s, calendars and NTSC
- Fixer will now use "MP3 Audio" and "CD" as the binding if no other binding information is available
- Fixed apostrophe/quotes
- Removed "General" and "General AAS" browse nodes from the mod notes
- Changed the logic to automatically merge Amazon's accent-less records for "China Miéville" and similar authors
- Implemented automatic merging with pre-existing titles
Known UK Publishers
- Aldine
- Allen & Unwin
- Allen Lane
- Allison & Busby
- Armada
- Armada Lions
- Arrow
- BBC
- Badger
- Beccon Publications
- Big Finish
- Bodley Head
- Boxtree
- Brown Watson
- Cape
- Cassell
- Century Hutchinson
- Chatto & Windus
- Collins
- Corgi
- Coronet
- Dobson
- Eyre & Spottiswoode
- Eyre Methuen
- Faber
- Fontana
- Four Square
- Futura
- George Allen & Unwin
- Gollancz
- Grafton
- Granada
- Hamlyn
- Hart-Davis
- Heinemann
- Hodder & Stoughton
- Hodder Headline
- Howard and Wyndham
- Hutchinson
- J. M. Dent
- John Spencer
- Jonathan Cape
- Legend
- Lions
- Magnet
- Mammoth
- Mandarin
- Mayflower
- Mayflower-Dell
- Methuen
- Michael Joseph
- Millennium
- NEL
- New English Library
- Orbit
- Orion
- Paladin
- Pan
- Panther
- Panther Granada
- Peacock
- Penguin
- Picador
- Piccolo
- Puffin
- Quartet Books
- Rupert Hart-Davis
- Scion
- Sidgwick & Jackson
- Sphere
- Star
- Tandem
- Target
- Telos
- The Science Fiction Foundation
- The Women's Press
- Titan
- Triad
- Triad Grafton
- Triad Granada
- Triad Panther
- Unwin Hyman
- VGSF
- Venture SF
- Virago
- Virgin
- Vista
- Voyager
- W. H. Allen Star
- W. H. Allen
- Weidenfeld & Nicolson
- William Kimber
You have to be careful with some: e.g. "Pan" alone is probably British, but "Pan Macmillan" is global.
Deferred Submissions
| ISBN | Title | Reason/Notes | Mod |
|---|---|---|---|
| 9780957156005 | "Topsy Turvy Tales" by Charlotte Boulay-Goldsmith | Scheduled for April 2012, but little information. Publisher is Humpty Dumpty Films Ltd & has an illustrator; may be picture book. Check again in mid-2012. | JLaTondre |
| 9780061472633 | "Prince Caspian Chapter Book #4" by an unknown author | Scheduled for April 2012, but no author information or cover scan as of February. Check again in mid-2012. | Ahasuerus |
| 9781405263795 | "The Mystery Series Collection: WITH Magic Faraway Tree, Wishing-Chair, O'clock Tales AND Magic Folk v. I" by Enid Blyton | Scheduled for April 2012, but no details as of February. May have been delayed or canceled. Supposed to contain "The Mystery of the Burnt Cottage", "The Mystery of the Disappearing Cat", "The Mystery of the Secret Room" and possibly the four books listed in the full title. Check again in mid-2012. | Ahasuerus |
| 9781908717061 | "Unbelievable" by Jenny Pickup | Scheduled for April 2012, but no details as of February. May have been delayed or canceled. Check again in mid-2012. | Ahasuerus |
| 9780007273782 | "The Inheritance" by Robin Hobb | Scheduled for April 2012, but no details as of February. May have been superseded by ISBN 978-0-00-744634-6. Check again in mid-2012. | Ahasuerus |
| 9780007444298 | "Cloven Hooves" by Megan Lindholm | Scheduled for April 2012, but no details as of February. This was supposed to be the first of multiple reprints of Lindholm's early works, but it may have been pushed back or canceled. Check again in mid-2012. | Ahasuerus |
| 978-0-06-207420-1 | "Wayne of Gotham " by Tracy Hickman | Scheduled for January 2012, then pushed back until May, but no details as of February. Check again in mid-2012. | Ahasuerus |
| 9781849703420 | "Malus Darkblade", a new (?) omnibus edition of Malus Darkblade novels (?) by Ian Irvine | Scheduled for February 2012, but no details or cover scan at Amazon UK or anywhere else. Check again in mid-2012. | Ahasuerus |
| 978-1-84149-829-4 | "Rebellion", part 2 of "Tainted Realm" by Ian Irvine | Scheduled for March 2012, but postponed along with other volumes in the series. Check again in mid-2012. | Ahasuerus |
| 978-1-84149-830-0 | "Justice", part 3 of "Tainted Realm" by Ian Irvine | Scheduled for September 2011, but postponed along with other volumes in the series. Check again in mid-2012. | Ahasuerus |
| 9780957120907 | "Stable Gossip" by Beth Williams (new author) | Scheduled for January 2012, but apparently postponed or canceled. Check again in mid-2012. | MartyD |
| 9781442460539 | "Maddigan's Fantasia " by Margaret Mahy | Scheduled for January 2012, but apparently postponed or canceled. Check again in mid-2012. | Ahasuerus |
| 9780062105547 | "The Hunters: Moonsong" by L. J. Smith | Scheduled for March 2012, but apparently postponed or canceled. Check again in mid-2012. | Ahasuerus |
| 9781937007157 | "The Last President" by John Barnes | Scheduled for March 2012, but postponed indefinitely. Check again in mid-2012. | Ahasuerus |
| 9780575095496 | "Untitled Neill 3 of 3: A Novel of the Dark Elite" by Chloe Neill | Scheduled for February 2012, but apparently delayed or canceled. Check again in mid-2012. | Ahasuerus |
| 9780575097261 | "Untitled Griffith 2 of 3: Vampire Empire Book 2" by Clay Griffith and Susan Griffith | Scheduled for February 2012, but apparently delayed or canceled. Check again in mid-2012. | Ahasuerus |
| 9780575094338 | Dark Mist Rising by Anna Kendall | Scheduled for February 2012, but seems to have been canceled. Check again in 2012. | Ahasuerus |
| 978-0451234841 | Velvet Moon by Sophie Renwick | Scheduled for December 2011, but delayed until early 2012 (?) Check again in 2012. | Ahasuerus |
| 978-1780690414 | Old Alcopop | Scheduled for 2012-01-01 but no cover or useful information. Check again in early 2012. | MartyD |
| 978-1408165768 | Petrify (Wired Up) | Under ASIN: 1408165767 no useful data are listed. Another page has useful but still sparse info, but gives the ISBN as 978-1408152690. From that page this looks like horror, and "Wired Up" seems to be a series. Check again in march 2012. | DES |
| 9780957018402 | History of the Future: Part 1: Elixir and Necropolis by Alastair Hunter | Scheduled for 2011-10-00, but hasn't appeared as of 2011-11-12. No cover scan and no Web presence. Check again in early 2012. | Ahasuerus |
| 978-0-9570019-1-6 | Talisman of El by Alecia Stone | The paperback version will be supposedly published in 2012-11. Check again around 2012-09. | Ahasuerus |
| 9781593934835 | Karloff as The Invisible Man by Philip J. Riley | No cover scan at Amazon.com and the publisher's online catalog hasn't been updated in a while as of late 2011. Check again in early 2012. | Ahasuerus |
| 0749040203 | Last Breath by Rachel Caine | No publication data at Amazon.com and no record at Amazon.col.uk for this digital reprint, which was supposed to be published on November 1, 2011; check in early 2012 | Ahasuerus |
| 978-0-85707-152-1 | Devoted by Hilary Duff | Was supposed to be a September 2011 released, but apparently has been pushed back until mid-2012. | Ahasuerus |
| 9780312943707 | Reckoning (Dark Storm) by Kris Greene | Wrong cover scan at Amazon - may have been delayed or canceled, check in November 2011. | Ahasuerus |
| 9781444903515 | The Highgate Vampire by Asa Bailey | Delayed until 2012-06. | Ahasuerus |
| 9780953745906 | Salome's Daughters: Book 1 of House of Bast | Delayed/canceled self-published book . Check again later in 2011. | Ahasuerus |
| 9780007315925 | Palace of the Damned | Appears to be a cancelled version of 978-0-00-731591-8. Check again later in 2011. | Ahasuerus |
| 978-0-06-073760-3 | Z | Amazon.com lists it as a 2020 (sic) title, but Amazon UK thinks that it will come out in September 2011. Check again later in 2011. | Ahasuerus |
| 9781427808264 | Chibi Vampire: The Novel Volume 9 | Appears to have been postponed. Check again later in 2011. | Ahasuerus |
| 9781609491406 | Ghosts of the Triad | Not enough data (no publisher). Check again later in 2011. | MartyD |
| 9781907151644 | Hollow Earth | Not enough data (no author). Check again closer to 2012-01-01 release date. | MartyD |
| 9781445441627 | Magical Christmas Tales | Not enough data. Might be "out" but can't tell. Check again later in 2011. | MartyD |
| 9781841499345 | "Untitled Terry DeHart" | Probably canceled, but not enough data to be sure. Check again in late 2011. | Ahasuerus |
| 9781908268136 | Kira by Marcus Woolcott | Not enough data. Check again in late 2011. | Ahasuerus |
| 9780340998168 | Vampire Thriller | Appears to be an early placeholder for Christopher Farnsworth's The President's Vampire. Check again in late 2011 to see if this ISBN was canceled. | Ahasuerus |
| 9780316129091 | Plants vs. Zombies | Not enough info; inappropriate price. Try again in June. | MartyD |
| 9780984235162 | Mountain Spirits | Not enough info. Try again in June. | Kraang |
| 9780007345397 | Azinger Untitled 2 | Not enough info. Try again in June. | BLongley |
| 9781846056994 | Star Wars Stand Alone 7 series no 2 | Not enough info. Try again in June. | BLongley |
| 9780571272082 | Dark Woods | Not enough info. Try again in a few months. | MartyD |
| 9781936558148 | The God's Wife | Possible vaporware; not enough info; no references elsewhere. Try again in a few months. | MartyD |
| 9781846470806 | Milicent's Book | Looks like non-SF, but insufficient info available to tell. Try again in a few months. | MartyD |
| 9781587671920 | On Dangerous Ground:: Stories of Western Noir | Possible vaporware; try again in a few months. Out of stock on Amazon. 2011-05-01 on B&N. No mention on www.cemeterydance.com. | MartyD |
| 9780982957943 | The Last Zombie on Earth by John St. John | Possible vaporware. 2010-11-15 on Amazon, but out of stock. No references to title or ISBN anywhere else. Can't find "Horror Zone" publisher. | MartyD |
| 9780956739209 | Children of Hagar by N. J. Evans | Delayed/canceled publication? Check in mid-2011 | Ahasuerus |
| 0062017365 | Kill the Dead (Sandman Slim) by Richard Kadrey | Pushed back until July 2011 | Ahasuerus |
| 9780553822007 | Armouron 5: Orphan Factory by "O. B. McGann" (Oisin McGann?) | Scheduled for publication in 2011-04 -- not enough data as of 2010-12 | Ahasuerus |
| 9780553822014 | Armouron 7: Dead-End Junction by "O. B. McGann" (Oisin McGann?) | Scheduled for publication in 2011-09 -- not enough data as of 2010-12 | Ahasuerus |
| 9780553821178 | Armouron: Lying Eyes by "O. B. McGann" (Oisin McGann?) | Scheduled for publication in 2011-04 -- not enough data as of 2010-12 | Ahasuerus |
| 978-1906817619 | The Wicker Tree | Originally scheduled for mid-2010, pushed back until 2011 | Ahasuerus |
| 978-0982433126 | Charlie 9-lives | Apparently not SF; unreleased so hard to be sure | DES |
| 9781599553986 | Hashbrown Winters and the Phantom of Pordunce | Unable to tell if this is SF or not, not yet released | DES |
| 9781907475689 | Cold Blood | Unable to tell if this is SF or not, not yet released | DES |
| 9781407536675 | Disney - Tron Book of the Film (Tron Legacy) | Can't tell if this is a novelization (IN), or a book about the film (OUT) or what. Defer until released and reviewed. | DES |
| 9780385752077 & 9780385752046 | Lucy and the Green Man | Can't tell if this is SF or historical, author has written both. defer until released and perhaps reviewed. | DES |
| 9781905716593 | Charles Dickens Greatest Novels: v. 2 | Possible vaporware, or rescheduled; no cover and out of stock on Amazon; no mention on Collector's Library. | MartyD |
| 9780545213608 | Adventures in Hyperspace #3 | Possible vaporware, or rescheduled; no cover and currently unavailable on Amazon; no mention on Scholastic's Hyperspace page. | MartyD |
| 9780199587001 | Great Gothic Horror 3 Vol Set | Possible vaporware; listed on Amazon and other places, but no cover, no content description. ISBN search on both Oxford USA and Oxford UK and manual search of all titles on Oxford World's Classics list turn up no results. | MartyD |
| 9780749942557 | Untitled Dream-hunter | Insufficient info (e.g., lack of actual title, no cover); ISBN search on Little, Brown site turns up no results. | MartyD |
| 9780340855164 | Deptford Mouselets 4 | possible vaporware; Amazon listing incomplete (no image, no proper title, no page count)and no mention at all on the Hodder Children's Books site, including "Sneak Previews", their forthcoming books list. | DES |
| 9780553822687 | Merlin: 3 Book Bind-Up | Insufficient information: No author, apparently no excact title. May be vaporware, no one appears to list this for even prospective sale. | DES |
| 9780857392053 | Mini Sagas Creative Capers Fantastic Fiction | Can't tell if this is spec-fic or not. Defer until released and more info is available. | DES |
| 9781926903002 | Infinity | Can't be sure this is spec fic; may possibly be vaporware; most of all, lists an "editor" as co-creator, can't tell whether the editor should be listed as a co-author or not. Defer until published and more info available, perhaps a "look inside" | DES |
| 9781900355131 | A Knight at Sea | Apparently not SF, but just possibly alt-Hist. Defer until published and more info is available. | DES |
| 9780955748332 | Hypknowlogy | Clearly announced, but not enough detail (no cover, binding, page count). Try again for a more complete entry a little closer to publication date. | MartyD |
| 9781907690129 | ....and Man Cre Ated God! | This may well be spec-fic, but I can't really tell, and i don't trust Amazon's subject lists. Defer until after publication when more info may be available. | DES |
| 9781445412245 | Ben 10 Junior Novel War of the Worlds | Probably an edition, perhaps abridged, of the H.G. Wells classic, but I can't be sure. Defer until after publication. | DES |
| 9781907499258 | The Fire Ruby: Pt. 1 | Probably not spec-fic, but hard to be sure. Defer until after publication and more info available. If approved, find pt 2 which was approved then deleted. | DES |
| 9781934876114 | Dragon Blade Warrior Vol 1 (Dragon Blader) | Not sufficient info to create proper record. May not even be final title. may be vaporware. Defer until after actual publication. (As Of 3 Dec 2011, no listing at all on Amazon US or on ISBNdb.com) | DES |
| 9780007379163 | Devil You Know | Appears likely to be vaporware. No mention on author's "In the Works" page, no mention on publisher's site. A more prominently mentioned Symphathy for the Devil in the same series isn't even out yet (announced for Dec 2009, now Amazon has it as coming in Apr 2011). Deferred. | MartyD |
| 9781606868683 | More Scary Stories to Tell in the Dark | Appears likely to be vaporware. "Perfection Learning" is not a publisher. Their site lists this ISBN as HarperCollins 1986, but searching by ISBN-10 and ISBN-13 basically finds nothing. Retry again in a month or two. | MartyD |
| 9781844185757 | Enchanted Tales | Too little data. Check again after 2011-10-31. As of 2011-12-03 "Out of Print" | MHHutchins |
| 9781908075239 | The Key to 203 | Too little data. Check again after 2011-10-15. | MHHutchins |
| 9781446417126 | Being Human Novel 1 (ebook) | No author credit. Too little information. Check again after 2012-01-26. As of 2011-12-03 "Out of Print" | MHHutchins |
| 9781446417133 | Being Human Novel 2 (ebook) | No author credit. Too little information. Check again after 2012-01-26. | MHHutchins |
| 9780062014504 | Unti Christmas Novella by Jeaniene Frost | Sound suspiciously like a UK listing for this US publication. Will check back later on whether the ISBN pops up on any other dealer website. | MHHutchins |
| 9781223011592 | Shadow's Witness by Paul S. Kemp | No publisher credit. ISBN can't be used to determine the publisher. | MHHutchins |
| 9781780690506 | The Back of Beyond by Peter Trusty | Too little information. | JLaTondre |
