ISFDB:Community Portal

From ISFDB

Jump to: navigation, search


ISFDB Noticeboards
Before posting to this page, consider whether one of the other noticeboards might suit your needs better.
Help desk
Questions about doing a specific task, or how to correct information when the solution is not immediately obvious.
• New post • Archives
Verification requests
Help with bibliographic, image credit, and other questions which require a physical check of the work in question.
• New post • Archives
Rules and standards
Discussions about the rules and standards, as well as questions about interpretation and application of those rules.
• New post • Rules changelog • Archives
Community Portal
General discussion about anything not covered by the more specialized noticeboards to the left.
• New post • Archives
Moderator noticeboard
Get the attention of moderators regarding submission questions.
 
• New post • Archives • Cancel submission



Archive Quick Links
Archives of old discussions from the Community Portal.


1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · 23 · 24 · 25 · 26 · 27 · 28 · 29 · 30 · 31 · 32 · 33 · 34 · 35 · 36 · 37 · 38 · 39 · 40 · 41 · 42



Contents

Roadmap 2017

As of late, I have been tweaking the software based on user feedback and in support of various cleanup projects. It was a good match while I was recovering from the most recent unpleasantness over the last couple of months.

Going forward I hope to get back into more heavy-duty development work. With that in mind I have compiled a list of software projects that are currently close to the top of my list of priorities for 2017. I'd like to post them and solicit feedback re: their desirability and relative priority. If a description seems too vague or incomplete, please don't hesitate to ask for a clarification. Detailed suggestions/feedback re: individual FRs may need to be spun off as separate Community Portal sections. Additional requests welcome!

Continuation and further development of currently active projects

  • Language cleanup:
    • Assign languages to all titles
    • Assign languages to all authors. Should we use the manual approach that we used with titles? Or can we automate some parts of the process based on the language of each author’s titles?
      I think that for the ones remaining (with 3 titles or less), we can just assign automatic English if all the titles they have are English (after all titles have languages). We will miss a few that are not English but we will also miss a lot of them if we go manually - they will require a lot of research at this point. If a non-English title ever appears, we can always change it. That won't be much different to what will happen now if someone adds a magazine in English and do not change the author language manually after the addition... Annie 20:38, 19 February 2017 (UTC)
      Or if someone is too worried about doing it, going the manual way will always work of course. Annie 05:38, 21 February 2017 (UTC)
      I guess the "low-hanging fruit" would be identifying language-challenged pseudonyms of language-enabled authors and auto-assigning the parent author's language to the pseudonym. Ahasuerus 16:07, 21 February 2017 (UTC)
    • Yes, I think we even talked somewhere about that. If those can be done, this will eliminate a pretty good chunk of what is remaining... Annie 16:25, 21 February 2017 (UTC)
  • Wiki cleanup
    • Move series/magazine-, publisher- and publication-specific etc Wiki pages to the database
    • Move Bio and Bibliographic data to the database
Globally OK.Hauck 20:00, 19 February 2017 (UTC)
Along with this (though it might not qualify as part of your currently active projects), I would like to add, it would be very good to try and untangle the main DB from the wiki DB (particularly with respect to login credentials and sessions, etc.; I have a few ideas on this). This would facilitate upgrading our MediaWiki software which we are sorely behind on (including many security/bug fixes as well as features). Uzume 20:41, 19 February 2017 (UTC)
Unfortunately, my knowledge of the MediaWiki software is very limited. Last we talked about development, Al was going to look into this issue, but he has been unavailable for a long time. I could try and get myself up to speed, but, unfortunately, learning new things gets harder as you get older. Ahasuerus 16:18, 20 February 2017 (UTC)
It is already untangled in terms of sessions - I get asked to login in wiki while my session is fine on the site and vice versa. So it is just the users that are the same. Annie 05:38, 21 February 2017 (UTC)

New projects

  • Create a separate queue for automated submissions, which will enable non-moderators to work on them (and let me concentrate on development). Rather time-consuming to implement. Discission moved here.
  • Internationalization:
    • Support multiple prices per publication.
    • Support multiple publishers per publication. May require more thought re: imprints.
      I wonder if that cannot be solved in a different way - allow books published from a joint venture to show on 3 publisher pages - a combined one and the two separate publishers ones. Or if we go for multiple publisher, we should have a way to see which books are from both publishers at the same time without the need to compare lists manually Annie 20:52, 19 February 2017 (UTC)
      An interesting point. I have never considered it, but the proposed behavior seems desirable. We'll have to think of the best way to implement it. Ahasuerus 02:18, 20 February 2017 (UTC)
      And here comes a live example that I and KarenHunt had been investigating: Братья по оружию. It is a joint venture between the Moscow based Астрель and the Minsk based Харвест with separate ISBNs for both (9785271439186 for Астрель and 9789851815919 for Харвест) and usually all copies of the books carry both publishers and both ISBNs (I've had a few similar ones). If you create a fictitious "Астрель / Харвест" (incorrect it is not an imprint) or "Астрель & Харвест", it will remove the book from the two individual publishers' pages. Adding two separate copies (one per ISBN) is not exactly correct either as it is the same book (ha - this is one of the other projects below - multiple ISBNs). The cleanest seem to be to go for one of them and note the other in the notes. None of those scenarios are very good though. Annie 01:09, 25 February 2017 (UTC)
      And just to illustrate why it is a bad idea to use the double name, this wonderful example of Russian thought has 4 valid ISBNs on the same publication - it is again a joint venture between Харвест and this time another of the publishing group АСТ's imprints/daughter companies - this time Хранитель instead of Астрель. But just to make it funnier, it also got ISBN for the parent as well and an ISBN for the pure Moscow "АСТ Москва" which was consolidating their list. So we will need yet another joint version for this one. As you can imagine, there are more of those. :) Annie 01:25, 25 February 2017 (UTC)
      We had a discussion of Russian publishers a while back, which is what pushed this FR up the list of priorities. There are other publishers who use multiple ISBNs, but Russian language publishers account for the majority of our "problem cases". Ahasuerus 01:34, 25 February 2017 (UTC)
      Well - I saw an example, decided to post it so it is in the thread. Sorry if it was repeating already known data - I do not remember stumbling on this discussion when I was reading archives. Annie 01:52, 25 February 2017 (UTC)
      Not a problem at all! I was just sharing additional background information. Part of my job description :-) Ahasuerus 02:03, 25 February 2017 (UTC)
    • ISBNs (mostly to support internationalization, but will also help in other cases):
      • Support multiple ISBNs per publication
      • Move catalog IDs to a separate field, which will help with pubs that have both an ISBN and a catalog ID, e.g. book club publications.
      • Add two checkboxes next to each ISBN: “derived” and “corrected”. The latter will let us capture two versions of invalid ISBNs, the stated one and the corrected one. Discussion moved here.
    • Add support for translators (as well as other roles like single-author collection editors?) Discussion moved here.
    • Move “Add Authors/Transliterated Titles/etc” buttons to the right to free up screen real estate. This will become more important with the addition of new multiply occurring fields (see immediately above.)
    • Change “Family Name” to “Directory Entry”. If we allow multiple values to support different alphabets/scripts, how will we sort by this field? Discussion moved here.
    • Separate translations from VTs on the Title page. Allow users to limit the list of publications to the ones associated with the canonical title.
      Won't that also allow us to group same language translations? If not, then can we have that as well? I would love to be able to see that there are 30 different language translations without seeing that there are 10 French ones until I really want this information :) Annie 20:52, 19 February 2017 (UTC)
      I don't recall this functionality mentioned in the past. I would suggest creating a separate section to explain what you would like to see and to gauge other editors' reaction. Ahasuerus 16:23, 20 February 2017 (UTC)
      I'd like to see this one. -- JLaTondre (talk) 01:22, 20 February 2017 (UTC)
  • Re-do verifications. This will let us have more than 5 primary verifications as well as multiple transient verifications per publication.
    I'd also like to see this one. -- JLaTondre (talk) 01:22, 20 February 2017 (UTC)
  • Create a history of changes to primary-verified publications by storing a snapshot of the way each verified pub looked like right before it was changed.
  • Support for third party IDs (OCLC, BLIC, LCCN, BNF, ASIN, Goodreads, etc). This will enable Fixer to submit ebooks without ISBNs, an increasingly common scenario. Discussion moved here.
  • Make the Quick Tags list user-definable.
  • Add support for additional title types like PLAY and FICTITIOUS ESSAY (requires discussion to come up with additional values.) Discussion moved here.
  • Cleanup reports:
    • Make additional cleanup reports available to non-moderators
    • Review old/incomplete FRs for cleanup reports, add missing title types
  • Add a “non-genre” field to author records; create a cleanup report to find non-genre titles by non-genre authors. Discussion moved here.
  • Add a disambiguator field to title records (requires discussion)
    Don't understand. Hauck 20:08, 19 February 2017 (UTC)
    Is that for the ability to add the reason for the variant (change in name, change in author, translation, abridgement)? If so, I'd love this one. If not - What are talking about? Annie 20:52, 19 February 2017 (UTC)
    The problem that we are trying to address here is that some authors have identically named titles. For example, H. P. Lovecraft's Summary page shows three "The Lurking Fear and Other Stories" collections published in 1947, 1964 and 1971. They are all different, but you can't tell until you drill down to the title level and check each title's Note field. Similarly, there are 2 version of the story "The Rats in the Walls" (1924 and 1956), 4 "[To Albert A. Sandusky]" poems and 9 (sic!) poems whose title is currently entered as "[To ?]". Analogously, Clark Ashton Smith wrote 4 different "Ennui" poems, 2 different "A Sunset" poems, 3 "sonnet" poems, etc. A "disambiguator" field would be used to capture a brief summary of what's unique about each title, e.g. "1964 version", or perhaps the first line if it's a poem. It would be displayed on the author's Summary page. In addition, we will need to decide whether to use this field to capture other information like "abridged", "revised" or "greatly expanded". There were different suggestions the last time we talked about it. Ahasuerus 16:53, 20 February 2017 (UTC)

Ahasuerus 19:50, 19 February 2017 (UTC)

Additional Proposed Projects

  • Add a "printing rank" field to order multi-reprinted titles without pub dates.Hauck 19:57, 19 February 2017 (UTC)
    Yes, please! And not only for dateless - even if we know the dates, a field for edition and printing will be awesome (so if I have 2nd edition, 3rd printing, I can look for 2.3 or something like that and find it without looking through 200 publications). Annie 20:52, 19 February 2017 (UTC)
    There is a Feature request to "Add a 'printing' field to publication records". It would be relatively easy to implement, but we need to decide what kind of data we want to capture. Unfortunately, "editions" and even "printings" do not always follow the standard numbering scheme, so "2.3" won't work. For example, this 1989 edition of Heinlein's "Have Space Suit—Will Travel" says "First Ballantine Books Edition: December 1977 Thirteenth Printing: October 1989". On the other hand, this 2005 edition simply says "First Pocket Books trade paperback edition [no printing number]". This is very common, so you can't expect "editions" to be sequentially numbered. We could limit this field to numeric printing numbers, but there can be any number of "first printings" from different publishers. It doesn't mean that we won't want to implement this field, but we need to understand its limitations. Ahasuerus 04:07, 20 February 2017 (UTC)
    The 2.3 was mostly an oversimplification for the multiple printing scenarios - not necessary requesting it to be numbers only. But we do need some guidelines for formatting in that field - or we will end up with unsearchable field. It will still be better than the current way to find where we are. Even if it is just printing in the field - the edition is kinda clear from the ISBN unless if it gets reused. Annie 04:13, 20 February 2017 (UTC)
    What!? I cannot put Chinese characters in the printing field? But I know of some publishers that mark their printing this way (OK, I am facetiously joking but it is true). Uzume 04:58, 20 February 2017 (UTC)
  • Add a new title type for excerpts, and one for plays/scripts. Neither of those fit nicely in SHORTFICTION. --Vasha 21:02, 19 February 2017 (UTC)
  • Another useful feature, mainly for links from external site to the ISFDB: stable IDs. Currently, if a title is merged into another its ID ceases to exist in the database. As a result, links from external sites to such a title record will become broken links. The software should instead keep track of merged IDs and redirect to the new ID (there was a discussion about this some time ago but I just couldn't find it). Jens Hitspacebar 21:10, 19 February 2017 (UTC)
  • [Discussion of third party IDs moved here.]
    1. The next item I would like to see is revamping verifications. I am not sure how valuable it is but it has plagued us for a while now. Uzume 21:36, 19 February 2017 (UTC)
    2. Finally, author and publisher directory updates seems like a good item. This could bed sooner but probably will be easier after more internationalization/localization is completed so doing it a bit later might prove less problematic. Uzume 21:36, 19 February 2017 (UTC)
  • There had been some discussion about reviews of magazines here. I think the discussion petered out rather than concluded, but did get into alternate ways of dealing with magazine series. I'm not pushing for it, just reminding. Doug H 21:59, 19 February 2017 (UTC)

Editor review of submissions: "automated" submissions sidebar

Create a separate queue for automated submissions, which will enable non-moderators to work on them (and let me concentrate on development). Rather time-consuming to implement. Ahasuerus 00:10, 20 February 2017 (UTC)

Yes, please :) I know that it is a time consuming change and so on but it should help in the long run... Annie 20:52, 19 February 2017 (UTC)
I am for editor review of submissions (by Fixer, etc.), however I question the need for a separate queue. Uzume 23:02, 19 February 2017 (UTC)

(unindent)I have some reservations about "automated submissions" (but that maybe based on my lack of understanding) referred to above in #Roadmap 2017. Why do we need another submission type like this? It seems to me this FR is just to allow editor reviews of submissions (like a voting system to help moderators close pre-reviewed submissions). Is there some reason some submissions should not allow editor review? Uzume 21:45, 19 February 2017 (UTC)

Because at the moment Ahasuerus is doing all of them. This is the process by which all newly published books are added to the system - so we do not wait for people that have them to add them. The idea is to allow the queues that the automated robot finds to be worked on by other people as well - which includes verifying that it is indeed eligible, cleaning the record based on our standard and so on. It is not about approval of entries from people, it is about adding the entries in the DB so the moderators can approve them. Annie 21:50, 19 February 2017 (UTC)
I do not see a difference with this. It is true bot generated submissions maybe of lower quality but then so are submissions from new users. I do not think we need a new field specifying the origin of the submission. I like the editor review of submissions concept (perhaps with comments and queued results to be approved by moderators). In the general sense, consistent high volume and high quality editor reviews of submissions could be a very good way of selecting prospective new moderators. Uzume 21:55, 19 February 2017 (UTC)
We do not have real bot submissions now - Fixer is manually operated by Ahasuerus, title by title, every month. That idea is to allow these queues to be processed by other people as well. Annie 22:03, 19 February 2017 (UTC)
I understand he is doing significant manual work on his side of bot before making submissions and to lighten his load, we want to be able to have his bot make lower quality submissions with better review from our community (instead of just him) and I agree with and would not argue on that point. However, it seems to me a generalized editor submission review process is a better solution than a one targeted just at this problem (which adds unneeded extra levels and fields for such). Uzume 22:10, 19 February 2017 (UTC)
And how is that different from making more people moderators if there are people that can do reviews like that? :) Plus - the process of accepting edits is already slow, do you really want to add more layers? A new editor will get discouraged fast. Plus - that won't solve the issue with Fixer and the sumissions - no matter who reviews, they need to get into the system one way or another. If you want all that it finds to be just added and someone reject it if it is not genre and/or not eligible, that would add a lot more work on the moderators... A Fixer, non-massaged entry is missing series information, have (or can have) a weird spelling on author and publisher, title (Amazon adds a lot of junk there) and it may need to be merged somewhere. No matter how many layers of whatever we have, if those just get dumped into the regular queues, the real updates will get delayed with days until the pure deluge of those is dealt with. Annie 22:18, 19 February 2017 (UTC)
I never suggested making editor review mandatory—just generalized. I do not understand why the normal queues would cause delay. Submissions are not required to be moderated in any particular order (I understand there are order issue with regard to moderation however). Uzume 22:36, 19 February 2017 (UTC)
No but delaying a new contributor edit while we are dealing with internal ones is a bad idea. And if you have 12000 Fixer submissions in the regular queue, how will a moderator find a new one. And even if they can, working on those ratty ones will be a longer and more involved process than on user-created ones. Thus the separate queues and the ability to do something before they hit the moderators. :) Annie 23:13, 19 February 2017 (UTC)
That sounds to me more like better management of the current queue is needed not another queue. Make the submission queue searchable and a moderator can for example just say "I do not currently want to see submissions by Fixer" (or say any user with a bot flag; this list is not very long Special:Listusers/bot). Uzume 23:40, 19 February 2017 (UTC)
The main difference is allowing editing before submitting - allowing the entry to be cleaned up before it goes into the proper queue for approval. This is not something moderators can do now - they can only accept and then edit after that. If you are saying that moderators should be able to directly edit ANY submission before acceptance, then yes, better management is all that is needed. But I am not sure we really need that. Annie 23:45, 19 February 2017 (UTC)
OK, so you are saying you want a separate queue and allow reviewing editors to edit/transmute the submission as part of the review process? That would be very powerful but also extremely difficult to code (since submissions are actually only semi-structured XML documents). Uzume 00:39, 20 February 2017 (UTC)
Thus the comment above that it will be very complicated :) If you read the FR, this is what it describes. Annie 00:40, 20 February 2017 (UTC)
Maybe Annie means a tiered submission process where editors can review automated submissions converting them to normal submissions or some such. But then in my mind "automated" just means "I want editor review first". Uzume 22:02, 19 February 2017 (UTC)
Not really. The current automated process is; Fixer finds the titles (wonderful), Ahasuerus submits them to the DB (from his name or from Fixer's name). Anything with low priority just stays in the queues until someone gets to them. Or forever. It is not about magazines, it is really about what the robot finds and needs to be processed. And you do not want to add to the moderator load all the "cleanup" that is needed - there is a LOT of work to be done on those submissions. And just dumping them into the DB without cleanup, even with a flag will dilute and make the DB useless fast.Annie 22:09, 19 February 2017 (UTC)
I am not arguing against editor review for Fixer submissions. I am arguing this is more elegantly solved with a generalized process of editor review not specific to Fixer submissions. Uzume 22:13, 19 February 2017 (UTC)
Baby steps. Plus Fixer (and any other robots that can find more entries). It is not Fixer specific really - it is the ability to handle those kinds of submissions in a slightly different way. And I still argue that if a person does the submission, you do not need it to be delayed further. :) Annie 22:20, 19 February 2017 (UTC)
So you are saying the "automated" flag adds another tier delaying such submissions. I do not think that is necessary (and if you read the FR a moderator can directly moderated such without editor review anyway). The point is we do not need the flag can just implement generalized editor review of submissions. Uzume 22:35, 19 February 2017 (UTC)
If someone new does not understand the page, they may click it, sending the submission in a long(er) queue. What we need is a double queue - one that only moderators can deal with (the ones that had been seen by a human and just need the second set of eyes) and one for any "ratty" submission where more people can help with. Which is what this FR is for - allow a new process for submissions. Annie 23:09, 19 February 2017 (UTC)
I am all for editor review of submissions to help with such "ratty" submissions. I do not think we need another queue (chosen at submission time), just better management of the current queue should suffice.

OK, after some discussion...if we are talking about a totally different process where someone or something provides suggested changes (I am intentionally avoiding the word "submission") which are reviewed and potentially generate submissions (a sort of reviewed and potentially modified transmutation from the original suggestion), that is a totally different beast and I am not at all sure it makes sense to leverage the current submission system for this sort of thing. I believe Fixer and other bot like "animals", normally do not make arbitrary submissions but only very specific types (mostly NewPub; someone tell me what other types it can make). As such, it might make much more sense to have separate NewPub (suggestion? submission?) queue. A reviewer could then load such NewPub items into the NewPub editor, potentially make changes, and make the submission (or mark the suggestion rejected). This would greatly simply such an implementation (which would still not be trivial). It might also make sense for bots to mark items in this NewPub queue as completed (or actually delete them) when it noticed the items appear in the main database that match the NewPub suggestion it previously made. For this purpose, there should probably be a way for bots to query this queue (with search requests like "give me all outstanding NewPub items I provided but I have not marked as completed/deleted"). Frankly, I see this as more of a remote bot NewPub queue than an automated submission queue. Uzume 01:24, 20 February 2017 (UTC)

AddPub as well when it manages to recognize where a new publication belong. Fixer founds a few thousand new ISBNs every month. They need to be added to our DB - but they do need a lot of cleanup. What is your proposal - how those say 6 0000 publications get added without the new process and without burning out all the moderators? If Fixer can just dump them in a new queue where someone can go, edit and then submit to the moderators, this is exactly what the FR is proposing - call it automated submission queue, call it "list of possible NewPub/AddPub", call it whatever you want :) So do you just disagree with the name? What you describe here is exactly what the FR is proposing... It is not about changing the current process, it is about adding a new process BEFORE the current one that will allow these ratty ones to be streamlined for the process. Annie 01:39, 20 February 2017 (UTC)
That is not how the FR is written "Change the submission table to support a new submission state, “A”utomated" implies modification of the current submission queue. I do not think that is appropriate if what is really needed is a remote bot queue with limited input types. Uzume 02:11, 20 February 2017 (UTC)
Look at point 10 in the FR - that is what ties it back into the standard process. Annie 02:13, 20 February 2017 (UTC)
I see that now, however, I still think that unduly complicates the submission system with little value. I believe we would be better off with a new different specialized queue than trying to shoehorn this functionality into the current submission queue. Uzume 02:20, 20 February 2017 (UTC)
The linked FR would effectively create a new specialized queue. Editors would then use "automated"/robotic entries in that queue to create regular submissions. Implementing various supporting mechanisms -- like putting the automated submission on hold as described in the FR -- would be somewhat challenging, but I think it should be doable. Ahasuerus 04:58, 20 February 2017 (UTC)
I imagine it would be challenging both on the API server side of things as well on the bot client side of that (what does/should Fixer do with such a remote queue item marked on hold?). Uzume 05:23, 20 February 2017 (UTC)
When Fixer sends a submission via the Web API, he marks the ISBN as "submitted" in his internal database. From that point on he ignores the ISBN.
When Fixer gets the next weekly backup, he updates his internal database with the data from the backup. At that point the status of moderator-approved ISBNs changes from "submitted" to "already in the ISFDB". Moderator-rejected ISBNs remain "submitted" forever. It doesn't really make a difference because Fixer ignores "already on file" ISBNs just like he ignores "submitted" ISBNs. Adding a new queue on the server side shouldn't change this logic on the client side. Ahasuerus 15:16, 20 February 2017 (UTC)

(unindent) If there's a separate queue, that queue will probably back up (esp. once the novelty wears off). If it does back up, we'll end up in a situation where more of its entries turn out to be duplicated by live-editor submissions that were created without consulting that queue. It might be better to get more moderators and let Fixer submissions flow through without any sort of pre-qualification/filtering that's being done now. We are not particularly aggressive about recruiting moderators. --MartyD 12:03, 20 February 2017 (UTC)

Unfortunately, the main problem is not that we don't have enough moderators. The main problem is that processing robotic submissions requires a significant amount of research and TLC. Some moderators have the time and the energy to do the necessary research. Others don't and simply click the "Approve" button if the submission looks OK. This is the main reason why I only submit on Fixer's behalf if:
  • I am 99% sure that the submission (usually an AddPub) is truly OK "as is", or
  • it's a relatively low priority ISBN and it's better to have it on file in a potentially suboptimal state than not to have it at all
Over the years, I have tried a number of strategies to improve the situation, all of them unsuccessful.
The "automated submissions queue" is the latest stratagem in a long list. The idea is that having human editors do the requisite research up front would result in higher quality submissions to be reviewed by moderators. Admittedly, if a moderator wants to pull up the new queue and simply approve everything in it, it won't stop it.
If there is a better solution to this problem, I'd love to hear it. The current situation is barely sustainable as it is and will continue to deteriorate as my productivity declines over time. (Not to mention the increased number of SF books being published.) Ahasuerus 19:15, 20 February 2017 (UTC)
That makes more sense and I prefer that idea. This is why I am sort of against the idea of another queue (particularly if it is shoehorned in the the existing one as the first part of the current FR describes) despite its possible value in being able to modify the entries at review time (it seems like much work where this is the only key value). That said, it might make sense to add generalized editor review for the main submission queue (it could be limited in some way if there are security concerns). Such reviews could help moderators more quickly decide when closing submissions. As Marty points out such things may turn into a novelty (but then same could be said of moderators; look at how many inactive or barely active ones we have at the moment), but it might ebb and flow with new or return editors, etc. Aside from the possible value of possible valuable reviews to speed in moderation, it seems like a better tool to use when hunting for moderator candidates (so though this too might be complex and end up perhaps with smaller value this part seems to me the most valuable part). Uzume 18:24, 20 February 2017 (UTC)
If we go this way, we really should have a prioritization of some type available so that live editor updates do not get backed up when Fixer dumps its submissions. It is very annoying to wait for a moderator to approve something so you can send the next update (especially when you are new) :) Annie 05:32, 21 February 2017 (UTC)

Adding support for translators and possibly other roles

Add support for translators (as well as other roles like single-author collection editors?) Ahasuerus 23:45, 19 February 2017 (UTC)

Ok for the former, no to the latter, why not editor of novels? Hauck 20:08, 19 February 2017 (UTC)
A collection editor usually chose the stories, a novel editor does not chose content in the same way. I think we should allow such editors be somehow recognized. Annie 20:52, 19 February 2017 (UTC)
I am with Hervé here. I would like to added more contribution types supported when we have no way to capture such now but I would be wary of taking what we already have and dividing it down (into different types of authors/editors or different types of artists, etc.). I definitely want to see translators (since our current rules have no fields to capture this short of ad hoc notes). I am curious why we cannot just add translators as authors of translation title records and just reach through the VT to find the original authors instead of adding a new contributor type (we do not need one for artists they just tend to have different types of works; a different language VT work is a different type of work). If we implement such, we could have a cleanup report for translations where the canonical authors (jumping through pseudonyms) match authors of the original work allowing us to convert these over time (translations without translators report). Uzume 23:20, 19 February 2017 (UTC)
So you are proposing the Bulgarian variant of Dune to have as an author the translator? How would you separate on the translator page the ones he did write from the ones he translated? And if a user does not understand out variant system, they can misunderstand who the writer is. Plus this way we are losing the ability to have the actual author name in the language of the variant... Artists are different because they are different works - the translation by definition has two authors - the original one and the translator. Annie 23:24, 19 February 2017 (UTC)
The software can display these differently by reaching through the translation variant and author psuedonyms to properly credit authors vs. translators. Uzume 23:33, 19 February 2017 (UTC)
How about an example. Dune by Frank Herbert is translated as Дюн by Франк Хърбърт, translated by Виолета Чушкова. Which name goes where in your idea? Annie 23:37, 19 February 2017 (UTC)
I am assuming we are referring to T1442269. I am suggesting we just add Виолета Чушкова to this record as an author and the software just credits the translators differently than the authors by chasing the authors of this record from A237335 to A30 (since A237335 is a pseudonym) and whatever number Виолета Чушкова ends up (since that is not currently in the DB) and comparing that to the authors from T2036 (which is the variant translation parent of T1442269) which is A30. The software can clearly detect that A30 is the original author and whatever number Виолета Чушкова ends up is not (and thus is a translator). In a similar vein, cleanup reports could be generated for translations without translators. It should be noted that all of the above data is already required to display and properly link these entries anyway. This proposal just suggests handling that data differently to allow for and credit translators (there is already an FR900 for displaying translations differently; I just want to add translator credit support to that and propagating it to more than just title pages). Uzume 00:10, 20 February 2017 (UTC)
None of your links work by the way - you may want to look at the templates again:) The example was from my library. I get where you are getting with that. But... what about the case where it is not just a translation but also an abridgement. This is when the second author gets added now. And that will make the translator author page a nightmare for the software to work out - in which cases the author is a translator, in which cases he/she is a co-author, in which case an author. Not to mention that the searches for translators will become overly complicated. Annie 00:14, 20 February 2017 (UTC)
Currently our rules rule out using the VT mechanism for any derived (where the contents is significantly different) works except translations so that should not be an issue. In partially translated works and the like there should be no VT parent and whether an author is an translator or not becomes muddled to begin with. I believe it would be best to list all contributors as authors and specify who did what in the notes (much as is done with certain types of authorship now). Uzume 00:26, 20 February 2017 (UTC)
Not true. Abridgements are variants - in a way a lot of translations are abridgements, especially the older ones. As are two parts of a split novel. Revised novels also get varianted... Annie 00:42, 20 February 2017 (UTC)
We did not used to allow VT title records except to change the title or author and then later allowed translations (i.e., changes in the language). If indeed we now allow other derived type VT title records, I would prefer a translation flag marking a VT title record as a translation over adding new contributor role flags (add a field to title records instead of to author records). The problem with using the VT mechanism for generalized derivatives is: what constitutes a derivative? Is something a derivative if it is barely inspired by something else? As you can see this gets rather sticky quickly and the resulting connections are more then the VT system can support (since that would actually require a generalized graph construct). Uzume 01:38, 20 February 2017 (UTC)
We are already using them for that (as much as I dislike it in some cases - split novels for example) - the question is how to make it a bit more manageable. And as for the flags - if a variant is an abridgement AND a translation from different people, how do we determine who is who? We can use additional authors but we will need a flag on the author as well... Annie 01:52, 20 February 2017 (UTC)
You cannot use VT to specify abridgement AND a translation by different people anyway. That would require more than a two title records to express (one would have to link the translation to the abridgement and the abridgement to the original; our VT mechanism specifically does not allow generalized graph structures). Uzume 02:08, 20 February 2017 (UTC)
You cannot have a variant of a variant - so if my Bulgarian book is an abridgement and a translation, it goes under the main work directly. You need one title to express - you just need to add notes. Annie 03:21, 20 February 2017 (UTC)
Exactly, and in the same notes that qualify such you can qualify the contributors as well. If you cannot have the titles linked perfectly (outside of notes) it makes little sense to try and credit people perfectly. Just as you have to pick the parent (even if it is not right) you can pick if it is a derived work or a translation (even if neither of those is entirely right either). If we ever support such a linking then the credits will be able to be right too. I see little reason to hack around credits to solve the real problem. Uzume 04:12, 20 February 2017 (UTC)
And here we ended up on a circle. I cannot see how your idea is easier than a new role (considering all the additional work that will be needed to populate summary pages. And if it is not separate, something will need to be done on the publication and title pages to identify them as translators/whatever they are anyway because new contributors will be confused. Annie 04:20, 20 February 2017 (UTC)
Easier solutions do not always make for better solutions (in fact the easy way often ends up being hard long term). I am not sure how new contributors would be any more confused than by which is the proper variant parent (which cannot be set properly too). Uzume 04:54, 20 February 2017 (UTC)

(unindent) The issue of translator support has been open for close to 5 years now, so I don't expect us to come up with a perfect solution in a few hours. There are various aspects to consider, e.g. the fact that there are awards given to translations rather than to titles.

The reason that I mentioned "roles" is that whatever design we come up with will ideally also support other "roles" like single-author collection editors. The exact roles that we will eventually choose to support are less important than the decision to make the design extensible -- or not, if we find that translators are sufficiently different from other "roles". Ahasuerus 05:11, 20 February 2017 (UTC)

Yes, I was not suggesting the translators issue would be solved anytime soon. I am against dividing our author records into roles however, and just wanted to state that there are potential solutions for the translators issue without depending on the introduction of divided contributor role tags. If we do end up supporting such role tags, I would recommend corollaries for publisher roles (or any other authority control item), etc. Uzume 05:16, 20 February 2017 (UTC)
I don't know how useful it is to researchers, but I like the idea of being able to capture/catalogue various roles. Part of the challenge right now is that the nature of the relationship is embedded in one of the two ends (mostly in the publication/title end, but sometimes in the author end). Thinking about the relationship in terms of a generic "contributor", I could see typed Contributor relationships instead -- the nature of the relationship stored on the relationship. There could be many of these, and a pre-defined set of types. One type would be "Author", another "Editor", another "Translator", and so on. The bibliographic summaries for people could be organized by roles, then title types. To keep things simple and avoid big mistakes, the software could provide some specialized handling for required roles (e.g., it could still present Author (for single works, collection, and omnibus) or Editor (for anthology and magazine)) and then "other contributors" or something like that for everything else (just like we get the Title record but then other Contents). --MartyD 17:42, 20 February 2017 (UTC)
I have been thinking along similar lines for a while now. However, redoing the author logic (and the variant title logic and a lot of other things) would be a daunting proposition. It may be less painful to declare authors and anthology/magazine editors a "special privileged case" and implement a generalized "contributor/role" mechanism for translators, other types of editors, etc, separately. Once the new mechanism is in place and everything is working smoothly, we can look into migrating "special case" authors/editors to the new system.
Granted, a two-step approach is not without its dangers -- I once worked on a multi-billion dollar project which took this route and never completed the second step -- but we may be better off even if we never complete the second step. Ahasuerus 23:59, 20 February 2017 (UTC)
Well, we need to solve it if we really want to claim that ISFDB is international. I do not want to add 3000 works just to have to edit them in a year to get the translators sorted... Just saying :) And I like Marty's idea. Annie 00:04, 21 February 2017 (UTC)

Non-Genre flag for authors

Add a “non-genre” field to author records; create a cleanup report to find non-genre titles by non-genre authors. Ahasuerus 00:04, 20 February 2017 (UTC)

This will open the gates to well-intentioned persons that will want to enter such authors (as with the "graphic" or "non-genre") as it will look possible. Bonus problem: how to define a non-genre author.Hauck 20:08, 19 February 2017 (UTC)
[Discussion of author birthdays and dates of death displayed on the front page moved here.]
I believe adding a genre field to authors should be fairly easy and provide good value (like moderator warnings on submissions when adding non-genre or graphic content to non-genre authors, etc.). @Hauck: all authors are non-genre by default and those "over the fold" get promoted to "genre" and we allow (nearly) all content for them. Uzume 21:36, 19 February 2017 (UTC)
Sure, and promoted by who? We're back to the remarkably precise definition of "over-the-threshold" (or "over-the-fold"). This may sound funny but trying to explain for the upteenth time to another new contributor all these perfectly nebular notions may prove an interesting expreience for those who do not practice it. Hauck 21:55, 19 February 2017 (UTC)
Promotion (and demotion) is via the moderated submission process (changing the author genre/non-genre checkbox). It could be restricted to moderators too (like author canonical renaming, etc.) though I am not sure that is necessary. Education is always useful and we already have the rule. The only difference is having a field to declare such. Uzume 22:58, 19 February 2017 (UTC)
I really do not like the idea. I do not think that we should have non-genre works even by the over the threshold authors but that ship had sailed. And how we will determine who is genre author? If someone has a story/essay here, they are an author with genre works. Anyone that has at least one non-genre work is again an author with genre works. Trying to draw the line in the middle is going to cause a lot of unnecessary edits.
I can think of a few Bulgarian authors that had written both SF and Crime works and are considered masters of both. I would not add their crime works here or call them non-genre because they just happen to also had written outside the genre. Even the ones that under the usual "rules" will be over the threshold. Annie 00:10, 20 February 2017 (UTC)
This is already specified in our Rules of Acquisition. I agree some cases are not clear cut. If you want to challenge the rule (or champion a specific author for promotion or demotion) that is a different discussion than implementing it in software. Uzume 00:20, 20 February 2017 (UTC)
It does not determine what a non-genre author is. Again - where would you like to draw the line? If you want it to be based on what an editor wants to nominate, it becomes meaningless... Annie 02:14, 20 February 2017 (UTC)
What you are saying is the current Rules of Acquisition are meaningless in this regard (and I agree that could be argued) but that is a different (though valuable) discussion. Uzume 02:38, 20 February 2017 (UTC)
No, what I am saying is that the rules determine which works are added but do not qualify authors as genre or non-genre ones (above the threshold is not a determination of that).Annie 02:52, 20 February 2017 (UTC)
This seems a pointless designation. If an author has written a genre story, they are by definition a genre author. If it is attempt to designate authors who write primarily genre stories, what is the point? If it is an attempt to designate authors who are 'above the threshold', that is not the same as authors who primarily write genre stories so the name is deceiving. And as Hauk has pointed out 'above the threshold' is a nebulous thing that is not well suited to a database field. -- JLaTondre (talk) 01:15, 20 February 2017 (UTC)
I am curious why you think this is not suited to a database field? I agree an author could be promoted or demoted to threshold levels over time (an author starts out in one genre and gravitates to another). Should instead every editor and moderator know and potentially reevaluate each author upon every submission? That would make for very complex submission criteria and potentially break the rules of acquisition as different people came to different conclusions at each point in time. Uzume 01:58, 20 February 2017 (UTC)
Everyone has a different opinion of 'above the threshold'. People already come to different conclusions. A database field is not going to change that nor is it going to change that different editors and moderators apply it differently. It would just be a field they can now edit war over. -- JLaTondre (talk) 02:17, 20 February 2017 (UTC)
Edit warring over that single field is considerably more noticeable than generalized edit warring over all the pubs and titles of an author which what we are already at. If there is an edit war over that field it will be noticed and brought to community discussion more easily and thereby a consensus can be reached. I agree we could also likely generate some sort of automated heuristic to flag if an author should be above the threshold (by counting title and/or pub records) but it would not take into account exceptions by consensus. But the proposed flag is meant to hold the current value of community consensus (which obviously can change). Uzume 02:32, 20 February 2017 (UTC)

Derived and corrected ISBNs

Add two checkboxes next to each ISBN: “derived” and “corrected”. The latter will let us capture two versions of invalid ISBNs, the stated one and the corrected one. Ahasuerus 00:12, 20 February 2017 (UTC)

Not frequent enough to clutter the screen, better use the note field instead.Hauck 20:08, 19 February 2017 (UTC)
I agree with Herve on that one. Annie 20:52, 19 February 2017 (UTC)
The proposed check boxes would only appear on Edit/Add/Clone Publication pages. Regular Publication pages would continue to display "(Bad Checksum)" for invalid ISBNs like they do now.
The advantage of having invalid ISBNs captured in the ISBN field vs. in the Notes field is that it enables searches. For example, if you search for the ISBN stated in Alpha 5 (0-345-24140-X), you won't find it because it's only available in Notes at this time. When dealing with invalid ISBNs, we have always had to choose whether to enter the stated version or the corrected version in the ISBN field. The other version was relegated to Notes where it became unsearchable. With the addition of support for multiple ISBNs, we will no longer have to choose -- we will be able to eat our cake and have it too! :-)
As far as derived ISBNs go, there are hundreds, if not thousands, of them in the database, mostly from the mid-1970s when the industry was transitioning from catalog IDs to SBNs and then to ISBNs. It's always made me uneasy that we claim, e.g., that Alpha 4's ISBN is 0-345-23564-9 even though it says "345-23564-9-125" in the book. If we add a check box to the edit page, the main Publication page will be able to indicate that the ISBN value was derived. Ahasuerus 03:15, 20 February 2017 (UTC)
I too agree and believe implementing ISBNs as identifiers (a different FR) will likely solve much of this problem anyway. The key is to differentiate identification (e.g., cover price, etc. which can be erroneous) from identifiers (which probably should not be erroneous). Uzume 00:23, 20 February 2017 (UTC)
I still believe most usages of this can be covered by treating valid ISBNs as identifiers (in the as yet unimplemented identifiers FR) and invalid ISBNs as catalog numbers. Here is some potentially pertinent information on this subject: MARC ISBN. Uzume 03:47, 20 February 2017 (UTC)
Back when I was working on the design of the "third party identifiers" FR, I considered making catalog IDs and ISBNs just another type of identifier. Eventually I decided against it for a number of reasons:
  • Unlike third party identifiers, catalog IDs and ISBNs are stated in publications
  • Also unlike third party identifiers, catalog IDs and ISBNs are displayed on various "Publication Listing" pages
  • ISBN searches are "privileged" in the sense that they are available via the main search box
  • ISBN searches use a special ISBN-10/ISBN-13 conversion algorithm so that a search on an ISBN-10 finds a matching ISBN-13 and vice versa
  • ISBNs can be "derived" or "corrected" as described above
In the end I decided that catalog IDs and especially ISBNs were sufficiently different animals to merit separate places in the database. Ahasuerus 04:17, 20 February 2017 (UTC)
Those are good points, however, an ISBN as an identifier can be converted to a 13 digit number in all cases (and left in original ISBN-10, SBN or other brokenness as stated in the publication in other cases like the catalog number field). When searching ISBN identifiers, ISBN-10s to search for can be converted to 13 digit numbers before comparisons (so there is no issue). It would not be so hard to have a combined search for these and the catalog field(s) (matching original input against catalog fields and any convertible ISBN-13 against the identifier fields). Though very common, all (and sometime any) ISBNs for an edition are not always stated in its printed publications (e.g., I have seen some that only have an GTIN-14 which contains the ISBN-13 and another digit). I am not sure it is worthwhile to implement ISBN as both catalog numbers and identifiers and to accomplish such (but it is something to think about). Another issue is broken punctuation in ISBN. I know sometimes an ISBN is stated in a publication with improperly placed hyphens (I have no idea how we handle this if at all; I am guessing we do not maintain this original statement). Uzume 04:49, 20 February 2017 (UTC)
There is a great deal (2427 lines to be precise) of ISBN formatting logic in this module. It's yet another way in which ISBNs are different from third party identifiers. Ahasuerus 15:20, 20 February 2017 (UTC)

Changing "Family Name" to "Directory Entry"

Change “Family Name” to “Directory Entry”. If we allow multiple values to support different alphabets/scripts, how will we sort by this field? Ahasuerus 02:39, 20 February 2017 (UTC)

Switching to unicode and then sorting by it will be cleanest. Until then I kinda like it as it is (as long at non-Latin entries actually have their transliterations properly and not getting defaulted to the name of the author in English. Annie 20:52, 19 February 2017 (UTC)
I have been looking forward to the DB moving to some Unicode encoding like UTF-8 for a long time (currently it is only pseudo-Unicode by way of using HTML entities). Multiple directory entries solves more than just multiple script/language issues (see ISFDB:Community Portal/Archive/Archive41#Sorting, author directory and nobiliary particles). Multiple scripts can be handled by multiple directories. Uzume 00:51, 20 February 2017 (UTC)
Multiple scripts/multiple directories becomes a bit of an issue with author names with letters from two separate scripts - Macedonian or Ukrainian names for example which combine Cyrillic (somewhat creative on some letters) and Latin letters. Which directory will have these forms? Or do we start a separate directory for each of them? Annie 02:44, 20 February 2017 (UTC)
This is not necessarily their real name anyway. I would suggest making entries for both (all Cyrillic and all Latin even if their real name is a mixture of the two). Uzume 03:03, 20 February 2017 (UTC)
For Ukrainian and Macedonian authors, it IS their real name. And we all hope we will get more international entries, right? Saying that they do not deserve their real names in a directory while we have everyone else's name there will be a bit... uninclusive. Annie 03:19, 20 February 2017 (UTC)
I do not see how that is an issue. The above mentioned "nobiliary particles" discussion proposed having multiple entries based on different cultural conventions in different places. Uzume 03:29, 20 February 2017 (UTC)
So separate directory for each language we support (I am just trying to make sure I understand the idea) Annie 03:37, 20 February 2017 (UTC)
No, not for every language. One per script (all Latin in one, all Cyrillic in another, etc.) implemented as time and demand allows. This means for example English language authors would be found in the same directory as German ones. Another example of directory entries that do not match real names would be Japanese (which would have kana entries) where I do not think we want to consider a kanji Chinese character directory (I have no clue how we would do such for actual Chinese names). Uzume 03:53, 20 February 2017 (UTC)

(unindent) Let's take a step back and examine the uses (current and proposed) of the "Family Name" (soon to become "Directory Name") field. At this time it is used to do 2 things:

Making this field a "multiply occurring field" will make it possible for an author record to appear on multiple Author Directory pages. For example, if we specify "de la Roche" as well as "Roche" as Mazo de la Roche's Directory Entries, his name will appear on two different Author Directory pages. Once we implement non-Latin Author Directories, it will also enable us to make the same author appear in multiple directories.

So far so good. However, once an author has multiple "Directory Entry" values, we will have no way of using them for sorting purposes. Unless we make one of them our "preferred sorting value", I guess. Ahasuerus 03:49, 20 February 2017 (UTC)

Agreed. And as such instead of having a "preferred" tag, why not treat this like transliterations. You have a Family name with "transliterated" Directory names on top. This would allow a mixed (potentially collating) name as a family name (we could have Cyrillic and Latin or kanji and kana in this field) and single script directory names on top. Uzume 03:57, 20 February 2017 (UTC)
I am not sure I understand what "on top", "mixed name" and "potentially collating name" mean in this context. It sounds like you may be suggesting that we keep the current "Family name" field and add a new multiply occurring field for "Additional Directory Entries". The latter would be used for names like "Roche" in the "Mazo de la Roche" example above as well as transliterations. The former would continue to be used for sorting purposes. Is that about right? Ahasuerus 15:45, 20 February 2017 (UTC)

Add support for additional title types

Add support for additional title types like PLAY and FICTITIOUS ESSAY (requires discussion to come up with additional values) Ahasuerus 02:41, 20 February 2017 (UTC)

Not enough candidates to warrant new types.Hauck 20:08, 19 February 2017 (UTC)
Maybe not for fictitious essays; --Vasha 00:52, 20 February 2017 (UTC)
"Fictitious essays" or "fictional essays" are basically "in-universe" essay. They are written as if the events of the work that they are associated with were true. They have prompted many discussions since it's not clear whether they are "essays" or "short fiction". Help currently says:
  • Some books contain fictional essays, purporting to written by a character in the book, as introductions or afterwords. There is no "FICTIONAL ESSAY" title type, so you have to choose whether the title is better described as SHORTFICTION or ESSAY.
There have been repeated request to implement a separate title type for them. Ahasuerus 02:48, 20 February 2017 (UTC)
Well then, if there have been repeated requests, clearly there are enough cases to justify adding the new type. And I will say it definitely sounds useful to me. --Vasha 03:37, 20 February 2017 (UTC)
but for plays, even if there aren't that many of them, the difference between them and SHORTFICTION is so obvious that they need their own title type. --Vasha 00:52, 20 February 2017 (UTC)
I would also like to see a separate title type for plays, but we will need to decide how to handle related cases first. For example, are movie scripts, TV scripts, etc separate types or are they all the same type? If they are the same, should we come up with a more generic term than "play"? Ahasuerus 02:51, 20 February 2017 (UTC)
"script" covers plays, audioplays, tv scripts, and even librettos I think. --Vasha 03:37, 20 February 2017 (UTC)
(And for the other one I would like to see a new type for, excerpts, there really are a lot.) --Vasha 00:52, 20 February 2017 (UTC)
The vast majority of excerpts are works of short fiction. Should we treat them as a separate title type or as a separate "length" value like novellas, short stories and novelettes? Ahasuerus 02:52, 20 February 2017 (UTC)
I'd like a separate length value for these - this will separate them from the real short fiction. Annie 02:54, 20 February 2017 (UTC)
Keep in mind that if we add "excerpt" as a new "length" value, it won't separate excerpts from other works of short fiction on Summary pages. Ahasuerus 03:18, 20 February 2017 (UTC)
I believe you mean it won't currently separate such (the software could be made to do such without huge issues though I am not convinced that is a good idea). I vote for an "excerpt" "length" value. Uzume 03:21, 20 February 2017 (UTC)
I do not like seeing novel excerpts mixed in with the actual short fiction on the summary pages. --Vasha 03:37, 20 February 2017 (UTC)
It does not do it now anyway (See Asimov - the third and the forth are excerpts... If we go for separation one day, we may start talking about separating all lengths from each other anyway. What I meant upstream was that it will allow the excerpts to be seen easier. And allow searching them - and searching without stumbling at them. Annie 03:40, 20 February 2017 (UTC)
Agreed. Though I am not convinced about separating such on author pages. One can create custom pages via search that show just what you want. Uzume 04:17, 20 February 2017 (UTC)
I would be strongly against separating all lengths on author pages. It's not of interest under most circumstances, plus every author has lots of unspecified lengths. --Vasha 04:21, 20 February 2017 (UTC)
I am not exactly vying for it but it might be interesting to consider user preferences allowing one to select which lengths one wants to see (much as language selection is possible this way). Uzume 04:25, 20 February 2017 (UTC)
I like that, it would speed up searching for titles on prolific author pages. Maybe checkboxes for all title types on the author page rather then setting it in user preferences.--Rkihara 18:23, 20 February 2017 (UTC)
The question wasn't choosing which title types to display, but rather displaying ss/novelette/novella separately. It's this that I think would not be good on summary pages. As a preference option, OK maybe. --Vasha 20:55, 20 February 2017 (UTC)
If the intent is to separate excerpts from other SHORTFICTION titles on Summary pages, then I believe they need to be turned into a separate title type. Using "length" values to create separate Summary page sections would go against the basic "one title type = one Summary page section" principle. Ahasuerus 23:43, 20 February 2017 (UTC)
Excerpts are short fiction -- it does not really make sense to have them as a separate type any more than novellas are separate type. At least I cannot see a difference... If we split them in a separate type, then why would not things like half a novel published in a volume go into that same new type? Where do we draw the line? At least as a part of the short fiction, that line is clear. Annie 17:05, 21 February 2017 (UTC)
Yes, sometimes excerpts are carefully selected so that they form a complete short story; more often they actually don't seem complete in themselves. --Vasha 17:39, 21 February 2017 (UTC)

(copied from a side discussion in another section) If excerpt got added to the mix [of length options] it would be hard to "sort" that with the others. Uzume 01:28, 25 February 2017 (UTC)

I think you may be the only one who likes the idea of "excerpt" being a length... everyone else was talking about it being a title type --Vasha 01:32, 25 February 2017 (UTC)
That was not how I read things (but perhaps I misunderstood). As title type seems that seems problematic. As a length, I could usefully apply it to essays and nonfiction (like an excerpt of a bibliographic index, etc.) as well. Uzume 01:43, 25 February 2017 (UTC)
I read it as a support for Length as well :) However to your latest point here: length may be showing when you are working on publications for all titles but if you read the documentation and look at editTitle for example, it is valid only for Short Fiction. Making it usable elsewhere will probably require a lot of other changes (even if it becomes a length). Annie 01:46, 25 February 2017 (UTC)
OK, I can sort of see excerpt as a length, in that I hesitate to assign a length to excerpts -- they don't strike me as "really" being a short story or novelette. But that is the very reason that they seem to me to be a separate kind of thing, not short fiction. Besides, I like the idea of having excerpts be a separate section on summary pages, for which they'd have to be a title type. But maybe I am the only one who wants that. --Vasha 02:15, 25 February 2017 (UTC)

Third party identifiers

Support for third party IDs (OCLC, BLIC, LCCN, BNF, ASIN, Goodreads, etc). This will enable Fixer to submit ebooks without ISBNs, an increasingly common scenario.

Good to have some way of identifying ebooks but most of those you listed are ephemeral. Particularly ASIN. (I don't put ASIN in the notes field of an ebook because there are probably numerous ones most of which will disappear soon.) Goodreads has the advantage that they keep records for out-of-print ebooks (it's their policy not to delete) but they're very unsystematic, their records can get merged with each other, etc. --Vasha 17:49, 20 February 2017 (UTC)
Library IDs (OCLC/WorldCat, BLIC, LCCN, BNF, etc) are generally stable, at least as stable as our IDs. We currently link to them in Notes, which requires crafting URLs manually, an error-prone and time-consuming process. In addition, manually built links can become defunct if/when the third party changes its URLs conventions, as we have seen over the last few years. ASINs may not be quite not as stable as, says, LCCNs, but in most cases they work even years after the data was captured. They are frequently the only way to link to the source of our data. Even more importantly, without ASIN support Fixer is unable to submit ebooks without ISBNs, which are increasingly common. Even major authors publish e-stories without ISBNs these days and Fixer can't do anything about them. Ahasuerus 18:10, 20 February 2017 (UTC)
I would certainly like to be able to use library IDs as identifiers for non-ISBN print books. Having support for that makes sense. But Worldcat doesn't list ebooks -- what about other libraries? --Vasha 18:30, 20 February 2017 (UTC)
Some do, but the intent of this FR is to support third party IDs regardless of whether they are for paper books or ebooks. Ahasuerus 18:46, 20 February 2017 (UTC)
As for automatically importing ebooks, that opens a can of worms. We had an inconclusive discussion a while back about what constitutes an "edition" of an ebook. Personally I would like to see them condensed not too many; the way Goodreads has so many is bewildering and ugly. So if you automatically import them there's have to be manual combining. --Vasha 18:30, 20 February 2017 (UTC)
Fixer has been generating ISBN-based ebook submissions for many years. Submitting ASIN-based ISBN-less ebooks should be no different than submitting ISBN-based ebooks once we add support for third party IDs. Where it will get challenging is if we start using another source of ISBN-less ebooks. Suppose Fixer creates a database record for an ISBN-less ebook whose ASIN is "ABC". 6 months later an editor (or another robot) wants to add a record for a Barnes & Noble ebook whose B&N ID is "XYZ". How can we tell whether it's same version of the ebook, especially if the publisher is not specified and the publication dates are close? OTOH, as long as we stick to ASINs, we should be in reasonably good shape since different ASINs are generally associated with different editions. Ahasuerus 18:56, 20 February 2017 (UTC)
Amazon actually doesn't have every single ebook. Graydon Saunders is a fairly notable example of an author who choose not to offer his books via Amazon because he objected to their terms. --Vasha 20:21, 20 February 2017 (UTC)
Oh, definitely. However, as long as we add these books manually, we are less likely to run into problems. It's the robotic submissions that worry me. I have seen many cases where an ebook was originally added by a human editor without an ISBN -- because Amazon doesn't display ISBNs for ebooks even if they exist -- and then Fixer came along and tried to add the same book using its ISBN. (This particular problem will be alleviated by adding support for third party identifiers.) If we were to start using other online sources to grab ISBN-less ebooks, the problem would become more severe. Ahasuerus 20:35, 20 February 2017 (UTC)

(unindent) I believe the next most interesting item is adding support for identifiers. This is a bit sticky but useful concept. The sticky part comes in where we apply these. I believe you were mostly considering at the pub record level, however, the identifiers in this area are themselves usually defined for the concept of an "edition" (which we have no real records for since our pub records most closely correspond to a printing, save for SFBC). This probably is not a major issue but something to consider. Uzume 21:36, 19 February 2017 (UTC)

True, different catalogs and Web sites create records at different levels. For example, an OCLC record may cover multiple ISBNs. Alternatively, multiple OCLC records may cover the same ISBN. I don't think there isn't much we can do about it. Adding support for third party IDs is the best I can think of. Ahasuerus 19:50, 20 February 2017 (UTC)

Next, there is the issue of untangling ISBNs. Currently, our pub record ISBN field represents an identifying mark of the publication (along with cover price, etc.) and we just sort of play games with it to allow is to link to other sites by using it as an identifier. Uzume 21:36, 19 February 2017 (UTC)

Well, ISBNs were originally supposed to be unique and they are still used as "mostly unique" by the industry, e.g. when ordering books. Of course, as we know, publishers can tweak things like cover art and price while keeping the same ISBN. And sometimes they mess up and reuse an ISBN which they didn't intend to reuse. Still, ISBNs are the only IDs that can be used cross-catalog and cross-site with a reasonably high chance of success. Our use of ISBNs to link to other sites is the way they were intended to be used. It's not perfect -- see the OCLC discussion above -- which is one of the reasons why adding support for third party identifiers is useful.
For example, suppose an editor wants to record that some of our data originally came from OCLC records. At this time the only choices are either to:
  • enter something like "Corrected page count from OCLC 1234567, artist from OCLC 987653", or
  • manually craft URLs pointing to the OCLC records
The second options provides more value but is error-prone and time-consuming, not to mention that URL structures can change over time. Once we add support for third party identifiers, editors will be able to enter OCLC record numbers directly. Ahasuerus 20:05, 20 February 2017 (UTC)

An easy way to untangle this is to run some code on our ISBN fields and move them to ISBN identifiers and keep the original field for verbatim publication catalog markings (including wrong/broken ISBNs, etc.). This way we change the current field to a verbatim publication identification field and move identifiers and external linking to use the newly introduced identifiers (including derived/corrected ISBNs etc.). Then we just have to update documentation to keep the fields separate (I recommend renaming the current field as catalog number and documenting it to apply to cover or copyright page catalog numbers, etc.; we could expand it to allow multiple values but frankly I would say that unneeded and can be covered in the note field). Finally, identifiers are also useful at other levels such as author and/or publisher authority control (e.g., VIAF, LCNAF, etc.), and for tags with regard to subject indexing terms (e.g., LCSH, FAST, etc.). Some identifiers also apply to the work (our title record) level (e.g., Open Library works, Wikidata, etc.). Uzume 21:36, 19 February 2017 (UTC)

Yes, author-based and title-based identifiers are something to consider. Currently we enter VIAF links as "Web sites". If and when their URL structure changes, we will have to change the URLs manually or write a conversion script. If we had author-based third party identifiers, the problem could be resolved in 10 minutes. I agree that they would be nice to have, but I see it as a much lower priority than publication-based identifiers. Ahasuerus 19:32, 20 February 2017 (UTC)
Maybe, but we could for instance move Wikipedia links to go through Wikidata identifiers with links like enwiki/Q982133, frwiki/Q982133 and jawiki/Q982133 for John Norman allowing the links to be translatable (without storing and maintaining all the various links) not to mention other possibilities like enwikiquote/Q982133 (provided the sitelink for such articles exist). Uzume 00:47, 25 February 2017 (UTC)

(unindent) OK, I have been thinking about this -- how about this for an addition to the edit-publication page. Below the ISBN field, the ability to add fields for an indefinite number of other identifiers (a button that says "Add Identifier" or whatever), and next to each of these fields a dropdown list to specify what the identifier is: ASIN, OCLC number, publisher's catalog code, however many options seems useful. (Naturally, the submission would be flagged if you try to submit a number without specifying its type -- and maybe, at least for some of them, could be flagged if it's not a valid format for the type specified.) Would that work? --Vasha 01:52, 2 March 2017 (UTC)

That's actually extremely close to the implementation that I sketched out last year :-) Ahasuerus 03:11, 2 March 2017 (UTC)

Changing the front page

As we discussed earlier, the list of authors born today is becoming longer and longer. It's up to 70+ authors now and will only get longer as we enter more DOBs.

Based on the last discussion, here is what I propose we do:

  • Create a new Web page which will show a list of all authors who were born/died on this date
  • Limit the left column ("Authors Born On This Day") to a list of 10 people born on this day
  • The list of 10 people will include award-winning/nominated authors as well as "marque" authors, i.e. the most viewed authors in the database (2%)
  • Display a link to the new page which shows all authors born on this day (see above) in the left column
  • Replace the right column ("Authors Who Died On This Day") with a list of most recently edited records. It will contain the first 10 values from the "Subject" column of the Recent Edits list.
  • Display a link to the full list of "Recent Edits" in the right column

How does it sound? Ahasuerus 01:36, 1 March 2017 (UTC)

I really do not see the value in this. As discussed, every heuristic has its flaws and though the list is now larger, it is relatively small (in terms of bytes transferred; I concede it takes up consider screen real estate vs. bytes transferred). If the screen real estate is the issue, why not just move those sections down below the "Selected Forthcoming Books" section. That way it is still there but way "below the fold" when viewing the page and one has to scroll to get there (meaning still there but less prominent). I thought I would note approximations of these lists can be generated with advanced searches like: Birth 03-01 and Death 03-01 Uzume 16:49, 1 March 2017 (UTC)
I think a smaller list of birthdays would in fact be more interesting. The long list doesn't catch the eye; people would be more likely to read it if it was shorter, and if the first few names included someone familiar. They'd note the familiar ones and maybe click through to the one or two they didn't know. --Vasha 18:38, 1 March 2017 (UTC)
If we are considering pushing the larger list to a separate page, why not make it more of a date-based author directory (meaning one can see such lists not only for today but also for other days and perhaps have it navigable via a calendar of sorts). We might even have links into these pages (I can possibly foresee links from authors birth and dates to these labelled things like "Other authors born the same day" and "Other authors who died the same day"). Uzume 18:46, 1 March 2017 (UTC)
We can certainly create an additional Web page which will look like a calendar. Clicking on one of the 366 choices will take you to a list of authors who were born/died on that day. Ahasuerus 20:15, 1 March 2017 (UTC)
I love that idea. :) Annie 20:35, 1 March 2017 (UTC)
FR 998 has been created. Ahasuerus 19:23, 3 March 2017 (UTC)
Sounds okay for me, except that I think the list of "Authors Who Died On This Day" would pair more graceously with the list of birthdays. Maybe it would be possible to feature the list of recent edits more prominently (analogous to the list for all birthdays). Stonecreek 20:01, 1 March 2017 (UTC)
I think there is enough real estate on the front page to keep the two author columns and add a third one for recent edits. OTOH, there have been complaints (both here and elsewhere) about the "Death" column being too morbid to appear on the front page. At one point someone even requested a user preference to hide this column. Ahasuerus 20:19, 1 March 2017 (UTC)
Well, as someone that prefers smaller screens on her desktops and laptops and with all the people using tablets and phones (I do the latter to edit now and then but I am not expecting to see everything well), I would be careful with adding more columns. Annie 20:35, 1 March 2017 (UTC)
It would be also nice to have a special representation in case an author's birthday happens to be 50, 100, 150, 250 years ago (regardless of prominence or awards). Stonecreek 20:01, 1 March 2017 (UTC)
That's a good point. We'll have to think of the best way to implement it. Ahasuerus 20:20, 1 March 2017 (UTC)
I like the proposals, especially the new list of recent edits, because that way the occasional user can immediately see if and how the site is still active. The "calendar" mentioned above is a nice idea, however it should be of low priority compared to other new features. Jens Hitspacebar 21:12, 1 March 2017 (UTC)
Here's the way I organized it for the speculative fiction portal on Wikipedia. There are additional items, too, but this should convey the general idea. ···日本穣 · 投稿 · Talk to Nihonjoe 21:23, 1 March 2017 (UTC)
Gets my vote!--Rkihara 18:59, 2 March 2017 (UTC)
That is exactly what I meant and yes it could be expanded to include things like awards (Person X won the A award for work W today/specifc day Y years ago, type of thing), and birth and death anniversaries as well, etc. They could be sorted by age (and possibly grouped in increments like 25 years or so), etc. I do not really see the need for the change to the front page (though it could be reorganized moving such parts down if it bothers people) however if the birth and death dates do get removed (or restricted to notable by some heuristic) from it, I believe it should be either totally disposed of (and just made into search links as I mentioned above) or expanded in the calendar directory fashion. Uzume 20:01, 2 March 2017 (UTC)
I still think that we should have a "representative sample" on the front page. A calendar is a good idea but removing the list from the page altogether just does not sound right :) Annie 20:08, 2 March 2017 (UTC)
You can see how the SF portal is organized here. Maybe that will give some ideas on organization? ···日本穣 · 投稿 · Talk to Nihonjoe 21:33, 2 March 2017 (UTC)
I've seen it. :) I find it overloaded and containing so many things that you cannot find anything. I like our much smaller and cleaner (and with a lot more white space) one a lot easier on the eye and a lot more likely to have something that catches my eye. It needs to be like that I guess - because of the scope but turning ours into something like that will be a bad move in my opinion. Annie 21:49, 2 March 2017 (UTC)

(unindent) Outcome: FR 999 has been created to reflect the originally proposed design. Once it has been implemented, we can tweak what is displayed in the columns. Ahasuerus 19:23, 3 March 2017 (UTC)

Mayan languages added

"Mayan languages" have been added to the list of supported languages. Ahasuerus 21:19, 2 March 2017 (UTC)

Yey! I will go and fix the two authors that need it. Annie 21:43, 2 March 2017 (UTC)

Standardizing "Edit" links

"[Edit]" links were added to the bibliographic pages a few weeks ago. They are currently displayed as regular HTML links in square brackets. At the same time we display the link that takes you to the tag editor ("or manage Tags"), the "VOTE" link, the "view full Note" link and the "View all covers" link using a special "inverted" font. See Garan the Eternal for an example of a title page that uses all 4 links.

Should we standardize the appearance of these links? If so, should we use the regular link format or the inverted one? It's a trivial change either way, we just need to decide what we want these links to look like. Personally I think that the "inverted" font looks spiffier. Ahasuerus 21:45, 2 March 2017 (UTC)

Now that you mention it, having the edit link in the inverted form would look good. But the location is right I think. (BTW, out of curiosity, how often does the "Vote" function get used?) --Vasha 21:51, 2 March 2017 (UTC)
According to the ISFDB Statistics page, 9,311 (or 0.72%) titles have votes associated with them. Ahasuerus 21:59, 2 March 2017 (UTC)
I prefer the standard links - it took me a bit to actually realize that the inverted font meant links when I joined. Annie 21:53, 2 March 2017 (UTC)
As I recall, some other editors have run into this problem a well. Hm... Ahasuerus 22:00, 2 March 2017 (UTC)
Seems quite obvious to me (and also a good way of visually distinguishing functions, versus links to data pages which are ordinary links), to add a data point in the opposite direction. --Vasha 22:18, 2 March 2017 (UTC)
I must be particularly dense. What is an inverted link font anyway? Anyone care to explain (Annie's comment about "links when I joined" does not really help me and I joined ISFDB years ago). Uzume 01:43, 3 March 2017 (UTC)
The links that look like they are highlighted in blue (blue background, white letters). Look at the link from Ahasuerus above - there are a lot of them. Annie 01:50, 3 March 2017 (UTC)
The links that I mentioned earlier use a blue background. The CSS definition is a follows:
.inverted {
   font-size: 90%; 
   background: #00f;
   color:#fff;
   text-decoration: none;
   padding-left: 5px;
   padding-right: 5px;
   white-space:nowrap;
}
I now understand, what is meant is the CSS class named "inverted" which has colors that apparently seem inverted from other links (I do see them as white on blue). Thank you. Uzume 02:34, 3 March 2017 (UTC)

I cannot agree on how spiffy such links appear, however, I do agree it makes sense to differentiate action links from normal navigational data links (which should likely be persistent and externally linkable; action links like edit and vote, etc. require a login and need not be persistent so could be renamed and moved in URL space). Uzume 05:04, 3 March 2017 (UTC)

Two new title types to implement

There seems to have been general acceptance of the idea of creating new title types for scripts and fake essays (what to do about excerpts, on the other hand, remains uncertain). I think they would be easy to implement -- almost all existing examples of scripts could be found by searching notes, and as for the essays, some creative searching could find them too. The only thing left to decide is what to call them. I suggest "script" and "fictive essay" (not "fictional"). --Vasha 23:23, 2 March 2017 (UTC)

"In-universe essay", perhaps? Ahasuerus 23:49, 2 March 2017 (UTC)
It ought to include things like Álvaro Menén Desleal writing a preface for his own collection of stories and signing it Jorge Luis Borges, which was intended and received as a joke between fellow South American spec fic writers. Borges was himself a writer of metaliterary pieces like the "Chronicles of Bustos Domecq", so does that make Desleal's essay somehow, some way, in-universe? --Vasha 00:41, 3 March 2017 (UTC)
The Contento1, Locus, Miller/Contento and FictionMags sites use "facetious article" with the abbreviation "fa". For scripts, they use "play" with the abbreviation "pl". FictionMags specifically states that play is meant to include scripts of films, radio and tv episodes. Their full list of types is here. I haven't done a full compare, but I believe that these databases (three of which are our standard secondary verification sources) essentially use the same set of title types. --Ron ~ RtraceTalk 00:50, 3 March 2017 (UTC)
"Facetious" suggests humorous intent which isn't necessarily the case; for instance, an introduction to the world of a fantasy series written as a supposed excerpt from a guidebook. --Vasha 00:58, 3 March 2017 (UTC)
For reference, "comic book" also suggests humorous (the Japanese word "manga" also has similar comical meaning) but methinks that term is well entrenched in English now (though we tend to use "graphic novel" here). Within some of the works of the Gor series written by John Norman, a pseudonym developed by the real author, are envelope notes. I believe this is mostly within the earlier works of the series but I know one example is Outlaw of Gor (I happened to have a copy on hand for reference) which has a leading "A Note on the Manuscript" and "A Concluding Note on the Manuscript" which are signed by John Norman and explain how he came to get the material from a lawyer friend named Harrison Smith without actually meeting the main character of the novel, Tarl Cabot. Further in that novel, the first chapter entitled "The Statement of Harrison Smith" is purportedly by the lawyer friend. The rest of the novel uses the first person to refer to the main character Tarl Cabot. I suppose these pieces of the novel might qualify under this proposed fake essay/facetious article title/work type, however, I am not convinced we need to separate these novel pieces as separate works (anymore than we denote all the chapters of a novel here). If we do in fact include such such a new title/work type, we need to be clear when it should be invoked. Frankly, I would rather we had another title checkbox or the like (similar to graphic and non-genre) to denote such things instead of a new title type (then it could be applied to any fictional title type that applies). In a similar fashion I prefer making excerpts a length. I see no issue adding a script/play title/work type and see it much as our current poem title/work type. Uzume 02:24, 3 March 2017 (UTC)
I believe there are two separate questions here. The first one is which framing stories and in-universe essays need to be entered as separate titles. The second one is how we want to enter the ones that should be entered separately. The current proposal deals with the second question only.
Making "fictive" a check-box on the Edit Title page is an interesting idea which hasn't been suggested before. Something to ponder... Ahasuerus 03:21, 3 March 2017 (UTC)
I fully agree. The point was to think about things as considering how we want to use something often influences how we want to and ultimately how we implement them. Uzume 05:49, 3 March 2017 (UTC)
Well, for what it is worth I needed a dictionary for the word "facetious" a few months ago when I crashed into our sources for the first time while looking for something. If we use this, a lot of international contributors will be a bit lost - the system is already complicated (EDITOR is already a challenge and it is technically never added manually; this one is something people need to be able to select from a list... I understand that it may be the standard but can we come up with something that does not require someone to look up the meaning of - not everyone had even heard of our sources; and even when they had, understanding what this type is when one had not seen a US magazine that contains them is a bit hard. I like "In-universe essay" - it is clear enough so someone seeing it on a list can recognize a piece of writing as being one of those. Just my 2 cents. Annie 16:59, 3 March 2017 (UTC)
Yes, seriously. "Facetious article" may be acceptable as insider jargon, but since this database is aimed at casual users, we need to pick a less bewildering expression. --Vasha 17:10, 3 March 2017 (UTC)
I still don't see the need to implement a new title type for 'fictional essays': if it's in-universe it is fiction, however 'serious' the overall tone may be, if it is rooted in reality it is non-fictional, and as such an essay (ESSAYS being the little offsprings of NONFICTION, SHORTFICTION the ones of longer works of fiction, i.e. NOVELS). Why make it complicated if it can be quite easy?
A 'script' or 'play' title type would be nice, though. Stonecreek 18:52, 3 March 2017 (UTC)
I agree with Christian about the first type, it's useless. The plays are, IMHO, not frequent enough to warrant a new type (why not also TVSCRIPT, FILMSCRIPT, PROSEPOEM, HAIKU, VIGNETTE, etc...). On a more general theme, please remember that the multiplication of possible choices in our categories (let's not talk about format/binding) makes the task of explaining (and correcting) them more and more difficult for those of us that undertake the task of "speaking-to-new-contributors". I'm not sure that all of the participants of this discussion remember clearly how complicated the ISFDB is for a new entrant.Hauck 19:02, 3 March 2017 (UTC)
I do not think that a "play" type would be at all confusing, and even though it's true that there's only about a hundred examples so far, still they're obviously distinct. As a bonus having this title type would allow doing away with some disambiguators like "Dracula (stage play)".
As for the confusingness of fictive essays, I'm coming around to the idea that having them as a check-option for shortfiction would be best. It's less serious if people don't check boxes that should be checked than if they assign things to the wrong type. --Vasha 20:22, 3 March 2017 (UTC)

(unindent after an edit conflict) Let's take a step back and consider whether "facetious/fictive essay" would be best handled as a separate title type or as a flag similar to the currently existing "non-genre" flag.

Aside from the Title page, which is self-explanatory, I can think of two bibliographic pages which will display the data differently depending on our choice:

  • the Publication page, specifically its Contents section, and
  • the Summary page

If we make it a title type, the Contents section of a Publication page will say something like:

  • vii • Introduction (The Statement of Stella Maberly, Written by Herself) • fictive essay by F. Anstey [as by uncredited]

If we make it a flag, here is what the same line will look like:

  • vii • Introduction (The Statement of Stella Maberly, Written by Herself) • short fiction [fictive essay] by F. Anstey [as by uncredited]

As far as the Summary Bibliography page goes, the difference is that making it a separate title type will result in a new Summary section. On the other hand, if we make it a flag, then the affected titles will appear in the existing "Short Fiction" section with a "[fictive essay]" indicator added.

My current thinking is that a separate Summary page section for fictive essays may be excessive, so I would be inclined to go with a flag. We will then need to decide whether the title type of "fictive essays" should be ESSAY or SHORTFICTION. Ahasuerus 18:56, 3 March 2017 (UTC)

"fictive essay" sounds like "a little bit pregnant" to me. And if we go with that, would someone decide that this is a good flag for a 1950s science article that was essay back then (because it was considered true) but since then had become invalid (so not non-fiction)? Annie 19:06, 3 March 2017 (UTC)
As for whether this type is useful... Well, the Mad Scientist Journal is almost nothing but in-universe pieces. Every issue has an "editorial", "advice column", "horoscopes", and "classifieds" supposedly written by mad scientists. I really don't know what to call those -- they aren't short stories or novelettes. They have to be something else, whether it's a flag or a new type.
Annie's concern would be addressed if this was handled as a labelled subtype of SHORTFICTION. I don't think anyone would shift an essay from nonfiction to fiction just because it lauded Wilhelm Reich. --Vasha 20:02, 3 March 2017 (UTC)
We generally decide whether a work is fiction or non-fiction based on the stated intent. If a story about a kidnapping by aliens is presented as a true account, then it's non-fiction regardless of how accurate it is. In addition, as per our definition of speculative diction, SF includes "works set in a future that is now in the past [and] that deal with technological advances that were futuristic at the time they were published", so I think we are covered there. Ahasuerus 20:25, 3 March 2017 (UTC)
I am mostly thinking about someone not reading the rules and seeing a name in the list that sounds like something else - all of our types are self-explanatory on their own (except EDITOR - but that is a different story). I know that docs clarify it, but we have a lot of those. :) Annie 21:49, 3 March 2017 (UTC)

Submission viewer tweaked

The "submission viewer" page has been tweaked to display the name of the reviewing moderator, e.g. see this Fixer submission. Ahasuerus 16:49, 3 March 2017 (UTC)

Ethereal Tales, #7

Can anyone see any reason why these two: Publication 1 and Publication 2 need to stay separate? It does look as an oversight when the second one was added but if someone can confirm, I can go ahead and merge all of the matching elements (and find out which ones are not matching and why). Annie 17:02, 3 March 2017 (UTC)

Well, no! This really is a double entry. Stonecreek 18:44, 3 March 2017 (UTC)
I'll go clear them up then. And there is a serial hiding in there that the PV'd copy had not handled very well so I will also send them a note on the changes. Annie 19:09, 3 March 2017 (UTC)
I also noticed that the dates of the issue and contents are set to something other than the cover date -- I guess that ought to be checked for all VWCrist's submissions. --Vasha 23:44, 3 March 2017 (UTC)
I found it when one of the authors from it popped as "no language" and "has more than 3 works" (I am systematically exterminating these when they appear) and there was a duplicate in the 4 stories they had. Haven't got to looking at who had added it or why yet. And now you mentioning the name, I saw it somewhere else today as well... And I need to sort out what is a poem and what a story because there are a few inconsistencies - which will require a message to the PV :) Annie 23:58, 3 March 2017 (UTC)

Record numbers on award pages

Award, award category and award type pages have been modified to display ISFDB record numbers. The availability of the "[Edit]" links matches what is displayed in the navigation bar on the left: only moderators can edit award categories and award types. Ahasuerus 20:54, 3 March 2017 (UTC)

Advanced Search - "NOT" changes

Based on the recent discussion of Advanced Search, I have added "is not exactly" and "does not contain" to the drop-down list. We'll see whether it approximates the requested "AND NOT" functionality.

Based on my testing, searches complete reasonably quickly if the results are manageable. For example, an Advanced Title Search on "Author's Name contain heinlein" AND "Title Year does not contain 19" generates a list of 21st century titles by Heinlein. If, however, you were to change the "AND" to an "OR" in this search, it would take well over 30 seconds to retrieve the data. Of course, the results wouldn't be particularly useful anyway.

The good news is that lengthy searches don't seem to affect other users (much) because our server has had multiple processors since January 2016. Ahasuerus 22:03, 3 March 2017 (UTC)

That is awesome. Thanks for implementing it :) Annie 22:42, 3 March 2017 (UTC)
That's great! --Vasha 23:31, 3 March 2017 (UTC)
Thanks!--Rkihara 02:28, 5 March 2017 (UTC)

Length/title type mismatches

Edit Title has been modified to prevent users from entering "Length" values for anything other than SHORTFICTION. New/Edit/Add/ClonePub is currently in progress. Ahasuerus 20:07, 4 March 2017 (UTC)

Done. If you find a title type/length mismatch, please let me know. Ahasuerus 21:32, 4 March 2017 (UTC)

Enchanced CHAPBOOK and OMNIBUS validation

Edit Title has been enhanced to perform additional pop-up validation for CHAPBOOK and OMNIBUS titles. You will get an error message if you enter series or synopsis information for a CHAPBOOK title. You will also get an error message if you enter a Content value for a non-OMNIBUS title. Also, Content values can't start with a slash and have to be less than 32 characters long. Ahasuerus 01:22, 5 March 2017 (UTC)

"Add New Omnibus" has been enhanced to use the same pop-ups as Edit Title. Ahasuerus 17:24, 5 March 2017 (UTC)

Date mismatch warnings

Import/Export submissions now show a warning when you try to import a title whose date is later than the publication date. I hope to add a similar warning to New/Edit/Add/Clone submissions tomorrow. Ahasuerus 23:11, 5 March 2017 (UTC)

ClonePub done. Also, EditPub has been modified to display date warnings for COVERART titles. It's a messy area and I need to tread careful lest I break something. Ahasuerus 23:13, 6 March 2017 (UTC)
Regular title warnings added to EditPub. Reviews and interviews are still outstanding. Ahasuerus 00:05, 7 March 2017 (UTC)
EditPub is done. AddPubs still need to be reviewed. Ahasuerus 20:51, 7 March 2017 (UTC)
NewPubs and AddPubs are done. At this point all Import/Export/NewPub/EditPub/ClonePub/AddPub submissions should warn you if a Contents title has a later date than the publication date. If you see anything odd, please post your findings here. Ahasuerus 21:44, 7 March 2017 (UTC)

Magazines and e-books

Under the current way we record magazines, there is no way to add an e-book version of the same magazine because it will show up on the grid as a duplicate. As a result we have some magazines that just show the duplicates, some where there are two separate series (for what is essentially the same magazine - causing the records to drift from each other when they are not updated together) or just not to have the e-versions recorded. The last option is what we have for the big magazines - which is in contrast to the books records where we are recording down to printings.

I know that some of you don't even want to think of the e-books but they are a reality and are here to stay. So anyone with ideas how to solve that little conundrum we are having? I do not think that keeping them in separate series really make sense (the drifting of content is going to continue), so we need other options. Moving the grid on the title level will require untangling the year records we have now... maybe allowing some intelligence in the grid where an e-book is shown only if there is no printed record and the printed record have links to the e-copy?. Or even allowing a checkbox to "not show in the grid" which suppresses the second copy from clustering but allows it to be available for search (a link will still be needed though - because we won't have the title to make it visible). Annie 16:07, 6 March 2017 (UTC)

Ha, the ebook coterie rears its ugly head again ;-). In this case, I'd prefer to have a different title series for the ebook (as we do with Clarkesworld) this won't clutter the grids, will allow as much drift as possible and will kept the problems specific to each format (e.g. if a text is deleted from an ebook by the publisher as happened IIRC with amazon) separate. We may even use the sub-series concept to link the print and the electronic version to the same parent. Hauck 16:31, 6 March 2017 (UTC)
Well, Analog and Asimov's have the same content so we need a way to make sure that if we keep them separate, changes are applied in both copies. Amazon does not delete stories from the middle of an e-book they did not publish :)Annie 16:45, 6 March 2017 (UTC)
IMHO it's an heavy trend (I don't know if it's the correct english idiomatism, we speak of "tendance lourde") to see a divergence in time between print and electronic as publishers make use of all the possibilities of the latter in term of improved (enhanced) content (it's already happening in other domains, e.g. for this magazine that I follow that has more content in the numeric version). So I bet that we will have to keep ASF or Analog separate at some point in the future. Hauck 17:19, 6 March 2017 (UTC)
True. Although I doubt that the pure text magazines will go that way anytime soon. I guess we will end up having two separate series, technically with the same content (and with links set by whoever enters the e-version) to ensure that they are linked. Or something like that. My main concern is data integrity - mistakes happen and when content is imported get populated and they won't get fixed if the magazines are not connected. Annie 18:27, 6 March 2017 (UTC)
This exactly the situation where moving magazines to be pub series over title series would help (I am not suggesting getting rid of magazine editorial records and series but just that magazines not be defined by them as they currently are with seriesgrid: "Issue Grid", etc.). It would also separate out all the magazine publications on publishers pages (which currently get lumped into pubs_not_in_series: "Publications not in a Publication Series for Publisher"). Defining magazines as pub series would allow for separate magazine runs while sharing the same editorial credits (regardless of possible content changes between runs). Uzume 16:30, 12 March 2017 (UTC)
Let me make sure that I understand. It sounds like you are suggesting that we:
  • Keep the EDITOR titles and the currently existing EDITOR series
  • Continue using EDITOR series to organize magazine records on Summary pages
  • For each MAGAZINE/FANZINE publication, enter the name of its title series in the Publication Series field (the original data population effort can be automated)
  • If a MAGAZINE/FANZINE issue had multiple editions, e.g. there was a electronic version and a print version or there were 2+ different versions published in different countries, disambiguate the publication series name
  • Rewrite the Series Grid software to use the newly created Publication Series data instead of EDITOR records
Is this about right? Ahasuerus 17:20, 12 March 2017 (UTC)
That's wonderful, I love it! --Vasha 20:05, 12 March 2017 (UTC)
Assuming that I interpreted Uzume's proposal correctly, my first concern is about data duplication. Every time the same data is recorded in two different places, it inevitably begins to diverge. Ahasuerus 01:06, 13 March 2017 (UTC)

(unindent) Yes, exactly. If we do not employ some automated mass migration magic (as you suggest), we would need an interim solution until everything is converted over. This would then allow editorial records to be later merged (we do not really need multiple records for multiple print runs unless there is some change to the editorial between runs which in most cases does not occur). For instance, once Clarkesworld (print issues) and Clarkesworld Magazine are converted to pub series, Clarkesworld (print issues) and Clarkesworld Magazine could be merged. Neil Clarke (and others earlier on) would still get appropriate editorial credits but the magazine runs would show up on Clarkesworld Magazine and Wyrm Publishing since they are now a pub series entity. There are some caveats however. For example, currently we allow titles series to link to other title series but this is not true of pub series. It would be complicated to create an automated mass migration of things currently organized like Vargo Statten British Science/Space Fiction Magazine (which is an series/issue grid of a title series with no EDITOR records but other title series with EDITOR records). Other differences to note are that pub series have transliterated names available and titles series to not (probably not an issue for this migration concept). Also currently seriesgrid: "Issue Grid" can be applied to other title series like Gor without much value, however, a pub series based series/issue grid could be usefully applied to pub series like OPTA - Aventures fantastiques. Pub series contain pubs and title series contain titles and as such this conversion would allow each pub in the pub series to also have a "Pub Series #" that could correspond to its issue number (I am not sure of a good way to automate such in a mass migration strategy). We would likely also want to change several other things like magazine searches and directories to find the pub series and not the editorial series, etc. The point is we sort of want it to diverge as they represent different things (the editorial of a magazine is related to but not the same as the magazine and its run itself). Uzume 01:14, 13 March 2017 (UTC)

As for french (and generally european publications series) I don't see the added value of putting them in a grid (vs our present system that allow a view either by chronogical or "numerical" order). Such a publication series (which is not that "special") would give a quite crowded (and IMHO unreadble grid) with #373 appearing at 15 points on the grid. The same can also be said of this american one. Note also that, because of our present knowledge and some publisher's practices, the "no-month" cell of the grid my be a bit overcrowded. Hauck 07:38, 13 March 2017 (UTC)

Updating Policy re: dates of birth

With identity theft becoming more of a concern and data deletion requests becoming more frequent, we need to update ISFDB:Policy re: living authors' dates of birth. Here is the latest major discussion of data deletion requests and other issues related to dates of birth. For simplicity's sake, I suggest that we use the same standard that Wikipedia uses:

If the subject complains about the inclusion of the date of birth, or the person is borderline notable, err on the side of caution and simply list the year.

We can eliminate "the person is borderline notable" part since we don't use notability as an inclusion criterion, but otherwise it seems to be a good fit. Also, since Wikipedia and the ISFDB server are currently subject to the same national laws, it's likely that we'll find out about any legal challenge to Wikipedia's policy before we face a similar challenge.

Based on these considerations and the linked Rules and Standards discussion, I plan to update the Policy page as follows:

If a person complains about the inclusion of the person's date of birth and provides evidence that he or she is the listed person, the ISFDB will replace the date of birth with the year of birth.

I am not very happy with the "evidence" part of the proposed language, but I see no other way to prevent vandalism and frivolous requests. Ahasuerus 16:09, 6 March 2017 (UTC)

OK for me for an american site, if we were based in France the total deletion of personal data on request is in the law. Hauck 16:23, 6 March 2017 (UTC)
Yes, "the right to be forgotten", which exists, in various forms, in a number of countries. If we ever move the server to another jurisdiction, it will be one of the things that we will need to consider carefully. Ahasuerus 16:54, 6 March 2017 (UTC)
That's why what was done without discussion and without my consent on the Moderator Availability's list (adding our complete names) is here considered very bad manners and may be downright illegal. Hauck 16:23, 6 March 2017 (UTC)
I was surprised to see the changes even though I think that they were well-meaning. Moderators generally add their own names to the Moderator Availability template and it's up to them to decide how to list their names. Ahasuerus 16:58, 6 March 2017 (UTC)
Are you talking about my recent changes to Template:Moderator-availability (as per 466661)? You will note, though I rearranged the names (making them more noticeable), I did not add any (most were in the original creation by Nihonjoe on 2015-06-04 as per 392440; some were added later with later moderator changes/adds). Uzume 00:21, 13 March 2017 (UTC)
I was referring to the addition of first names and last names, e.g. "Chris J" became "Chris Jensen" and "Dwarzel" became "Desmond Warzel". Of course, their names can be easily found on other Wiki pages or in the database itself, but it's generally up to each moderator to decide how they want their user name listed. Ahasuerus 01:02, 13 March 2017 (UTC)
I noticed Mhhutchins and Hauck recently pulled their names out but I did not add those, however, it seems you are right, a few did somehow sneak in on that edit (I did not think I pulled those in but as you say they are directly linked from their user pages which are linked in the template). My apologies. Uzume 01:45, 13 March 2017 (UTC)
I'm fine with codifying a mechanism for the removal of DOBs, but I agree the evidence part will be tricky; we will probably also need to have some guidelines as to what we will consider acceptable as evidence. Albinoflea 16:27, 6 March 2017 (UTC)
A couple of years ago an author registered as an editor and asked to have her DOB reduced to YOB. I said I would do that if she could prove who she was and that I would send an email to her website for confirmation of identity. No response, but she made the edit as an editor. I decided to let it go through.--Rkihara 18:49, 6 March 2017 (UTC)
In the case I have at hand, it's the YEAR that's private, and the month and day are publicly available, so the proposed wording wouldn't cover the desired/"needed" change.
I guess I'm more inclined to France's approach as a default: We should agree to remove any private or quasi-private identifying information upon the individual's request. A possible private-vs-public bar could be "can[not] be found via Google search". Since we have no way to prevent anyone from adding it again, and we have no way to record that the information is less-than-full information is recorded at the author's request, we would have to be clear that there's no guarantee of its staying that way (and anything public/published is almost certain to come back, which would be justification for not wasting time removing it, beyond the fact that removing it isn't doing anything to enhance information security). --MartyD 19:30, 6 March 2017 (UTC)
Keep in mind that once we finish migrating Wiki-based publisher and series/magazine pages to the database, we will add a "Note" field to author records. Once it's available, we will be able to add notes like "Date of birth removed as per author's request", which should help prevent DOBs from reappearing. (The {{BREAK}} syntax will come in handy.)
As far as this particular case goes -- the year is private while the month and day are public -- I suspect that accommodating it would quickly lead to, as you wrote, having to remove any biographical information upon request. We'll have to revisit this issue when we move our "Bibliographic" and "Bio" pages to the database, but for now I am not ready to go that far. Ahasuerus 20:02, 6 March 2017 (UTC)
P.S. As far as defining "publicly available sources" goes, it's an interesting line. Wikipedia's policy says:
  • Do not use trial transcripts and other court records, or other public documents, to support assertions about a living person. Do not use public records that include personal details, such as date of birth, home value, traffic citations, vehicle registrations, and home or business addresses.
Which, as far as I can tell, would pretty much limit us to Google and other search engines if we were to follow it. And that's not necessarily a bad thing, although there is a possible wrinkle: some search engines serve different results depending on the user's location. Ahasuerus 20:12, 6 March 2017 (UTC)
Would it be possible to add a switch (basically, a checkbox next to the field when in the editing screen) that turns off the display for that information until a date of death is entered? It could be there for date of birth, location of birth, and legal name, since those are likely the fields someone might be concerned about. That way, the information could still be in the database, but it wouldn't appear to the public unless they were no longer living. ···日本穣 · 投稿 · Talk to Nihonjoe 20:35, 6 March 2017 (UTC)
Is it really worth the effort if the information is easily and freely available on the internet? Maybe we could avoid the issue by posting the links, or citing the references giving POB, DOB, etc., instead of listing it explicitly for living authors? Then the authors can petition the root sources.--Rkihara 21:58, 6 March 2017 (UTC)
I am all for citing our sources once we have the Author Note field. Take the submission that prompted this discussion. The author in question stated that her full DOB and place of birth are not publicly available and sure enough, googling doesn't seem to find anything. So where did our data come from? It may have been a print source like Contemporary Authors or it may have come from some database that us not indexed by Google. It's hard to tell without a cited source.
As far as leaving the field blank and linking to the source goes, I suspect that it would make our data much harder to work with. URLs change all the time even when the underlying data remains the same. Ahasuerus 23:20, 6 March 2017 (UTC)
If it's the submission that MartyD has on hold, I posted the DOB and POB (I keep a short record of authors that I enter DOB and DOD for). The source was Contemporary Authors Online, which lists education, hobbies, and career hightlights.--Rkihara 00:11, 7 March 2017 (UTC)
That's a good example. Contemporary Authors Online is an established biobibliographical source which is used by Wikipedia (see this example) and other sites, but, as far as I know, it's not available without a subscription. I believe its data is made available to educational institutions and subscribers. On the other hand, they offer a free trial, so if someone is interested in a particular author's biographical data, all they have to do is set up a free trial account and look things up. So... Can we say that its data is publicly available? Ahasuerus 02:06, 7 March 2017 (UTC)
I would say that CA is so widely cited and available that it can be said to be public knowledge. It is by subscription, but I have access through several channels, including my local library, which also carries the hardcover edition, and other biographical sources.--Rkihara 04:31, 7 March 2017 (UTC)
Thanks for clarifying, I am sold on the notion that the CA data is "publicly available". Ahasuerus 16:02, 7 March 2017 (UTC)
We could build in the ability to archive online sources (using Web Citation or another archive service). I use this all the time when writing articles on Wikipedia to avoid link rot. ···日本穣 · 投稿 · Talk to Nihonjoe 23:58, 6 March 2017 (UTC)
Looks interesting. Do they let Web site owners delete copies of certain pages the way the Internet Archive does? Ahasuerus 16:42, 7 March 2017 (UTC)
They have this policy. ···日本穣 · 投稿 · Talk to Nihonjoe 19:51, 8 March 2017 (UTC)
I see. It sounds like they too have to worry about dealing with various cans of worms. Ahasuerus 22:18, 8 March 2017 (UTC)

(after sleeping on it) I guess we have a few separate issues here.

The first one is how we want to define the "publicly available sources" which we are permitted to use. I think it would be prudent to explicitly exclude government (courts, taxes, immigration, vehicle registration, etc.) records for living persons. We can let other projects take care of the headaches that they cause. For example, just recently the SFE3 editors spent a fair amount of time exchanging e-mails, including scans of immigration records, with a certain SF writer who had disputed the year of birth listed by SFE3. We bowed out of that dispute by saying that we'd quote whatever SFE3 settled on.

The second issue has to do with documenting our sources of biographical data as well as author requests for data deletion. The plan has always to add a Note field to Author records and then to migrate Wiki-based Bio and Bibliographic pages to the Note field. (We still need to decide whether we want to have a separate "Bio" field as well.) However, the original plan was to wait for the publisher/series/magazine Wiki cleanup project to be completed before adding a Note field. Given the fact that we still have 500+ Wiki pages to migrate and the need to support the ability to document the sources of biographical data, I think we should implement the Author Note field sooner rather than later.

The third issue has to do with accommodating author requests to remove otherwise eligible data. Earlier we seemed to agree that we needed to ensure that the requester was the person whose data s/he was asking to remove. However, I think we also want to allow authorized representatives (Web masters, spouses, etc) to make these types of requests. For example, the person in the previously mentioned SFE3 controversy communicated via his Web master.

The fourth issue is what types of biographical information we should be willing to remove upon request. At this point we have seen requests to delete:

  • the year, month and date of birth
  • the month and date of birth, but not the year of birth
  • the year of birth, but not the month and the date of birth
  • the exact place of birth, but not the country of birth

Should we use the Wikipedia standard ("If the subject complains ... list the year") or should we be more accommodating? What about requests to remove legal names? Ahasuerus 18:09, 7 March 2017 (UTC)

I think that issue #4 could be handled by removing the entire date and giving the online reference which contained the information. The author can hardly object to that, and it redirects the author’s attention to where it belongs - as we're just the messengers.
I would also suggest adding the following statement in bold at the bottom of biographical info, “Some personal information has been redacted at the request of the author or the author’s representatives.” This will also warn off editors who might re-enter the redacted data.--Rkihara 18:30, 7 March 2017 (UTC)
Well, we already support the {{BREAK}} template in Note fields. Once we have the previously mentioned Note field, we could add support for a {{REDACTED_PERSONAL_INFO}} template, which would display the message that you proposed. Ahasuerus 19:01, 7 March 2017 (UTC)
The author may WANT their date of birth to be private, but as we all know with varying degrees of chagrin, once something is on the internet, trying to get rid of it is like trying to mop up a flood with a kleenex. Nonetheless, there's a difference between having the information "out there" and having it on a semi-authoritative site that might be thought of as a reference source -- like Wikipedia, SFE3, or the ISFDB. I can see someone not wanting their info on the latter even when they can't do anything about the former; their info wouldn't be formally public. How sympathetic do we want to be to that? --Vasha 19:31, 7 March 2017 (UTC)
Admittedly it's a fine line. For example, suppose we were to adopt Ron Kihara's proposal immediately above: "issue #4 could be handled by removing the entire date and giving the online reference which contained the information". In the case that prompted this discussion, we would then remove the author's year of birth and add the following note: "Year of birth available in Contemporary Authors. Redacted as per author's request." If someone wanted to steal the author's identity at a later point and googled "[author name] year of birth", Google would helpfully serve our Summary page. The bad guy could then drive to a local library and look things up. Is that something we have to worry about? Where do we draw the line? Ahasuerus 22:26, 8 March 2017 (UTC)
Well, we don't have to specify what was redacted, and it will take more effort to find. If I can find the data (and I spend only a few minutes per), then someone who really wants to find this data wll find it too. I also think if an author wants the year of a birthdate redacted, we should redact it all. It really irritates me to see, e.g., born October, or born October dd, with no birth year. The same with place, but not country of birth.--Rkihara 23:21, 8 March 2017 (UTC)

Omnibus or collection

As I am always getting confused - if we have a book containing 1 novel and 3 stories, is that an omnibus (as it contains a novel) or collection (because the 3 stories are not published independently)? In my mind, a novel in the content makes in an omnibus but our conventions are a bit confusing. Annie 22:30, 7 March 2017 (UTC)

Probably collection. See the OMNIBUS bullet of Help:Screen:NewPub#Publication_Type and its specific Heinlein example. This may be pure Male Answer Syndrome, but I believe the intention behind OMNIBUS is that it collects multiple works previously published standalone (so short stories that appeared only as part of some published collection or in a magazine don't qualify). --MartyD 01:43, 8 March 2017 (UTC)
These days any story can be published on its own - making this a collection of two published works (which we call an omnibus as one of them is a novel). If we want to say that we need at least 2 novels/collections/anthologies/non-fiction (we say at least 1 now) to declare it an omnibus - I am fine with that and I think that was the intent behind this definition. But this is why I asked. Annie 02:57, 8 March 2017 (UTC)

Author Note and Author Bio: one field or two fields?

As per FR 307 and the earlier discussion of biographical data, we plan to migrate all of our "Bio" and "Author" pages from the Wiki to the database proper. Changing the software to support a new field is not hard, I can do it in a few hours. However, first we need to decide whether we want to add:

  • one new field, basically a generic "Note" field similar to all the other Note fields in the database, or
  • two fields: one for general purpose bibliographic notes and another for biographical data, which would match the current Wiki layout

It stands to reason that all Wiki-based "Author" pages will be migrated to the new "Note" field. The real challenge is deciding whether the contents of Wiki-based "Bio" pages can be accommodated by the "Note" field as well. Based on a cursory review of the "Bio" pages, we'll be dealing with the following scenarios:

  • Information about the source of our author data, e.g. "Middle name from Locus". Sometimes it can get complicated, e.g. Bio:A. V. Clarke, and require the use of the {{BREAK}} template. This data will always go into the Note field even if we decide to have a separate Bio field.
  • Additional information about pseudonyms like "A Ziff-Davis house name". Will be migrated to the Note field.
  • Basic biographical information, e.g. "Born Andrei Perlmutter" or "Poul Anderson's son-in-law" or "Vernor Vinge's wife in 1972–1979". Can be easily accommodated by the Note field.
  • More detailed biographical articles like Bio:Irene Maude Swatridge. They can be accommodated by the Note field, but they may require the use of the {{BREAK}} template.
  • Really detailed bio pages, usually added by the author, e.g. Bio:Charles G. Waugh. They are the exception rather than the rule, but they do exist. The {{BREAK}} template will effectively move them to a separate Web page, so I think we can put them in the Note field without causing a problem, but I can see how having a separate field may be useful in certain cases.

Given the above, I am 95% sure that we can get away with having just one Note field. Does this sound reasonable? Ahasuerus 00:09, 8 March 2017 (UTC)

I do not think that there is a clear line between bibliographic and biographic note in the case of authors -- does a note on pseudonyms belong to the bibliographic part or is it part of the biography for example? I vote for one note field only. Annie 00:27, 8 March 2017 (UTC)
I agree one Note field seems adequate. --MartyD 01:45, 8 March 2017 (UTC)
To be frank, I've always found that the "biographical" part falls largely outside our scope. IMHO we're a bibliographical tool and not a place for self-promotion or egoboost and so only the basics should be listed (the existing fields are sufficient for me). As a frequent moderator-of-new-contributors, the "biographical" pages are where there is the more "pruning" to do. Hauck 07:25, 8 March 2017 (UTC)
I agree that author-provided biographies can get out of hand, but first a little bit of background. Back in the 1990s, during the "ISFDB 1.0" era, i.e. pre-moderation, there was an "Author Note" field. It had all kinds of data, much of it not very good. It was a learning experience.
When we started work on ISFDB 2.0, the plan was to rely on Wikipedia to house biographical data. However, we quickly discovered that Wikipedia's "notability" criteria were too restrictive for our purposes: we have tens of thousands of authors who do not have (and never will have) a Wikipedia article. That's when the idea to use our Wiki to store author bios for less notable authors was born. Eventually it led to other issues, which is why the Wiki migration project was started a couple of years ago. Ahasuerus 15:32, 8 March 2017 (UTC)
P.S. The original 589 "author notes" from the 1990s and the early 2000s are still stored in the database although you can't access them. Many of them can be safely dropped, e.g. the following note about L. E. Modesitt, Jr.: "Author of over 20 fantasy and SF books, including the best selling Magic of Recluce series". However, there are a number of more useful ones, e.g. "Son of Fritz Leiber", "Married to SF author Charles Sheffield (1998-2002)", etc. Once the new field becomes available, I will create a cleanup report which will facilitate migrating the relevant ones. Ahasuerus 16:11, 8 March 2017 (UTC)
That is not very different from the other wiki pages cleanup - there is a lot of outdated information in them as well (and a lot of irrelevant one that is superseded by adding books and similar actions. Annie 16:14, 8 March 2017 (UTC)
True, although some older "author notes" are really strange. For example, here is the internal note for Edgar Pangborn: "None yet.<br><b>Sources:</b> TEoSF.". Make of it what you will :-)
Another difference is that we will be migrating three different types of author notes:
  • 589 database-resident notes mentioned above
  • a couple thousand Wiki-resident pages in the "Author" and "Author talk" namespaces
  • a couple thousand Wiki-resident pages in the "Bio" and "Bio talk" namespaces
Ahasuerus 16:29, 8 March 2017 (UTC)
That sounds like a fun project. I've seen stranger in some of the pages I cleaned - some of the 2008 notes were... interesting. :) Annie 16:39, 8 March 2017 (UTC)

(unindent) OK, "Author Note" is now available. Here is what A. V. Clarke's Summary page looks like now. It demonstrates the {{BREAK}} functionality, which can be used in a variety of ways. Ahasuerus 22:04, 8 March 2017 (UTC)

The changes make some pages look pretty stark, for example Bill Thompson's page. It would look better if the author name was in a larger font and bold, and "author:" removed, since it's redundant.--Rkihara 23:26, 8 March 2017 (UTC)
Is the starkness caused by the removal of the "Biography" and "Bibliographic Comments" lines for authors who do not have Wiki-based comments? As far as the author name goes, we currently display it twice: once in the header after the words "Summary Bibliography" and then again on the "Author:" line. It's done the same way on all bibliographic pages, including Title, Publication, Publisher, and Series pages. Ahasuerus 23:46, 8 March 2017 (UTC)
That's basically it. If no language is defined, there's just the author name in small font on one line. Maybe replacing the author line with the header, "Summary Bibliography: John Doe" in the large font would help the appearance? I think it's the white background drawing my attention away from the header on the black background.--Rkihara 00:48, 9 March 2017 (UTC)
Keep in mind that the Bio/Biblio links will be removed from all the other pages Real Soon Now (tm), so this issue will affect more than just the Summary page. For example, consider this series. Once the "Bibliographic Comments" line disappears, the header will consist of just one line. Ditto this publisher etc.
I am going to go ahead and remove these links from all series/magazine/publisher pages that do not have associated Wiki data. Once I do that, we will have a better idea of what the pages look like and will better position to decide how to change their layout. Ahasuerus 01:04, 9 March 2017 (UTC)
Done. I am going to create another section to discuss the layout implications. Ahasuerus 01:42, 9 March 2017 (UTC)
Done. Ahasuerus 01:54, 9 March 2017 (UTC)
And when are we getting a report? :) Annie 22:16, 8 March 2017 (UTC)
Real Soon (tm) :-) Ahasuerus 23:46, 8 March 2017 (UTC)

New cleanup report: "Author Notes to be Migrated from ISFDB 1.0"

"Author Notes to be Migrated from ISFDB 1.0" is now available in the main Cleanup Reports menu.

Please note that the data in the "Old Note" column comes from ISFDB 1.0 (1995-2004) and is not accessible in any other way. The data in the "New Note" column comes from the newly added "Author Note" field.

The report is unusual in a couple of ways:

  • The data is not generated nightly. Every time you access this report, you get a fresh copy.
  • There is no way to "ignore" author records or to remove them from the report. Once the useful tidbits (and there are some) have been migrated from the "Old Note" column to the "New Note" column, the old notes will be deleted from the database and this report will be deactivated.

Ahasuerus 00:58, 9 March 2017 (UTC)

Changes to database-Wiki links

The way Wiki links are displayed on the database side has been changed as part of the Wiki migration project. In the past the links were displayed regardless of whether a matching Wiki page existed. As of 15 minutes ago, the links are only displayed if a matching Wiki page exists. (Once the Wiki migration project has been completed, direct links to existing Wiki pages will be removed and we will use "Web pages" fields to link to the Wiki exclusively.)

Now that 99% of all links are gone, we need to decide whether the resulting layout looks good. For example, here is what the Summary page for an author without a language looks like. Here is an author with a language and no other data. Here is a publisher with a single publication. Do the pages look too sparse/stark? Ahasuerus 01:53, 9 March 2017 (UTC)

I like the wiki links being hidden, but I don't like hiding the empty database fields. I think that could discourages editing for a newer user who may have information, but isn't even aware that the fields are available. It also makes the page look pretty sparse. -- JLaTondre (talk) 02:16, 9 March 2017 (UTC)
Contra JLaTondre, I think it would look terrible to have a list of fields reading "unknown" on every page; and I don't think that it would encourage that much extra editing.
However, the name of the author/publisher/whatever should definitely be larger or highlighted so it stands out from the other elements of the page. --Vasha 02:21, 9 March 2017 (UTC)
I agree with JLaTondre's remarks. I don't like this "new" look (that's personal, of course) and it doesn't give a hint of all what's possible for our contributors. As I understand it, the "ISFDB-wiki" links will be among the web links listed, so the understanding that these specific links are "ours" and so are editable may be a bit complicated for the average user. Hauck 07:16, 9 March 2017 (UTC)
Most of the existing wiki-pages will be converted to the DB field - the only ones that will remain as wiki pages will be the ones that are too long or complicated to move (and I suspect even some of them will get moved sooner or later). Annie 07:32, 9 March 2017 (UTC)
Let me make sure that we are all on the same page. Bibliographic (Summary, Series, Publication, etc) pages display two different types of data. There are "database fields" and "Wiki links". What JLaTondre is proposing is that we display empty database fields. Here is what the previously linked Summary page would look like if we were to display the empty fields:
Author: Kathleen Notestine
Transliterated Name: 
Legal Name: 
Transliterated Legal Name: 
Birthplace: 
Birthdate: 
Deathdate: 
Language: 
Email Address:
Webpages:
Note:
It would be easy to implement, but I don't think it would look better than what we have now. Ahasuerus 14:24, 9 March 2017 (UTC)
As I mentioned previously, the missing fields make it look too stark, somehow deprecated. I can't put into words exactly what it is that bothers me about it. Removing "author:," centering the name at the top of the field in a larger, bold font size, and removing the header "Summary Bibliography:" would do a lot to improve the appearance, IMHO.--Rkihara 16:33, 9 March 2017 (UTC)
One thing that comes to mind is that we use HTML bullet ("unordered") lists to offset "Author", "Language", etc fields. A bullet list consisting of one bullet looks odd. In addition, the offset makes these fields align with the contents areas of other sections like "Short Fiction" or "Poems", which also isn't quite right.
There are a couple of things that we could do about this. First, we could move the "Author Record #" field, which is currently right-justified, to the main bullet list immediately under the Author Name field. That way the list will always have at least 2 bullets. Second, we could move the list to the left so that it would be aligned with section names rather than with section contents. Ahasuerus 17:01, 9 March 2017 (UTC)
That would help. After thinking about it a bit, I think that without missing field headings, what remains doesn't give you anything to focus on. The white background draws my eye in, then it jumps to the line below Showing all translations (can be changed in My Preferences) because it is bold and colored blue at the end. Then my eye wanders around to the author: line above, but it keeps getting distracted by the line below. It's like an outline with plain headers and subs in bold. That's why I think the author should be centered in bold, large font. I have a friend who is a graphics designer and deals with this kind of problem in his work. I'll ask him for his opinion.--Rkihara 17:24, 9 March 2017 (UTC)
Why not change "Author:" to "Canonical Name:" like in the edit form (we already have the author name above in the "Summary Bibliography:" and put the list of works, etc. in a different "Works" (or similarly named) box much like the "Contents" box of publications (which have the publication data in one box and the "Contents", etc. in a separate box below it). It seems to me, this type of delineation would also be useful in other places like title displays (where title data could be in a separate box from the list of publications of such title, etc.). This would help underscore which data belongs to the author record and which were just associated with it. Uzume 19:06, 12 March 2017 (UTC)
I am afraid I don't understand what you are proposing. Ahasuerus 21:21, 12 March 2017 (UTC)
I am proposing to put the list of credited titles on author pages into a separate box clearly delineating which is author data and which is author associated data. For example P348699 currently displays the publication data in a box and the associated contents in a separate "Contents" box below it. Do the same thing with author's works in a "Works" box on author pages (and relabel "Author:" as "Canonical Name:" as it is labelled in the author edit form). I am not suggesting (re)adding empty fields to the display (though that could be done as well). To get technical, right now all author data and associated author data on author pages are in <div id="main">. Publication data are in <div id="content"> and associated publication data are in <div id="ContentBox"> (and <div id="VerificationBox">, etc.). With proper styling, this will clearly separate what can be edited in the author record and what cannot directly be edited there. Uzume 21:50, 12 March 2017 (UTC)
Oh, I see. Thanks for the explanation! Come to think of it, we already have a line ("<hr>") which is supposed to separate author-specific data from the author's titles, e.g. see Poul Anderson's Summary page. However, it is only displayed if we have an author image on file. Let me see if I can make it appear for all authors... Ahasuerus 22:19, 12 March 2017 (UTC)

(unindent) The software has been modified to display a thin gray line between the author-specific data and the list of author's titles on all pages. In the past the line was only displayed if the author had an image on file. Ahasuerus 22:33, 12 March 2017 (UTC)

Yes, something like that (though I still like the separate box delineation better; I notice a similar hr rule is used for non-genre). That does make Kathleen Notestine and A. Testa (the sample authors mentioned in this thread) look a little better. I would still recommend changing the label "Author:" to "Canonical Name:" to match the edit form. Also a similar delineation would be good for publishers (as mention above for Barzakh) and as I mentioned for titles too. Uzume 23:36, 12 March 2017 (UTC)
Well, "Canonical Name" is, as Help states, is "the name under which a particular author's bibliography is organized". For pseudonyms, the displayed name is not the same as the author's canonical name. I agree that synchronizing the field label on the Summary page with the field name on the "Author Editor" page would be good, but I am thinking that we may want to change the latter rather than the former. Ahasuerus 00:12, 13 March 2017 (UTC)
I agree that makes that wording confusing. Perhaps "Author Name:" is better and the Help could be updated to define "Author Name" as "the name under which a title/work is credited" and "Canonical Author Name" could remain as "the name under which a particular author's bibliography is organized" (even though we do not have anything where that is used except perhaps in pseudonyms and variants where we could distinguish "Credited Author Name" and "Canonical Author Name"). Uzume 01:36, 13 March 2017 (UTC)

The Illusion Seekers

Something does look weird here. The Illusion Seekers. Why is this story credited to unknown and then varianted for the proper author? If the reason is because P. F. Costello is a pseudonym then shouldn't instead it go to whoever is pseudonymed for? As it is - it is extremely hard to get a listing for "P. F. Costello" (the show all titles format is awkward and unwieldy for things like that) and this specific story only shows on the unknown list - which is unmanageable.

Is this by design? If so, why? Annie 18:45, 9 March 2017 (UTC)

One more like that The Monarch of Mars - seems to be my day to stumble on them. Some background will really help. Annie 18:53, 9 March 2017 (UTC)
"P. F. Costello" and "John Pollard" were house names owned by Ziff-Davis. Z-D used a lot of house names, most famously "Alexander Blade". Sometimes we know the actual author, other times we don't. We start by creating a title record with the house name listed as the author. Then we variant it to the actual author (if known) or to "unknown" (if unknown.) If and when we discover the actual author, we update the canonical title.
So yes, it's done by design. However, I agree that the format used by "view all titles by this pseudonym" is not perfect. Suggestions welcome! Ahasuerus 19:19, 9 March 2017 (UTC)
Someone coming in the site for the first time will be very very confused. Knowing that these are Z-D house names, can't we have a new author for all Z-D house names and not alias under unknown - which looks just weird... At least this way you can see that it went there because it is a House name. Annie 19:25, 9 March 2017 (UTC)
Well, it's a thought, but let's consider the implications. What about Daisy Meadows, a 21st century house name with hundreds of books to "her" name?
I guess one way to approach this issue would be to create a new author record for "undisclosed house name" and variant all undisclosed titles by "Alexander Blade", "Daisy Meadows", etc to it. Ahasuerus 22:02, 9 March 2017 (UTC)
But Daisy Meadows is not pseudonymed under unknown. I do not have issues with her display - all she needs is an author note (already there - just need moving) and all is set. It is the ones that ended up into unknown (because some books are really unclear) that are the problem. And yes - having a "undisclosed house name" will be preferable to the current "unknown". Annie 22:16, 9 March 2017 (UTC)
Keep in mind that pseudonyms and VTs are handled separately. There are many Daisy Meadows titles whose author we do not know -- e.g. this one -- and whose canonical author is currently "unknown". In this respect "her" bibliography is similar to what has been done with the Ziff-Davis pseudonyms. The difference is that there is no pseudonymous relationship between "Daisy Meadows" and "unknown".
We could certainly remove the "unknown" pseudonym from the Ziff-Davis pseudonyms to match what has been done with Daisy Meadows, but it could have unfortunate side effects. For example, consider John Dexter, a house name used by Greenleaf Publications in the 1960s. At the moment "his" Summary page reads "Pseudonym. See: Marion Zimmer Bradley", which makes it appear like it was an MZB pseudonym. A few weeks ago I received an e-mail from an MIT librarian who wanted to add another "John Dexter" title to the ISFDB and to his catalog. It was our data that had suggested to him that the book was written by MZB.
Thankfully, Robert Silverberg was another SF author who wrote as "John Dexter". Once we add his Dexter titles and link everything, it will become clear that "John Dexter" was a house name. However, we are not so lucky in the case of "John Pollard". If we were to remove "unknown" from the list of pseudonyms, it would look like it was a personal pseudonym used by Howard Browne. I guess it will be less of a concern now that we have the Author Note field. Ahasuerus 22:50, 9 March 2017 (UTC)
I understand how we ended up here. It is just a bit weird the first time you see one of those. Thanks for all the background! Annie 23:01, 9 March 2017 (UTC)
We could perhaps implement a new feature where an author is linked to a publisher instead of another author (as is done for pseudonyms; I always thought the design of keeping author and publisher as separate "authorities" was a dangerous one but it was made a long time ago). Then such house names could be unlinked from "unknown" (but known titles could still linked to the proper authors where available) and they could be shown on publisher pages and vice versa. Uzume 19:17, 12 March 2017 (UTC)
I am afraid I don't understand what it means to have an author "linked to a publisher instead of another author". Ahasuerus 21:17, 12 March 2017 (UTC)
Basically a pseudonym of publisher index/pointer (vs. of another author) field (or add a type field so you know which table to index into and reuse the existing index field). Uzume 21:34, 12 March 2017 (UTC)
Before we discuss the implementation details, I would like to understand what it means to have an author "linked to a publisher" functionally. Ahasuerus 21:46, 12 March 2017 (UTC)
It means it is a pseudonym associated with a publisher as house names commonly are. Is there something I am missing (I thought that was what we were talking about in this thread)? Uzume 23:26, 12 March 2017 (UTC)
Authors and publishers are so different that I am having trouble grasping how they could be linked. It's not like an author, a pseudonym or even a house name has to be limited to one publisher. For example, Alexander Blade was a Ziff-Davis pseudonym, but later on "Alexander Blade" stories appeared elsewhere, e.g. in Palmer Publications, Inc. magazines owned and edited by Ray Palmer and in Greenleaf Publishing Company magazines owned and edited by William L. Hamling. Once the stories fell into public domain, they were reprinted by project Gutenberg and a variety of PD publishers.
Then it gets worse. Much worse. To quote SFE3:
  • The use of house names was at its most extreme and most devious at the Ziff-Davis magazines edited by Raymond A Palmer. They have never been satisfactorily unravelled and all records are now believed lost. In these cases most pseudonyms which became house names were originally used by an individual writer but then became farmed out to others, adding to the confusion. Alexander Blade, for example, the most common house name, was originally used by David Vern (see David V Reed) but soon appeared on work by almost all the regular Ziff-Davis contributors, including the serial "The Eye of the World" (June-July 1949 Fantastic Adventures) written by Don Wilcox. Both P F Costello and Gerald Vance were originally the personal pseudonyms of William P McGivern but were later used by Chester H Geier, Rog Phillips and others. E K Jarvis was the alias of Robert Moore Williams until used by others in the 1950s. S M Tenneshaw was the alias of William L Hamling, and he took it with him when he began Imagination, but it has been used by several other writers. Ivar Jorgensen, sometimes misspelled Jorgenson, was originally the pen name of Paul W Fairman, but in later years it was used by Howard Browne, Harlan Ellison, Henry Slesar and Robert Silverberg, the latter usually in collaboration with Randall Garrett. Silverberg once quipped that he started out enjoying the work of Ivar Jorgensen and grew up to be him. Most of the work under these and other Ziff-Davis house names has yet to be accurately attributed. It is believed that Leroy Yerxa wrote almost the entire December 1943 issue of Fantastic Adventures with stories under house names except for one, Morris J Steele, which was usually the pen name of Raymond A Palmer himself but was here used by Berkeley Livingston.
Ahasuerus 00:54, 13 March 2017 (UTC)

8 cleanup reports for authors

8 author-specific cleanup reports have been added for "Bio" and "Author" pages. The rules for their migration are the same as for all the other cleanup reports. The data will become available tomorrow morning. As of last Saturday, we had 1,698 "Author" pages, 2,147 "Bio" pages, 30 Talk pages, and 156 orphan pages. Ahasuerus 21:59, 9 March 2017 (UTC)

Never had so much fun with clean-up rapports!...Dirk P Broer 21:47, 10 March 2017 (UTC)
For anyone that works these for the first time, don't forget after you move the data to add one of the Deletion templates to the wiki page so that the moderators can remove it. The templates are here Annie 21:50, 10 March 2017 (UTC)
Thanks for the info!--Dirk P Broer 22:44, 10 March 2017 (UTC)
For Fanzines and Magazines, We do not delete the wiki pages until at least stub pages are created for ALL issues references in the wiki (especially when all issues are eligible as it is with the fanzines). So I had to reverse your request for a delete on the fanzine. If you feel like creating the stubs - go ahead. But otherwise we lose the information for existing issues that someone collected. Annie 22:54, 10 March 2017 (UTC)
I created a similar category mechanism for marking wiki spam (see Category:Spam) back in March 9, 2011 and I had it linked to the top of the Moderator noticeboard (I am not sure when it was removed; sadly the wiki history gets purged so though I can look at the history of edits I cannot tell what was actually edited when—just the edit comments survive). This mechanism might be partially superseded, however by the newer deletion mechanism more recently created on October 30, 2016 (see ISFDB:Help desk/archives/archive 25#Mixed up series). That said, it is possible to have multiple deletion candidate categories/subcategories for different kinds of deletions requests. I just now added my category to Category:Deletion candidates (directly without using a template). I am not sure if this will be interpreted as a deletion request or allow it to be used as a sort of subcategory for a different type of deletion requests. Uzume 20:00, 12 March 2017 (UTC)

Templates in Notes?

Now that "Author Note" is live, it occurs to me that we may want to add support for Wiki-style templates. For example, if I type {{A|Vernor Vinge}} here, everyone will see Vernor Vinge and be able to follow the link to the ISFDB record. We can add the functionality to the database side if it is deemed desirable. Ditto for publishers. Ahasuerus 23:17, 9 March 2017 (UTC)

That would be very nice :) Annie 23:43, 9 March 2017 (UTC)
OK, template support has been added. At this point the software supports the following three templates in the Note fields:
  • "A", which does what I mentioned earlier, i.e. {{A|Vernor Vinge}} will display a link to Vernor Vinge's Summary page
  • "OCLC", which links to OCLC, e.g. "{{OCLC|880913738}}" will display a link to this OCLC record
  • "LCCN", which links to the Library of Congress, e.g. "{{LCCN|89040470}}" will display a link to this LoC record
We can easily add support for other sites. The only constrains are:
  • They need to have stable URLs
  • They need to allow linking by record number
We may be able to accommodate more complex cases as well, but they will require additional research. ASINs are next on my list. Additional suggestions welcome!
P.S. In case anyone is wondering, the "third party identifiers" that we discussed a couple of weeks ago will use the same logic when implemented. The difference is that using these templates in Note fields allows more flexibility, e.g. "corrected page count from {{OCLC|880913738}}, artist from {{OCLC|880914329}}". On the other hand, a separate field for third party identifiers will support robots like Fixer. Ahasuerus 19:02, 10 March 2017 (UTC)
Can we add a template for Publishers for the Publishers "Notes" field? That will allow the links that connect different publishers (when they move as imprints between publishers and so on) to be spelled in an easier way. Annie 20:01, 10 March 2017 (UTC)
When you have a chance, If you use Scott Thomas here, you get to the page. If you use the same in a note, it gets elsewhere (see my submission here for an example). Is that expected? How can I point to the correct one in this case? Annie 20:16, 10 March 2017 (UTC)
Intra-ISFDB linking templates use the Regular Search logic. If a search string matches only one author name, then the user is redirected to that author's Summary page. If, on the other hand, a search string like "Scott Thomas" matches the names of multiple authors, then the "Search Results" page is displayed.
Granted, it would be easy to change the intra-ISFDB templates to use the Advanced Search logic which supports the "exact" functionality, but there is a catch. An Advanced Search always displays the Advanced Search Results page even it finds only one matching record. That's even worse than the current behavior.
What we need is new logic that would combine the "exact" functionality of the Advanced Search and the automatic redirect logic of the Regular Search. Alternatively, we could change the Advanced Search logic to use the same redirect functionality that Regular Search uses, but I think some editors objected the last time this issue came up.
So expected behavior. No problem - I was not sure if you are aware of it happening. How about allowing to link by number in such cases? Or will that be too much work and not worth it (standard links work after all so one can use that if they wish)? Annie 20:45, 10 March 2017 (UTC)
Let me see if I can tweak the Regular Search logic to accept a new parameter to support "exact" searches... Ahasuerus 22:02, 10 March 2017 (UTC)
Done. "Publisher" has been added as well. Ahasuerus 23:36, 10 March 2017 (UTC)
As far as publishers go, I originally implemented a "P" template, but removed it before installing the patch. The linking problem was even worse: a Regular Search on "Tor" finds hundreds of matching publisher names. Ahasuerus 20:33, 10 March 2017 (UTC)
What is the advantage of adding third party identifiers if we are already implementing them as markup in notes? It seems redundant to me and the only difference is whether they adhere to a structured data relational model or are semi-structured data like those of document-oriented databases (or object-relational databases). Both are searchable but semi-structured data may be better supported with a different query language (e.g., XQuery, XPath or SPARQL, etc. vs. SQL). The one downside to this new markup is we are effectively creating our own template processor, macro or lightweight markup language (which can be nontrivial) and we might be better served to use an existing one (which could solve other problems too like restricting the types of markup available instead of making raw HTML available which can be unsafe e.g., if I can get it past moderation which might not be hard since it has no real display, I could add a script tag with JavaScript to note records, etc.). I would recommend adopting an extended (with our identifier markup extensions, etc.) version of reStructuredText (good support in Python and it is becoming a ubiquitous de facto standard in places like GitHub, etc.). I realize updating all the existing notes would be a large task but it could eventually prove worthwhile (and we could implement an interim solution supporting both until they are all converted). Uzume 20:57, 12 March 2017 (UTC)
As I mentioned earlier, "the difference is that using these templates in Note fields allows more flexibility, e.g. "corrected page count from {{OCLC|880913738}}, artist from {{OCLC|880914329}}". On the other hand, a separate field for third party identifiers will support robots like Fixer."
Fixer's internal database contains hundreds of thousands of ISBN-less records. Most of them have a third party identifier like an ASIN, an LCCN or an OCLC number. Fixer can't submit them because he has no way of knowing if they are already in the ISFDB database. Once we add support for third party identifiers, it will become possible for Fixer to start submitting them.
As far as inserting malicious JavaScript into Notes goes, does adding support for templates make a difference given that we already support a subset of HTML in notes?
Re: using a third party template processing tool like "reStructuredText", what would be the advantages? As far as I can tell, the software already supports the limited template functionality that we need. Ahasuerus 21:13, 12 March 2017 (UTC)
Fixer (and other bots) can be made to use the new template markup for identifiers when making submissions and can query such via a note search (perhaps with an added API for bot ease of use). Search could be adapted to handle searching for so marked identifiers in notes (to ease searching for, e.g., {{OCLC|###}}, etc.). This could be not much different than the current ISBN search API that I believe Fixer already uses to check if records already exist (with ISBN X, etc.). Uzume 21:30, 12 March 2017 (UTC)
I don't think querying the live database would be viable since there are hundreds of thousands of ISBN-less records in Fixer's internal database. Especially considering how long Publication Notes searches take. A better solution would be to take advantage of the fact that Fixer has access to a modified version of a recent ISFDB backup. It would be possible to pre-parse all publication notes and build a table of ASINs, LCCNs, etc that are referenced in Notes.
In any event, there is nothing preventing editors from adding notes like "Not to be confused with {{OCLC|123456}}, a German translation of this book", which would mislead Fixer into thinking that we already have OCLC 123456 on file.
As a general rule, structured data is better than semi-structured notes, which should be used sparingly. Once we add support for third party identifiers, I expect them to be used in the majority of cases with templates only used when more specific information needs to be recorded in notes. Ahasuerus 22:02, 12 March 2017 (UTC)
I am sure the designers of so called NoSQL databases and RDF data models (as used by Dublin Core, BIBFRAME, RDA, ISBD, and other recent bibliographic models) would disagree with statements like "[a]s a general rule, structured data is better[...]" (many applications are moving away from relational fully structured data and towards other models like directed graph models, e.g., Semantic MediaWiki, Wikidata using Wikibase, etc.). That said, I agree it might not be the most applicable solution for our "note" needs (which as you mention might contain ancillary reference comments about other things). Uzume 22:47, 12 March 2017 (UTC)

(unindent) The following templates have been added:

  • ASIN: Amazon.com
  • ASIN-UK: Amazon UK
  • BL: British Library
  • BnF (also "BNF"): Bibliothèque nationale de France
  • DNB: Deutsche Nationalbibliothek

Ahasuerus 20:41, 10 March 2017 (UTC)

"Publisher" has been implemented. Ahasuerus 23:36, 10 March 2017 (UTC)
It would be awesome if a list of these templates displayed when editing, or a link to a help page that lists all of them with examples of how to use each one? ···日本穣 · 投稿 · Talk to Nihonjoe 00:46, 11 March 2017 (UTC)
Good point! I have updated the Edit pages with a link to the list of supported templates. How does it look? Ahasuerus 01:38, 11 March 2017 (UTC)
Looks good. Are the template case insensitive so {{OCLC|880913738}} and {{oclc|880913738}} produce the same thing? ···日本穣 · 投稿 · Talk to Nihonjoe 07:49, 11 March 2017 (UTC)
At the moment the templates are case sensitive, but I guess there is no reason for them to be. Let me see what I can do... Ahasuerus 16:42, 11 March 2017 (UTC)
OK, all linking templates are now case-insensitive. "OCLC", "oclc" and "oClC" should generate the same links. Ahasuerus 19:53, 11 March 2017 (UTC)
Awesome. That should nip that potential issue in the bud. ···日本穣 · 投稿 · Talk to Nihonjoe 23:20, 11 March 2017 (UTC)
Shouldn't other templates like {{BREAK}} be case insensitive too? --Vasha 07:24, 12 March 2017 (UTC)
It's probably better to make it case-insensitive, although {{BREAK}} is a special template with special limitations. For example, you can only have one in a note. Ahasuerus 15:34, 12 March 2017 (UTC)

Author Note in Advanced Author Search

Advanced Author Search has been updated to support Author Notes. Ahasuerus 23:03, 10 March 2017 (UTC)

Can I search for identifiers in our new note markup (FormatNote templates)? If so, why bother with implementing third party identifiers separately? (see my comments above in #Templates in Notes?) Uzume 21:09, 12 March 2017 (UTC)
Answered above. Ahasuerus 21:14, 12 March 2017 (UTC)

Data deletion policy -- proposed language

Starting a new section since the "Author Note" field is now available and the last discussion is kind of buried anyway.

Based on what we discussed earlier and the original Rules and Standards discussion, I would like to propose the following data deletion language:

  • If a living author (or his/her authorized representative) requests that the ISFDB remove the author's detailed biographical information, the ISFDB will comply after confirming the requester's identity. The ISFDB will remove as much biographical data as needed in order to accommodate legitimate privacy concerns while preserving, to the extent possible, the work of the editors who have compiled the data. A note will be added to the author's record stating that some information has been redacted upon request explaining what type of information has been removed and why.

The idea here is to express our intent and codify our de facto policy while giving editors some leeway when dealing with specific requests. Ahasuerus 00:23, 11 March 2017 (UTC)

I am not sure if we need to add it in the language or just in the rules but I think the notice for the removal should list what had been removed (for example removing a birth day when a birth place/country was never entered should be distinguishable from a request to remove both - so that a different editor knows which fields should not be filled in). Alternatively, we can add a new default value of "withheld" valid in each field (including the date ones) - which will serve the same purpose. Annie 00:40, 11 March 2017 (UTC)
I like this, and I agree with Annie. Otherwise, we'll either have people with good intentions filling in "missing" information or people unwilling to fill in unredacted missing information. --MartyD 12:44, 11 March 2017 (UTC)
OK, I have adjusted the proposed language above. I don't think "withheld" would work in his case because it doesn't support birth dates like "1943-00-00" where the "00-00" part has been redacted. Ahasuerus 20:02, 11 March 2017 (UTC)
Yes, I meant the notion of identifying what was redacted, not "withheld". I think your revised wording does the trick quite nicely. --MartyD 11:52, 12 March 2017 (UTC)
I believe we should also codify that such data can be added back upon the requester's death (it ceases to be a privacy issue then; at what point does such such information eventually become public domain without restriction?; there should be some defined end to the redaction). That way such redacting comments can be removed while updating so marked records as long as I add death dates as well. Uzume 21:19, 12 March 2017 (UTC)
True, but I hope it is covered by the "living author" language in the proposed text. Ahasuerus 21:22, 12 March 2017 (UTC)
Fair enough. I often try to kill off authors from Oldest Living Authors (by researching their deaths and adding the data). Uzume 22:05, 12 March 2017 (UTC)
Me too!--Dirk P Broer 23:14, 13 March 2017 (UTC)

Outcome: ISFDB:Policy has been updated with the proposed language. The affected record has been updated and the author has been notified. Ahasuerus 14:54, 15 March 2017 (UTC)

Change for handling suffixes in author legal names

There has been a small change to Template:AuthorFields:LegalName to allow for capturing of suffixes that are part of an author's legal name, when we have evidence that its inclusion in the legal name is actually the case. It should be added to the end of the normal legal name entry separated by a comma (Lastname, Firstname Middlenames, Suffix). --MartyD 13:36, 12 March 2017 (UTC)

Additional cleanup reports made available to all editors

Additional cleanup reports have been made available to all editors. More to come. Ahasuerus 18:54, 12 March 2017 (UTC)

  • Even more cleanup reports have been made available to all editors. In addition, a display bug which affected non-moderators has been fixed. Ahasuerus 14:45, 13 March 2017 (UTC)

Magazine dates cleanup

Here's an idea for a cleanup project: finding magazines whose date is incorrectly set to the exact date of publication rather than the cover date. Checking all magazines with a day value other than 00 wouldn't catch all problems but would be an improvement. (Manageable if you could ignore at a stroke all issues of a magazine that correctly has days.) Once found, fixing the problem would be a bit of a pain because of having to correct the dates of covers and contents too. --Vasha 20:17, 12 March 2017 (UTC)

Why not have the records reflex the exact date of publication when known (some magazines have issues specified by year and month and then complete publication dates also printed within them and sometimes these are intentionally different; some magazines do not associate year and month with issues and we can only use exact date of publication)? Uzume 22:01, 12 March 2017 (UTC)
Well, it's a very longstanding bibliographic practice here and elsewhere to give the date of a magazine as the cover date; the Hugo awards goes with that too. Usually the table of contents, masthead, or cover will say "May" (ergo, date of YYYY-05-00) or "May-June" or "Summer". Only if such a date is found nowhere in the magazine should the publication date be used. And if the cover date is something like "Summer" or "First Quarter" then the format of the date is YYYY-00-00. That isn't controversial, is it? --Vasha 23:22, 12 March 2017 (UTC)
Note: we should probably change Help:Screen:EditTitle#Year to "Date" to match the edit form. Uzume 22:01, 12 March 2017 (UTC)
Help updated; thanks.
Re: the original proposal, it would be nice to see a few examples of incorrectly set dates. Once I have a better idea of the discrepancies involved, I could run a couple of database queries. That should help us determine how many mismatches we are looking at and what types of cases are common. Ahasuerus 22:43, 12 March 2017 (UTC)
re: exhibit A: Clarkesworld (print issues) (Are all those really printed in October 2012?) and exhibit B: Planet Stories: Adventure House Reprints. Uzume 22:56, 12 March 2017 (UTC)
I was actually referring to this where the date of the January 2017 print issue of Clarkesworld is given as 2017-01-12 (the ebook is correctly 2017-01-00). --Vasha 23:22, 12 March 2017 (UTC)
For reference, Template:PublicationFields:Date used at Help:Screen:EditPub#Date currently states books should use stated publication date (where available) but magazines should use stated cover date (even if there is a stated publication date). This seems strange for reprints and such things. It should be noted, magazines cannot really use editorial title dates for anything useful here because typically magazine pubs are grouped by year of editorial (to force the situation for multiple publication runs of a magazine like reprints and ebooks, etc. we create another series of editorial titles to define "another magazine" run; this is why I prefer pub series be used for magazines but we do not currently do that). Uzume 00:02, 13 March 2017 (UTC)

Yearling Newbery

We have three 'yearling newbery' publishers [1]

  • Dell Yearling Newbery
  • Yearling / Yearling Newbery
  • Yearling Newbery

with 9, 1, and 1 publications. Perhaps Yearling Newbery be considered a publication series of the Yearling imprint, or various Yearling imprints, which may or may not have multiple parent publishers. At the moment none of the several 'yearling' publishers, with or without 'newbery' in its name, has a Yearling Newbery publication series. If these were 100 years older I would make Yearling Newbery a publication series, presumably across multiple renditions of Yearling, and eliminate the three publishers listed above.

Three of the publications by these publishers are primary verified including one by me, transient. --Pwendt|talk 16:30, 13 March 2017 (UTC)

The Strugatsky's 'Midday' future history

There's a COLLECTION that's bound to be a variant of this 'novel'. In the light of the afterword, the listing in 'Bibliographisches Lexikon der utopisch-phantastischen Literatur' and the handling of analogous titles like this one it really seems more apt to see this as a COLLECTION: the various 'parts' were and are published independently; there also is virtually no fix-up involved, the stories only are set in various times. Any comments? Stonecreek 17:22, 13 March 2017 (UTC)

Checking FantLab, I see that 4 of the stories were originally published as standalones. The names of the characters were reportedly changed when the stories were added to this book in 1967. So I guess it's a, um, "limited fix-up"? Ahasuerus 17:56, 13 March 2017 (UTC)
I looks like it was a COLLECTION initially but was later often referred to as a NOVEL, even by the authors themselves, for e.g. in Boris Strugatzki's remarks in Werkausgabe #5 which I just skimmed through quickly. He begins the text talking about the novel ("Roman"). Later he writes there that they initially wanted it to be a novel with a consistent story, but then gave up on that and regarded it as a collection of more or less independent, mostly unrelated stories written at different times and on different occasions (page 833). Moreover, there's also PDF with a complete bibliography of Strugatzki's publications in German at Golkonda's website, and the information on page 26 supports that: it's initially listed as a collection there (or "Episodenroman") but later also as a novel. fantlab.ru also lists it as a collection ("Полдень, XXII век (Возвращение) — это сборник коротких рассказов" => "a collection of short stories" if Google Translate is correct). However, finally, on page 832 in the aforementioned remarks in Werkausgabe #5 there's a hint that the Strugatzkis revised the texts for a Russian edition of complete works from 2001. These remarks are quite extensive and I don't have the time right now to read it all, but it might turn out from these remarks that there's a COLLECTION (early versions) and a revised NOVEL (later versions). Jens Hitspacebar 18:33, 13 March 2017 (UTC)
Based on the available information it seems all publications in our database are COLLECTIONs and I'll change them accordingly. Stonecreek 19:54, 13 March 2017 (UTC)

Verification sources are now available as templates

Our standard verification sources (see Reference:Verification_Sources) are now available as templates. If you enter {{Tuck}}, {{Locus1}}, etc anywhere in notes, the displayed version will show the complete name of the source. It will also be linked to the corresponding ISFDB (or non-ISFDB, as the case may be) Web page. Ahasuerus 18:43, 13 March 2017 (UTC)

Is there a template for {{SFE}}? Chavey 21:44, 15 March 2017 (UTC)
"Clute/Nicholls", "Clute/Grant" and "SFE3" are currently available. Ahasuerus 21:48, 15 March 2017 (UTC)
Thanks. I didn't think to try the "3". Chavey 03:08, 16 March 2017 (UTC)

Displaying images for mixed INTERIORART/COVERART titles

Earlier today I was looking for something else when I stumbled upon the following FR:

  • Currently, when a coverart title is displayed the thumbnails of the coverart images for the parent title and its variants are displayed at the bottom of the title record. Occasionally, a coverart title is a variant of an interiorart title (e.g. interior art was first publication). I'd like to request that thumbnails for the variant coverart titles be displayed on their parent's title page just as they would be if their parent were coverart.

As a point of reference, we have 326 COVERART titles varianted to INTERIORART titles. It would be fairly easy to do what the FR requests. However, first let's make sure that we understand what the change would entail.

Consider this COVERART title, Black Gods and Scarlet Dreams by Michelangelo Merisi da Caravaggio. The linked Title page displays the cover scan which we have on file for the pub associated with this COVERART title. So far so good.

This COVERART title is varianted to the same artist's Medusa, an INTERIORART title. Since it's not a COVERART title, we display either the scans of the covers of the 4 pubs in which this title appears or, depending on the user's preferences, a link to the "All Covers" page.

If we were to change the display logic as per the FR, the Title page for this INTERIORART titles would display the cover scan associated with the COVERART titles -- if and only if the COVERART title is associated with any cover scans. At the same time we would the lose the ability to view the covers of the books in which this INTERIORART can be found.

Is this a desirable trade-off? Ahasuerus 21:57, 13 March 2017 (UTC)

No. Hauck 07:51, 14 March 2017 (UTC)
Can we have both versions (also known as "have your cake and eat it")? Leave the regular logic in place but add a new link for the special case where interior art is the parent to allow the new logic to be shown? Annie 22:56, 15 March 2017 (UTC)

Should entirety of Diabolical Plots be catalogued?

Diabolical Plots pays pro rates, so I've been cataloguing its fiction even though it's web only. I just noticed that the site describes itself like so: "Diabolical Plots is a Sci-fi/Fantasy zine that covers virtually every media related to the genre from books to movies to video games. This site also features regular content related to the craft of writing." Now, despite that self-designation as a zine, it sure looks more bloggish, not organized into issues and with a lot of chatty posts by the editor. I think it's another one to be treated the same way as Tor.com, fiction only-- just because it calls itself a zine doesn't mean we have to treat it as one. What do you folks say? --Vasha 23:08, 13 March 2017 (UTC)

FR 980 - Add checkboxes for importing/exporting COVERART and INTERIORART titles

As per FR 980, the Web pages that let editors import and export contents have been modified to allow skipping COVERART and/or INTERIORART titles. If you encounter any problems, please let me know. Ahasuerus 00:15, 14 March 2017 (UTC)

Title/publication page layout synchronized

The other day Uzume pointed out that the layout of the Publication page had gotten out of sync with the layout of the other bibliographic pages. The former had separate boxes for publication metadata, "Contents" and "Verification Status". The rest of the bibliographic pages didn't.

I have modified the Title page to use separate boxes for the following sections:

  • Title Metadata
  • Variant Titles and Serialization
  • Awards
  • Publications
  • Reviews
  • Bibliographic Warnings

Glory Road can serve as an example of a title which has all of the listed sections. Its variant Het Pad van Roem is an example of a minimalistic title. Please note that various bullet lists and tables have been adjusted to align vertically across boxes. As always, you may need to force a page refresh (Control-F5 in most browsers) to see the new layout.

This patch is experimental. It's hard to explain visual design in words, so I figured it would be easier to tweak the layout of one page and gather feedback. Once we agree on the optimal layout, I will implement the changes across all bibliographic pages. Ahasuerus 20:08, 14 March 2017 (UTC)

I think I like it. Have to look some more to be sure. --Vasha 20:11, 14 March 2017 (UTC)
I like it. It makes it easier to see where each section begins and ends. ···日本穣 · 投稿 · Talk to Nihonjoe 20:57, 14 March 2017 (UTC)
It pushes the publications way too low on the page even for cases where there aren't that many variants or awards... Can we add links in the top section that allows you to jump to a lower one? If this is doable, then I think it will look better. Also - don't "Bibliographic Warnings" belong only to the publications while reviews are tied to titles. It almost feels like the warnings should be moved to a separate page (similar to the BREAK for notes) so that a casual user do not see them... Annie 21:12, 14 March 2017 (UTC)
And one more thing. Looking at Weirdbook #33, it is hiding all empty sections except for that last "Bibliographic Warnings" one - which is empty but still shown. Shouldn't that be treated consistently? Annie 21:17, 14 March 2017 (UTC)
You have a point about publications being low. But I think the variant titles should still be above the publications anyhow. Makes more sense that way. --Vasha 21:28, 14 March 2017 (UTC)
Thus my question of having in-page links so people can jump into what they are interested. I never proposed to change the order. :) Annie 21:43, 14 March 2017 (UTC)
Some minor glitches (all seen on Firefox 52/Ubuntu):
  • the right bracket of the "[edit]" link next to the ID is overlapping with the border on the right. It needs a small margin on the right.
  • the "Awards" box needs a small margin at the bottom (like e.g. the "Variant Titles" already has) because right now it is almost touching the lower border —The preceding unsigned comment was added by Hitspacebar (talkcontribs) .
Also happening with Chrome. All boxes could use a right side margin (see pub table and pub pictures). -- JLaTondre (talk) 22:37, 14 March 2017 (UTC)
Regarding Annie's comment: It would be nice to tighten up some of the spacing. Reduce the divider between boxes, reduce white space before (slightly) and after the Variant Titles, Serializations, Publications, etc. headings. The text size for those headings could be reduced slightly as well. -- JLaTondre (talk) 22:37, 14 March 2017 (UTC)
Working on it... Ahasuerus 22:51, 14 March 2017 (UTC)
I like these suggestions, along with the one above about having "Jump to:" links. If we have those links, it may be good to add "Return to top" links, too, at the bottom of each section. ···日本穣 · 投稿 · Talk to Nihonjoe 22:50, 14 March 2017 (UTC)

(unindent) OK, the spacing has been adjusted. Please use Control-F5 to load the new layout. (There is more white space between the word "Publication" and the table that follows it than in the other boxes; it will take another patch to get everything in sync.) Ahasuerus 00:06, 15 March 2017 (UTC)

Fixed. Ahasuerus 18:53, 15 March 2017 (UTC)

Re: the "Publication" box being too far down, I have a different idea.

At this time the second box from the top contains a list of VTs and a list of serializations (if they exist.) We can reorganize it as follows:

  • Do not display the name of the VT's/serialization's author if it is the same as the name associated with the canonical title. We don't display it on Summary pages and it will free up a fair amount of screen real estate.
  • Take advantage of the freed up real estate and display the VT/serializations as a table with four columns: VTs, translated VTs, serializations, translated serializations. This is something that was requested a couple of years ago, but we had to have the language assignment process completed first.

This will reduce the amount of "stuff" between the box with the metadata and the "Publications" box. Ahasuerus 00:06, 15 March 2017 (UTC)

Nice thought; I'd really like to see how that looks. How will it work on narrow screens though?--Vasha 00:18, 15 March 2017 (UTC)
Well, at most the new table would contain 4 columns. That's not too bad considering that the "Publications" table contains 11 columns. Ahasuerus 00:23, 15 March 2017 (UTC)
I like the new layout but one minor nitpick. The "Other Titles" table has entries with the heading "Date", however, clearly these are not complete dates but rather just the year from the (variant) title records. Uzume 23:30, 2 April 2017 (UTC)
That's a good point. I have changed the headers to read "Year" instead of "Date". Ahasuerus 19:58, 3 April 2017 (UTC)

Bibliographic Warnings

There is a fair amount of history here. Back in 2006, i.e. when the first version of the "ISFDB 2.0" software was released and submission moderation was enabled, our data was very (and I mean very) dirty. For example, one of our original robots had run amok and imported thousands of "out of scope" records from library catalogs. Many other records were missing even the most basic bibliographic data.

During that era, title-level bibliographic warnings were useful because they let you identify potential problems with multiple publications at a glance. However, as data quality improved, the usefulness of bibliographic warnings declined. These days, if a publication record doesn't have some bibliographic information, chances are that there is a good reason for that.

All of the above explains why:

  • bibliographic warnings are now disabled for unregistered users
  • registered users can disable bibliographic warnings under User Preferences
  • the logic behind bibliographic warnings has been repeatedly fine-tuned, e.g. to ignore missing ISBNs for books published before ISBNs were introduced

Having said that, as long as the user chooses to to view bibliographic warnings, I think it is useful to display the word "None" if the logic finds no missing/suspicious data. Ahasuerus 00:39, 15 March 2017 (UTC)

New day, new thing to learn. :) Maybe reverse the default for them? I'll admit I never realized I can fully disable them but then I had not looked at this page for awhile (and when I did, I am not sure I knew what all those are so I just left it to show everything). :) Thanks for the explanation! Annie 01:09, 15 March 2017 (UTC)

2017-03-15 downtime

We are getting low on disk space, so it's time to purge Wiki history again. All but the last 50 versions of each Wiki page will be deleted.

The server will be unavailable between 10:30am and 10:50am server time. It's possible that it may take even longer. Ahasuerus 14:06, 15 March 2017 (UTC)

Everything should be back up. Ahasuerus 14:44, 15 March 2017 (UTC)

Templates for SFE3 and FantLab

SFE3 and FantLab are now supported as templates -- see Help:Using Templates and HTML in Note Fields for details. Ahasuerus 14:27, 15 March 2017 (UTC)

If you are taking requests for other languages, SFBG (http://www.sfbg.us/publisher/code , http://www.sfbg.us/pubsequence/code and http://www.sfbg.us/book/code) will be very useful for anything in Bulgarian (which I am still planning to start working on - I guess waiting to add Translator support will be waiting too long). The codes are stable (and once this is incorporated, the owner of the site is willing to send lists of changed codes if that ever happens (usually for disambig reasons and really rarely). Annie 19:02, 15 March 2017 (UTC)
Done. And yes, translator support is the big elephant in the room. I am currently working on less impactful FRs while considering various design approaches. Ahasuerus 19:26, 15 March 2017 (UTC)
Consider faster (she said looking innocently). :) Annie 19:28, 15 March 2017 (UTC)
Unfortunately, I am still waiting for medical nanobots to be perfected. However, the latest combination of drugs seems to give me more energy, so I guess the answer is "Maybe" :-) Ahasuerus 19:40, 15 March 2017 (UTC)
I have a weird idea about translators... why not create a template for adding a translator to a note - that for now will just print the names with some prefix but then when we figure out how we will support them can automatically start forwarding to where they belong (author pages, new pages, whatever)? Worst case scenario - they never link but at least they are uniform enough and thus convertable if we decide something else. If you go for that, it needs to be a three part one and two part one (Translator|name|transliteration) and just (Translator|name). Or we can leave the transliteration alone and just have the name as credited in the book. Annie 19:53, 15 March 2017 (UTC)
It would be easy to create a new template (let's call it {{Tr}} for now) that would have the same functionality as {{A}}. It would work well for certain translators like Brian Stableford and Michael Kandel who are also authors in their own right. It would be less useful when the translator hasn't written any SF. My primary concern is that it could confuse some users who, after clicking the link, would get something like "A search for 'F. Lancel' found 0 matches". Let me think about it some more... Ahasuerus 20:22, 15 March 2017 (UTC)
Don't make it a link initially - at least it will be standardized and can be changed later to a link. Baby steps - no matter how you implement the support, having all translators noted the same way cannot hurt and will allow automatic conversion. Annie 20:37, 15 March 2017 (UTC)
My idea is to basically prepare for either linking (so change the template, the already entered will work with no change) or for moving to a separate field/relationship - in which case a predefined format will help. It is the same as making a rule except that with a rule we will need to be careful for punctuation and what's not - this will clean up the view so we know where the name ends, will ensure that multiple translators are entered consistently and will allow counting to be done in the DB (and even determine when we have a lot of repetitions). Not best use of templates - but best way I can think of to prepare the data (and after all the cleanup I did in the last months, prepared data (the Italian magazines translations for example) is better than having all kinds of weird conventions and manual parsing and reading long messages to find the info you need. Annie 20:56, 15 March 2017 (UTC)
Oh, I see. It's a thought. Let me see what I can do... Ahasuerus 21:20, 15 March 2017 (UTC)
OK, I have created a new template, "Tr", on the development server. If you enter {{Tr|Brian Stableford and Rob S. Pierre}}, the note field will display "Translated by Brian Stableford and Rob S. Pierre". Any additional information like "as by Jean Valjean" will have to be added manually. It's far from perfect, but it can be viewed as the "low hanging fruit" of this thorny issue. Is this what you had in mind? Ahasuerus 22:02, 15 March 2017 (UTC)
Yep. Any chance to allow {{Tr|Елена Павлова|Elena Pavlova}} that gets populated as "Translated by Елена Павлова (Elena Pavlova)"? If not, will {{Tr|Елена Павлова (Elena Pavlova)}} work? Then all we need is a decent set of conventions (always use "and" for multiple translators for example) and we will at least have the data look the same (and downstream can keep playing with it).Annie 22:16, 15 March 2017 (UTC)
{{Tr|Елена Павлова (Elena Pavlova)}} should work. Actually, rereading what I wrote above I see that I was unclear. I should have written "Any additional information like "as by Jean Valjean" can be added manually as part of the template parameter". Thus {{Tr|Brian Stableford and Rob S. Pierre as by Jean Valjean}} will be displayed as "Translated by Brian Stableford and Rob S. Pierre as by Jean Valjean". Of course, you can also choose to put "as by Jean Valjean" outside of the scope of the template. Sorry about the confusion! Ahasuerus 22:28, 15 March 2017 (UTC)
OK... The problem with this approach is that this will make it very hard to just lift and put directly into a field/point to an author automatically. The multiple translators probably will need manual intervention anyway (unless if someone adds the template twice but for transliterations I kinda want to make it easy to just get the second part, ignore the third during a migration - so it can be automated. And adding it outside of the template defeats the purpose of trying to standardize.
Here is an example. A lot of stuff was translated through Russian into a lot of the Eastern European languages. So a statement such as "Translated by Име(Ime) from the Russian." will now become "Translated from the Russian. {{Tr|Име}}. If I add (Ime) in brackets or after the statement, we won't be able to run a script that says: "Look for "{{Tr|Име}}". If found, delete from notes and use "Име" to add to new field/link to author. No manual cleanup needed. Allowing the triple format, will allow you to just ignore the last part if we are linking to authors links (as we will have transliteration there already and it may be different from what will be here - as it may be common name and not exact transliteration) or keeping it with us if we are moving that to a separate section. :)
Now - if that is too complicated, I am fine with what you have now. I am just thinking of the best way to allow automation to help us when we need to change those - which is why I also want the template to start with. Annie 22:50, 15 March 2017 (UTC)
[After sleeping on it.] I think my main concern is that any kind of automation would be tricky to pull off, especially given the fact that our design hasn't been finalized yet. There are all kinds of cases that will require manual review, e.g. "Translated by Michael Kubiak (chapters 1 to 7) and Frauke Meier (chapters 8 to end of novel)" or "Translated by the author and Cecil Hemley" (real life examples.) For this reason I view the "Tr" template as a tool that will make it easier to enter and find translation credits rather than as a tool that will facilitate automation. Ahasuerus 20:20, 16 March 2017 (UTC)
That's fine. At least it will allow an easy report on them for manual migration. And yes - there will be corner cases but if used carefully, it will allow some automation. Maybe. I just think that we are overthinking the corner cases - most of those credits will be a very straightforward single name. Anyway - still better to have it than not - even if it is just to structure the statement better and allow easier finding of cases where it is there. And it should be easier to recognize when it is just a name and when it is more (manual move of the ones that are not just names leaving just names behind as we did with languages/English for example. So are we getting "Tr" as a template? :) Annie 20:26, 16 March 2017 (UTC)
Done -- see the announcement below. Ahasuerus 20:55, 16 March 2017 (UTC)
More seriously though - I will start with the Bulgarian authors I think, giving it some more time. Thanks for adding it. Annie 19:28, 15 March 2017 (UTC)
One for the Deutsche Nationalbibliothek would also be useful. --Vasha 19:43, 15 March 2017 (UTC)
It's cleverly disguised as "DNB" ;-) Ahasuerus 19:46, 15 March 2017 (UTC)

Charles Robinson, Charles Robinson

Our Author record 88489 conflates two illustrators

The header data including all 5 webpages belong to Charles Heath.

So do all the pre-1970 titles and the 1998, 2000, and 2003 COVERART.

LC Catalog credits young Charles for all of our 1970 to 1982 COVERART book titles except the 1976, not found there.

There are some more 1970s/80s titles, INTERIORART only.

I suggest that young Charles gets all 1970s/80s titles.

--Pwendt|talk 19:21, 15 March 2017 (UTC)

I found more info on young Charles Robinson [2] and added him at WikiData. Here I will now begin to disambiguate him as "(I)" --so that Charles Heath retains numeric id 88489. This should proceed publication by publication if/where that works to cover titles uniquely. Right? --Pwendt|talk 19:59, 17 March 2017 (UTC)
7 PubUpdate in the queue as I depart. The remaining 1970s/80s titles (6) appear in multiple publications (5) or belong to Charles Heath (1 or 2). --Pwendt|talk 20:59, 17 March 2017 (UTC)

Title page changes -- VTs

The VT/Serializations section of the Title page has been changed. VTs' authors are no longer displayed if they are the same as the parent's authors. If authors are displayed, they use the standard "as by" syntax and square brackets, e.g. see Glory Road and Inferno. Ahasuerus 21:19, 15 March 2017 (UTC)

Phase 2: The way VTs are displayed has been altered as per our earlier discussion. The new section is called "Other Titles" and contains a table with up to 4 columns. The columns are as follows:
  • Variant Titles
  • Translations
  • Serializations
  • Translated Serialization
"If This Goes On —" provides an example of a table with all four columns. "Coventry" shows a more common scenario with only 2 columns. Glory Road has 3 columns. To Sail Beyond the Sunset has only one column.
These changes are still raw and may need tweaking. For example, this section is currently called "Other Titles", but only because I couldn't think of another term that would encompass all possible permutations. Ahasuerus 23:47, 15 March 2017 (UTC)
I like it a lot. Maybe use "Other versions" instead of "Other titles"? Annie 23:50, 15 March 2017 (UTC)
Looks good! To make it look more consistent with the other tables the <li> elements should be removed and instead an even/odd style added to the rows (same way as in "Publications" table). Also some padding of the table cells left and right would be good. Jens Hitspacebar 23:57, 15 March 2017 (UTC)
For even better readability the year and language could be put into separate columns, with the main table header spanning two or three columns and the second row containing the column header for each row. Jens Hitspacebar 00:02, 16 March 2017 (UTC)
OK, the bullets have been removed. I'll see what I can do about adding alternating background colors tomorrow morning.
I am not sure about having separate columns for years and languages: the worst case scenario would be 10 columns (2+3+2+3) in the table. I'll have to toy with the table layout. Ahasuerus 03:05, 16 March 2017 (UTC)
Very nice, I like it.--Rkihara 03:12, 16 March 2017 (UTC)
As for my suggestion to split it up into columns: the worst case scenario with 10 columns might indeed look a bit cluttered. Some brainstorming: what if the four "Other Titles" tables would be converted into two different sections containing two tables each - i.e. a grouping of the tables. There are two grouping possibilities:
  1. Grouping by "is it a serialization or not?": One section called "Variants" with tables "Variant Titles" and "Translations"; and a second section called "Serializations" with tables "Serializations" and "Translated Serializations";
  2. or grouping by "is it a translation or not?": One section called "Same-language Variants" with tables "Variant Titles" and "Serializations"; and a second section called "Translations" with tables "Translations" and "Translated Serializations".
As a result it wouldn't look so cluttered. The cost for that would be usage of more vertical screen space and - depending on the amount of data - more scrolling until you get to the "Publications" section. I'm actually not sure right now if that's indeed a better solution, just thinking aloud, but I like #2 better than #1. You'd also get rid of the "Other Titles" term that way. Jens Hitspacebar 09:24, 16 March 2017 (UTC)
It was one of the approaches that I considered earlier this week. As you said, the cost would be losing some of the vertical space that we have just recovered. My plan is to polish the current implementation and then review the alternatives. Ahasuerus 16:33, 16 March 2017 (UTC)

(unindent) Italics removed, background color added. Looking into adding more columns. Ahasuerus 16:19, 16 March 2017 (UTC)

the background colors should be the light shades used for the puolications. --Vasha 17:30, 16 March 2017 (UTC)

(unindent) The "Other Titles" section has been changed as per Hitspacebar's original suggestion. Date, language and title/author are now three separate sub-columns within each column. Some examples:

Re: background colors. As Vasha pointed out, there is a discrepancy between the way "Other Titles" and "Awards" are displayed (darker colors) and the way "Publications" are displayed (lighter colors.) Upon closer examination, it turns out that the "lighter colors" scheme was originally added due to a programming error. Oops! However, we have been using it as part of the standard "Publications" table for about a year now and there have been no complaints. On the other hand, there are plenty of other tables which use the original, darker, scheme starting with the Forthcoming Books page. I guess the question is whether we want to standardize the color scheme and, if so, which scheme we want to pick. Ahasuerus 19:38, 16 March 2017 (UTC)

Did you loose the language in the translated serializations somewhere? :) Annie 19:44, 16 March 2017 (UTC)
Fixed! (Apparently "0" and "1" are not the same thing. Who knew?..) Ahasuerus 19:54, 16 March 2017 (UTC)
I like the darker ones better - just provides better contrast. Annie 19:44, 16 March 2017 (UTC)
Looks better and less cluttered than I feared when I came up with the idea. Actually, I like it a lot better this way. As for the colours: I'd prefer the lighter ones of the "Publications". The darker ones look so... dark :) Jens Hitspacebar 20:09, 16 March 2017 (UTC)

(unindent) Had been looking at some art titles (for example) and the new columns make the view there a bit... weird. I wonder if it is time to have a different view for art titles: 2 columns (one for cover art, one for interior - that is a more meaningful distinction than language here, both having language and year columns.). Any thoughts on that? Annie 20:20, 16 March 2017 (UTC)

Well, COVERART titles can't be serialized, so the way things work now, they can't have more than two columns in the "Other Titles" section. Perhaps we should add a third column for INTERIORART reprints? Ahasuerus 20:30, 16 March 2017 (UTC)
P.S. And, conversely, a "reprinted as cover art" column for INTERIORART titles? Ahasuerus 20:31, 16 March 2017 (UTC)
Well, that would be nice! Stonecreek 21:13, 16 March 2017 (UTC)
My point is that the language distinction is less interesting than the type distinction here. Yes - that will be nice. :) Annie 21:18, 16 March 2017 (UTC)
OK, I have added new columns for various INTERIORART/COVERART combinations, translated and otherwise. Here is what the originally linked COVERART title looks like now. Ahasuerus 22:50, 16 March 2017 (UTC)
And here is an example with all 4 columns for art over here just in case someone is interested. I like it :) Annie 23:05, 16 March 2017 (UTC)

I know it has been mentioned that the "Other Titles" and "Awards" (as well as "Bibliographic Warnings" but that is after) boxes push the "Publications" box somewhat far down the page. As a possible workaround for most users, JavaScript could be added to hide the content until opened (like what is sometimes done on pages with a "+" character button). I realize this would not work for our JavaScript-less users (e.g., text browsers which I believe we still support to some degree for viewing but not editing) but it might help for the vast majority of users. Uzume 00:06, 3 April 2017 (UTC)

A [more] button is a good idea.--Wolfram.winkler 07:29, 5 April 2017 (UTC)

Display parameter for templates in notes

In the course of this discussion, the question arose: would it be desirable to have the option to display a linked record (the discussion concerned authors, but it's not limited to them) with a different link text than the page title? I think yes, and in particular it would be good to be able to display disambiguated authors without the disambiguator which is not very informative. Ahasuerus, if I'm summarizing him correctly, thinks that for the sake of consistency and avoiding any possible confusion, the link text should always be the exact page title. --Vasha 03:30, 16 March 2017 (UTC)

That's right. I hope that "Written by Stephen King (I)" would be more informative and less likely to mislead than "Written by Stephen King". It should alert users to the fact that we have multiple "Stephen Kings" on file and prevent them from jumping to unwarranted conclusions. Ahasuerus 04:07, 16 March 2017 (UTC)
I prefer to see the name as is in our DB as well. Annie 21:22, 16 March 2017 (UTC)

"Tr" template added

A new template, "Tr", has been added -- see Help:Using Templates and HTML in Note Fields for details. As discussed earlier, it's nothing fancy, but using it should make migrating translators to the new system -- once we have it in place -- easier. Ahasuerus 20:55, 16 March 2017 (UTC)

I just tried it, but the template doesn't get parsed. Jens Hitspacebar
Hm, it works on the development server. Investigating... Ahasuerus 21:25, 16 March 2017 (UTC)
Fixed. Apologies, human error. Ahasuerus 21:34, 16 March 2017 (UTC)
Thanks a lot! BTW: I'm on the verge of starting to enter all the nominations for the category "best translation" of the Kurd Laßwitz Prize. Question is: is translator support, which would be able to link the translation to the award, more or less around the corner, i.e. should I wait for it? Otherwise I'd start entering them as "Untitled awards" and rework them later once translator support is done. No pressure, just asking :) Jens Hitspacebar 21:47, 16 March 2017 (UTC)
At this point we have gathered the requirements, which include support for "awards given to translations". They are not final yet, but I think we are pretty close. However, the design hasn't been finalized yet -- which can be a big can of worms -- so I wouldn't say that translator support is around the corner. However, I expect that it will be the next major project that we will tackle (emphasis on "major".) Ahasuerus 22:55, 16 March 2017 (UTC)

Titles with title date before the date of the earlier associated publication

(text moved from this Rule and Standard discussion):

I see that there are 2,098 title records that include the phrase "originally published", "originally appeared", or "originally printed". Ideally, none of those would be necessary because you could just look at the title record and SEE where the first appearance was. But it would take an awful lot of work to achieve that ideal. Chavey 03:03, 17 March 2017 (UTC)

Well... if we all agree that this is what we want to do, 2K are not that many. Plus it can take as long as it takes. Half of them will be as easy as creating the record based on the note; some will need research. Still not undoable. :) Annie 03:22, 17 March 2017 (UTC)
I am afraid there are many more permutations. For example, take "A Bottomless Grave" by Ambrose Bierce. It was first published in 1888, but our earliest pub appeared in 1958. The note field reads "First published in the San Francisco Examiner, February 26, 1888.". There are over 12,000 titles like that although some of them already have the first edition pub on file.
Or take "The Young King": the note doesn't explain where the story first appeared.
OK. So we have more than we thought and it may be a huge project to work through them. There will be some that will never be reconciled (we may know it was in a magazine in 1904 but no clue which issue) - not a reason not to mark them in some way that will allow someone with time and will to do research to reconcile them one day. And the ones that we know the history of? It is not undoable. Just because we may not be able to get them all, should we just give up and just say "oh, well, this train left the station, let's leave it like this"? :) Annie 04:10, 17 March 2017 (UTC)
Oh, I am not saying we shouldn't do it. I am just pointing out that there are some really big and juicy worms in this particular can :-) We'll have to think of ways to create at least somewhat manageable cleanup reports. Ahasuerus 04:16, 17 March 2017 (UTC)
Alphabetically. Decade per decade (starting from current and going back - the very old ones will be the pain to figure out). Random 300 per day. Some other new inventive mechanism :) And then will slowly get fixed. If we can identify the majority of them, reports will be the least of our problem I suspect. Plus - I am willing to speculate that from the 12K you are finding at least some have dates just messed up and need adjustment and not a new pub. Or we have the pub and it just need its content. :) Annie 04:24, 17 March 2017 (UTC)
I'll need to write a query to check how many titles predate their earliest pub. Ahasuerus 03:56, 17 March 2017 (UTC)
That will be an interesting list to look at. You will need to add pub-less titles as well - the ones that the report for pub-less skipped and anything that the report caught but was ignored by a moderator. Some of the ones that are like that (with just a note) are hiding there. Annie 04:10, 17 March 2017 (UTC)

(unindent) Preliminary results excluding publess titles (which are handled by a different cleanup report) and 0000-00-00 pubs:

+----------+--------------+
|      218 | ANTHOLOGY    |
|      700 | COLLECTION   |
|     2891 | COVERART     |
|    20530 | INTERIORART  |
|        8 | EDITOR       |
|    10038 | ESSAY        |
|      423 | INTERVIEW    |
|     5677 | NOVEL        |
|      312 | NONFICTION   |
|      122 | OMNIBUS      |
|     4911 | POEM         |
|     1646 | REVIEW       |
|       51 | SERIAL       |
|    33174 | SHORTFICTION |
|      205 | CHAPBOOK     |
+----------+--------------+

Ahasuerus 04:27, 17 March 2017 (UTC)

These are all titles where the date is before the newest pub in their list? Something does not sound right at all... Some of the containers are because of early ebook that is still missing (unknown ISBN I suspect) but these numbers are way too high. Or am I missing something? Annie 04:36, 17 March 2017 (UTC)
It looks legit to me. Here are some randomly accessed titles: an interview, a review, a serial (an improperly entered date, easy to correct), an EDITOR record, a novel, a poem. Ahasuerus 04:50, 17 March 2017 (UTC)
Time for a cleanup report I guess (with an option for moderators to ignore?)), type by type until we hit the big ones? They need either a note on the discrepancy or it to be resolved. Now that I know about them, they are bugging me - a bibliographic DB should make sense -- and that thing does not... (note to self - next time don't ask questions like that...) :) Thoughts? Annie 05:00, 17 March 2017 (UTC)
Chiming in to say, that when I was going through adding contents to 2016 anthologies/collections last December-January, there were a LOT of reprint stories to be added where we didn't yet have them in the database. If I'd stopped to add the first publication for each one, I wouldn't be finished yet. I just did notes. So yeah, 33,000 short stories to do actually sounds believable, unfortunately. --Vasha 06:22, 17 March 2017 (UTC)
I should start by mentioning that we already have this FR to add a cleanup report which would "verify if the minimum of the publication dates of a title is equal to the date in the title data".
Amusing. As usually my FRs were ignored, I've learned quite early to stop using them. Hauck 18:10, 17 March 2017 (UTC)
Well, at least now there is the warning in the Edit results so it should reduce the number of those (if editors and moderators care enough to resolve the issues that is). May be a few years later but it is a step in the right direction IMHO. Annie 18:27, 17 March 2017 (UTC)
Spot checking the data, I see a few different scenarios. First, we have (apparently) incorrectly entered dates. For example, the current date of the previously linked interview is "1997-00-00", but the first known publication date is "1998-03-00". If the interview was recorded in 1997 and published in 1998, we need to change the title date to 1998-03-00. Similarly, I have seen many introductions which were dated based on the author's signature ("Robert Silverberg, New York City, August 1971") rather than on the date of its first publication. We also have a number of serialized novels whose title date is currently set to the date of the first serialization, e.g. Pellucidar. As per Help:Use of the SERIAL type, "when cataloging novel length fiction, the date of the original serialization is not used and the date of the first book publication is used instead. This is an old (and arguably unfortunate) bibliographic convention, and we follow it." In most cases they should be easy to fix.
Second, we have titles which first appeared in "publications" which we typically do not enter. For example, as per Wikipedia, Tolkien's "Bilbo's Last Song" "was first published on a poster containing illustrations by Pauline Baynes in 1974, the year after Tolkien's death."
Other cases are more straightforward. Of course, some publications will be hard to find due to age, rarity, etc. Ahasuerus 17:45, 17 March 2017 (UTC)
Yes, there will be unresolvable ones(such as Wikipedia) but most of them can be resolved - easily or not. I am just thinking that if we keep ignoring these, the number of titles will multiply and a few years down the road, we will be staring at too many titles to tackle... And in the meantime we have a mix of notes (that cannot be searched well to see how many stories were published in a magazine), wrong dates (that really need fixing) and no notes at all where our data is plain wrong or where it may be correct but noone knows why. I do not expect the DB to be always correct (human beings and all that) but when we know about errors, why not fix them? I have similar misgivings for anthologies and collections (and magazines) with no content but lack of data is a bit better than wrong one:) Annie 18:27, 17 March 2017 (UTC)
I am sure we all agree that errors should be fixed. We just need to come up with the best way to approach the process. For example, suppose we add a new cleanup report. The first time it runs, it finds an 1888 title with no pubs prior to 1896. An editor checks the usual bibliographic sources but doesn't find anything. Should we just "ignore" the title record? Or should we create a new template for these types of situations? Something like {{Unknown1stAppearance}} which would be displayed as "The publication in which this title first appeared is currently unknown" and which would be used by the cleanup report to ignore the title? Ahasuerus 21:26, 17 March 2017 (UTC)
Sorry for going on and on and trying to convince the already convinced :) I was thinking of a new template (which is why I mentioned "marking them" somewhere earlier). This way someone can still figure them out later and we will know that we had looked at these and fixed errors - and they are still standing because we cannot find the bibliographic sources... yet. Annie 21:57, 17 March 2017 (UTC)
What about webzine publications, won't those have to remain as notes? --Vasha 22:17, 17 March 2017 (UTC)
We do not set the date for the webzine-first stories based on the webzine first date though, do we? At least my reading of the rules is that we do not (except for the special cases like tor.com that we index which we do index). If we do, then the whole "we do not add webzines" is getting a bit... weird. :) And we need to start talking about that outside of this conversation :) But yes - if it is out of scope here, we will either need to add as non-genre (magazines, newspapers, books) or note (the poster example above for Bilbo). Webzines... hm... comes down to what we count as first date for the story I guess. Annie 22:23, 17 March 2017 (UTC)
Well, if we decide that a web-first publication should not determine the date, then we nonetheless definitely should make a note mentioning that publication. However, I can tell you, from reading the credits/copyrights for all those anthologies I just added, that it is nearly universal for those credits to state a first publication if it's in a webzine; copyright dates are a bit funnier, since some people re-copyright all their stories when they republish them in collections, but if they use earlier copyright dates, then they will always be the date of the web publications. Shouldn't we fall in line with such a universal practice?
One thing that is not consistent is whether such credits pages mention early publications on the author's personal blog or site. Sometimes; more often not. (I know this because I'm in the habit of trying to check these credit pages by googling, and thus I find unmentioned web publications.) --Vasha 22:39, 17 March 2017 (UTC)
Oh, I know. As I said - I think we need to start talking about the web-based stuff again sooner or later - the world is moving, some of our rules do not make much sense in it :) Part of the reason we do not use copyright dates is because it is not always based on a publication. And in the self-publishing world, it gets even trickier (copyright of when they created it and not when published - I was chasing a few of those the last few days). But then "first publication" becomes an interesting concept. And we need a better definition of it for cases like this. Annie 22:46, 17 March 2017 (UTC)
Ooh, I don't envy you trying to identify the first appearance of self-published things! Often, if they've been removed from sale, not much trace of them remains; and if they are still on sale, they may well bear a date on Amazon or Smashwords that has no relation to when they were first sold. Your best hope is if the author provided a good bibliography on their site. Here's offering you some virtual aspirin for the headaches... --Vasha 22:54, 17 March 2017 (UTC)
Well... I am doing some due diligence but if something is untraceable, I use whatever date I can find and note where the date comes from. If one day an older one is discovered, it will get fixed. This whole conversation started because of the dual practice of saying "First published in London News, 25 May 1934" vs creating a record for "London News, 25 May 1934" as non-genre and adding the story there depending solely on how the editor feels about it. In one case it is searchable and we can see how many stories "London News" published; in the other - not so much. It just got a bit bigger because of the way the DB works :) Annie 23:18, 17 March 2017 (UTC)

Amazon Templates stopped Working

The new Amazon templates have stopped linking. They worked great on the first day then stopped.--Chris J 20:33, 17 March 2017 (UTC)

Sorry, it was an unintended side effect of the last template change. Things should be back to normal now, e.g. see this pub. Ahasuerus 21:01, 17 March 2017 (UTC)

tor.com stories and chapbooks art

We have the series of chapbooks over here and the webzine entries here. The stories are properly merged. The art - not so much :) From the few I looked at, the covers of the chapbooks is the same as the art on the site (which we had indexed). Any objections to varianting them properly (both are always visible - one in Amazon, one in tor.com so we will be able to see and make sure they are the same indeed)? Or am I missing a reason not to? Thanks! Annie 22:19, 17 March 2017 (UTC)

Certainly, why not? --Vasha 22:48, 17 March 2017 (UTC)
PS: and it is varianted and not merged because it is covers on the chapbooks :). My initial message does not make that clear. Annie 22:57, 17 March 2017 (UTC)

Publisher page changes

The layout of the Publisher page has been aligned with the recently tweaked layout of the Title pages. Here is an example of what a publisher with a limited amount of data looks like. And here is what a publisher with a lot of data looks like. Ahasuerus 14:13, 18 March 2017 (UTC)

Synchronizing the layout of bibliographic pages

Based on the discussion of the Title page that we had a few days ago, I have further synchronized the layouts of the "content boxes" on the Title, Publication and Publisher pages.

Please note that the layout of what's inside these boxes is still somewhat inconsistent and will be revisited later, but at least the relative position of the boxes, their margins, white space padding, etc are now the same. I plan to make the same changes to all the other bibliographic pages shortly. Ahasuerus 19:35, 21 March 2017 (UTC)

Publication series done. Ahasuerus 21:02, 21 March 2017 (UTC)
Regular (title) series done although the Series Grid page for magazines is still outstanding. If you see anything unusual, please let me know. Ahasuerus 01:59, 22 March 2017 (UTC)
Series grid done. Ahasuerus 19:50, 22 March 2017 (UTC)
Author pages done. Let's see if these changes help with the previously reported "starkness" problem -- see Kim Stanley Robinson, Bob Smith and Bob Smith (I) for examples of long, medium and short Summary pages. Ahasuerus 20:46, 22 March 2017 (UTC)
I like the look of having the head-information and the bibliography in separate boxes on the author pages. --Vasha 21:20, 3 April 2017 (UTC)

New: Trackers to help keep up with current short fiction

I've created two wiki pages (with a lot of help from Annie-- thanks again!) that are intended to help with adding newly-published (not reprint) short fiction to the database: the Magazine Issue Tracker and the Anthology and Collection Tracker. They are schedules of expected forthcoming publications that can be used on a monthly basis for database updates; they also show which issues and volumes have contents records.

Please take a look and tell me what you think of the design, and of the "How to Use" information at the bottom of the pages. Improvements will be gratefully accepted. --Vasha 19:53, 21 March 2017 (UTC)

Pretty cool. Thanks for assembling! My two suggestions:
Put a legend of the colors at the top of the table, for quick reference.
Maybe specifically label as 2017? And what will happen when 2017 rolls over to 2018? It will be here before we're ready... Albinoflea 03:40, 23 March 2017 (UTC)
I would think that in 2018, the current tables with data will be moved to an archived page (just so the remaining 2017 ones can be finished) and this one will remain empty again for "Current Year". Vasha, what do you think? Annie 03:45, 23 March 2017 (UTC)
Good thoughts. I will put in a header that identifies this iteration as 2017, but keep the page title generic. And as Annie says, at the end of the year we can put something like a box with links to archived pages at the bottom, and change the header to 2018. --Vasha 04:28, 23 March 2017 (UTC)
Changes made; how's that? --Vasha 04:47, 23 March 2017 (UTC)
I've also lightened up the background colors. --Vasha 05:21, 23 March 2017 (UTC)

Format question: dos vs tp

I had been adding some entries here and it strikes me that most of the records are set to "tp" instead of "dos". As it is a double cover (cover on the back and on the front), double-novel (*novel 60s style anyway), it looks to me like the definition of "dos" and that overwrites the more generic "tp" (anything that is not already covered and is paper)... Am I misunderstanding what "dos" is supposed to be used for? Or do most of these need a format conversion? Annie 23:53, 22 March 2017 (UTC)

Yes, my understanding is that "dos" would take precedence over "tp", but I'm guessing that since the two novels aren't rotated 180º to each other (at least from the cover examples shown) that these might not technically be valid "dos" candidates. At least that's my take. Perhaps the documentation should be clarified. Albinoflea 03:26, 23 March 2017 (UTC)
Hm, I did not think about the rotation (never seen one of the double novels that are usually used for an example) and you are probably right. I think that the help page needs a bit of an update - may be obvious for most people around here that they need the rotation but it tripped me. Thanks! :) Annie 03:43, 23 March 2017 (UTC)
Only if it's rotated and a part therefore readable from the back, or if there are two backs glued together it's a dos. That's how I understood it so far. The Dos-à-dos binding Wikipedia article has a good description. Jens Hitspacebar 11:21, 23 March 2017 (UTC)
That is explaining a rotation on the other axis from what I was thinking. Still not what Armchair fiction are doing though so I will fix those. Now you made me want to go and find one of the Ace doubles just so I can look at the thing... Annie 17:55, 23 March 2017 (UTC)

David Gemmell Award

Can we add this award to the available awards in the database? It's a popular vote (people vote on the website) set of awards given out each year (since 2009 for the Legend Award, and 2010 for the other two) for fantasy books. There are three specific awards:

  • Legend Award - awarded to the fantasy title judged the year’s best, as chosen by open vote
  • Morningstar Award - awarded to the the author judged to have made the year’s best debut in fantasy fiction, as chosen by open vote
  • Ravenheart Award - awarded to the artist(s) responsible for the year’s best fantasy book cover art, as chosen by open vote

The short name for the award is "Gemmell Award". The website for the award is http://www.gemmellawards.com/. ···日本穣 · 投稿 · Talk to Nihonjoe 21:32, 23 March 2017 (UTC)

Their online voting system is somewhat unorthodox, but I recognize the people who administer the award as established professionals. Hopefully they have added safeguards to prevent multiple voting and such. Otherwise the awards look legitimate and I have no objection to adding them. Ahasuerus 00:43, 24 March 2017 (UTC)
Okay. Let me know when it's been added and I'll populate it. ···日本穣 · 投稿 · Talk to Nihonjoe 19:01, 24 March 2017 (UTC)
Not seeing any objections to adding it. This one should be fairly quick to add the content since it hasn't been around for too long. Thanks in advance, Ahasuerus. ···日本穣 · 投稿 · Talk to Nihonjoe 18:33, 5 April 2017 (UTC)
Done, including all 3 categories. Ahasuerus 18:56, 5 April 2017 (UTC)
Awesome, I'll fill them up over the next couple days or three. ···日本穣 · 投稿 · Talk to Nihonjoe 19:44, 5 April 2017 (UTC)

Add language request

Can Akkadian be added as a language selection? I'm concerned with The Epic of Gilgamesh. Thanks, Ldb001 14:20, 24 March 2017 (UTC)

Akkadian is an ISO-2-recognized language (language code akk), so it shouldn't be a problem. Let me double-check with Linguist to see if there is anything else we may need to add to support Gilgamesh. Ahasuerus 14:28, 24 March 2017 (UTC)
No problem with Akkadian as a language. But remember that the original language of The Epic of Gilgamesh is Sumerian, which is utterly different… Linguist 14:33, 24 March 2017 (UTC).
Should we add both Akkadian ("akk") and Sumerian ("sux") then? They are both ISO-2-recognized languages, so it wouldn't cause any issues. Ahasuerus 15:09, 24 March 2017 (UTC)
Might as well. As the French idiom goes, ça ne mange pas de pain (there's no harm in it) :o) Linguist 15:19, 24 March 2017 (UTC).
Done! Ahasuerus 15:45, 24 March 2017 (UTC)
Thank you for taking care of that so quickly! My friend wikipedia tells me that the two surviving tablet sources of the epic proper are in Akkadian and Old Babylonian (which is called a dialect of Akkadian). Some older Sumerian associated material also exists. I'm sure that modern texts draw from all sources to make as complete a version as possible, but because the Akkadian source is more complete, for shorthand Akkadian seems to be a good choice. I have no knowledge of what proportions of what sources have been used for this translation.
I don't know protocol for including a hypothetical (lost) original form which may have been in Sumerian. At any rate the Akkadian/Babylonian is apparently the oldest version of the epic that exists. Thank you again for the language support, Ldb001 22:53, 24 March 2017 (UTC)
I have handled these sorts of situation by making a variant parent title without any supporting pubs. You can then add notes to this parent title records (and if the pub ever is found it can be added later). This would make things like an English translation also exist under the missing Sumerian parent record too (regardless what it was really translated from; just make name good notes when there are translations of translations). Uzume 23:50, 2 April 2017 (UTC)

Possible new reports

Would it be easy to create "award eligible" reports that listed:

  • All the novels that came out in the previous year
  • All the novellas that came out in the previous year
  • All the novelettes that came out in the previous year
  • All the short stories that came out in the previous year

And so on, for whichever categories we would cover? It might be a good way to collaborate with the Hugo, Nebula, Dragon Award, etc., crowd to provide an easy set of reports that showed this information. What do you think? ···日本穣 · 投稿 · Talk to Nihonjoe 17:23, 29 March 2017 (UTC)

You can run an Advanced Title Search on Year=2016 and Title Type=NOVEL and Language=English (or any other language) to get a report of novels first published in 2016. Ditto for novellas etc. The report is limited to 100 titles per page though. Ahasuerus 18:13, 29 March 2017 (UTC)
Yeah, that limitation is why I thought it might be useful to have a system-generated report that is run once a year instead. ···日本穣 · 投稿 · Talk to Nihonjoe 19:31, 29 March 2017 (UTC)
Except that it won't be run once a year because we add more stories daily - I added more 2016 ones a few days ago for example. Annie 19:50, 29 March 2017 (UTC)
(after edit conflict)
We are constantly adding/merging/deleting titles, so a report that is run once a year would quickly get out of sync with the rest of the database. On the other hand, it wouldn't be a particularly resources-hungry report (it takes less than a second to compile on the development server), so we could make it available as an on-demand report.
The main report page will presumably include the following drop-down lists:
  • Year, limited to 1900-current year
  • Title type/length, including every title type and every length types (SHORTFICTION would include all short fiction)
  • Language, including "all"
At 5K-10K lines per report it will be a bit unwieldy, but probably better than paging through the same information 100 titles per page. We also have lists of Most-Reviewed Titles by year and by decade. Ahasuerus 19:59, 29 March 2017 (UTC)
So, is there enough support for the proposed page to justify an FR? Ahasuerus 21:28, 30 March 2017 (UTC)
I support the idea. ···日本穣 · 投稿 · Talk to Nihonjoe 22:50, 30 March 2017 (UTC)
Oh well - why not. However - I have a question on the proposed implementation: what year will be a story shown in if it is a German story from 1989 but published in English in 2014, new name in 2017 in English? Awards-eligibility-wise, this should be 2014 for English but that will mean looking at a variant date - and finding the correct variant based on the language. And then the same would be nice for other languages. So if I filter by German and 1989, I expect to see the story; (English, 2014) should also show it and the 2017 variant is never accounted for. Is that the plan? :) Annie 23:10, 30 March 2017 (UTC)
Award eligibility rules can be complicated. For example, the Hugo rules say:
  • Works published in prior years outside of the USA are eligible if they were published for the first time in the USA in the current year.
Other awards may have different caveats and quirks. I doubt we can account for all of them, especially considering that it is a moving target. My plan was more modest -- simply provide a list of the titles that match the entered criteria. It will be up to the user to tweak the search criteria depending on his or her needs. Ahasuerus 23:41, 30 March 2017 (UTC)
That's fine as long as in my example above, English, 2014 pulls the story but English, 2017 does not - despite having both as variants :) Actually for what I would like to use that, that is not a problem either way - but for what the conversation started as, it does make sense. Can we add a "translated" flag as a column (or even a filter) to indicate which of the found stories are not originally in the language you filtered on? Annie 23:57, 30 March 2017 (UTC)
We could do what "Show All Titles" does: add a column for the parent title and perhaps another one for the language of the canonical title if it's different from the variant's. Ahasuerus 00:15, 31 March 2017 (UTC)
Sounds good. Annie 00:53, 31 March 2017 (UTC)
OK, FR 1024 has been created. Ahasuerus 15:37, 31 March 2017 (UTC)

Perry Rhodan help needed

Can someone that understand the series of Perry Rhodan figure out which of the series should the three listed here go and add them to their proper series (and single title record as these are from the same year). Thanks! Annie 18:10, 29 March 2017 (UTC)

Noone knows where these three fit? :)Annie 18:01, 31 March 2017 (UTC)
I used to have to find this on my own (as I said, data consistency is not a favorite sport here), by analogy with this, I would have entered "Perry Rhodan (2. Aufl.)" (coherent with #PR2 catalog number) and then merge by year.Hauck 18:11, 31 March 2017 (UTC)
But we should not need to do that when there are a few people that actively add these and understand the series... Oh well - did not realize that you were fixing those on their behalf - I will fix them then. Thanks! Annie 18:39, 31 March 2017 (UTC)
Merge submitted, moderator needs to approve. Annie 18:49, 31 March 2017 (UTC)
And all done. Thanks.Annie 22:20, 31 March 2017 (UTC)

Make Variant - submission review changes

The "review" page for Make Variant submissions has been changed. The following lines have been added:

  • Length
  • Juvenile
  • Non-Genre
  • Graphic
  • Novelization

When varianting to an existing title, any mismatches will generate a warning. Ahasuerus 21:26, 30 March 2017 (UTC)

Note that we still haven't decided what to do when a text "changes category" in translation (e.g. "upward" when translated from english to french or the other way from german to english). Hauck 09:08, 31 March 2017 (UTC)
Well, the ss/nt/nv division only applies to English anyway. Maybe other languages shouldn't have length set? --Vasha 14:21, 31 March 2017 (UTC)
Maybe we should not even list titles not in english. Hauck 17:03, 31 March 2017 (UTC)
Why would it only apply to English? Bulgarian has 3 different words for these as well for example (as does Russian and I suspect most other languages). And online bibliographies split the stories in the 3 groups. So the division is very much applicable for non-English as well. Annie 16:49, 31 March 2017 (UTC)
PS: And even where novelette is somewhat of an artificial construct (classically the split is in 2), the other 2 had always existed. I am strongly against not listing the type in other languages - a novella and a short story are as different as a novel and a short story, regardless of the language. Plus the standard English split had become a de-facto standard in the SF field anyway. Annie 17:13, 31 March 2017 (UTC)
OK... So what is the word length of the three categories in Bulgarian? --Vasha 16:50, 2 April 2017 (UTC)

Additional ISBN warnings

The software has been updated to display yellow warnings if:

  • the entered ISBN contains 10 characters and the publication appeared after 2007, or
  • the entered ISBN contains 13 characters and the publication appeared prior to 2005

Ahasuerus 23:35, 30 March 2017 (UTC)

Excellent... Thanks! Albinoflea 03:32, 31 March 2017 (UTC)

Bibliographic warnings: "missing page count"

Title pages have been modified not to display "missing page count" warnings for webzines, audio books and digital books. Ahasuerus 17:29, 31 March 2017 (UTC)

Title type/publication type validation

The 4 publication-specific edit forms -- NewPub, AddPub, ClonePub and EditPub -- have been updated to check for EDITOR mismatches. Duplicate EDITOR titles, EDITOR titles in non-magazine/fanzine pubs, and missing EDITOR titles in magazine/fanzine pubs should result in pop-up error messages from now on. You may need to do a full form re-load (Control-F5 in most browsers) for the new functionality to become available immediately.

If everything works out and there are no issues, I will add similar checks for other publication/title mismatches starting tomorrow. Ahasuerus 02:10, 1 April 2017 (UTC)

It turns out that the pop-up validation part of the new code didn't play nice with older browsers. I have disabled it pending an investigation. You will still get a yellow error message if you try to create a submission with an invalid or missing EDITOR title. Ahasuerus 15:29, 1 April 2017 (UTC)
The pop-up validation code has been rewritten using older, hopefully backward compatible, features. When entering/editing the next publication, please force a full page re-load (Control-F5 in most browsers.) If you encounter any issues, please let me know. Ahasuerus 16:54, 1 April 2017 (UTC)

(unindent) Additional validation has been added. CHAPBOOK publications are now required to have one (and only one) CHAPBOOK title and at least one SHORTFICTION/POEM/SERIAL title. Ahasuerus 23:34, 1 April 2017 (UTC)

Will it allow the chapbook to be added (NewPub) without a story inside (so it can be imported later)? Or will a story always need to be added and then merged? Annie 00:36, 2 April 2017 (UTC)
At this time the validation logic requires at least one SHORTFICTION/POEM/SERIAL title for new CHAPBOOK publications. However, it would be easy to change if the import route is common. Ahasuerus 01:45, 2 April 2017 (UTC)
I much prefer to import than to merge (all those reprints and kindle singles and what's not these days - and if we will ramp up on the ebooks adding, there will be even more of these) but if the agreement is that there are too much issues if it is left this way, I will change my usual routine. But we will need a "short fiction duplicates" report - at least chapbooks with no content have a report (I think?) and this will help with spotting the second steps not being taken. Annie 02:36, 2 April 2017 (UTC)
Now that you mention it, it makes sense. Let me revert the problematic part of the last patch... Ahasuerus 03:38, 2 April 2017 (UTC)
Done. Ahasuerus 15:40, 2 April 2017 (UTC)

(unindent) ANTHOLOGY, COLLECTION, NONFICTION, NOVEL, and OMNIBUS titles are no longer allowed in CHAPBOOK publications. Ahasuerus 16:07, 2 April 2017 (UTC)

NOVEL titles are no longer allowed in MAGAZINE/FANZINE publications. A reference title (i.e. a title of the same type as the publication record) is now required when editing/cloning publications. Ahasuerus 21:43, 2 April 2017 (UTC)
The "reference title" requirement has been streamlined for Clone/Import/Export submissions and should no longer generate spurious errors. Ahasuerus 13:24, 3 April 2017 (UTC)
Putting them in as SERIAL (complete novel) seems odd to me -- I know there must be a reason. What's the backstory? --Vasha 22:52, 2 April 2017 (UTC)
That's right, there is a reason (or perhaps multiple reasons) behind the current approach -- see this Help section for the gory details. Ahasuerus 13:24, 3 April 2017 (UTC)
Thanks, that makes a lot of sense. So, I decided to look for novels that have their title date erroneously set to the date of a serialization, and quickly found one, A Yank at Valhalla. Anyone else want to have a go at continuing the search? --Vasha 14:38, 3 April 2017 (UTC)
What about this, a serial never printed in book form. Should the date be "unpublished"? --Vasha 14:43, 3 April 2017 (UTC)
You are right, the use of dates for SERIALs and their parent titles has been somewhat inconsistent. The reasoning described in that Help section has one significant flaw: it distorts Author Biblio pages by messing up the order in which the author's works appeared.
Consider Jack Williamson's Summary page. At this time the first title in the "Novels" section is The Green Girl, which was first serialized in 1930 and published in book form in 1950. The reason it appears first is that someone set the NOVEL title's date to 1930. On the other hand, the title date of the Golden Blood is set to 1964, the date of the first NOVEL publication, even though it was first serialized in 1933.
My current thinking is that there is got to be a way to accommodate the two sets of concerns using some software trick, but I haven't been able to come up with a good design so far. Ahasuerus 19:00, 3 April 2017 (UTC)
I cannot attest to this being a good design but we could have parent titles always be empty and all pubs be under a variant. This way the parent could be the real original publication date (regardless of publication method) and the child novel title record the original novel publication date (and of course serials always represent their dates since they are basically one-to-one with their pubs unless there is a true reprint). And other variants are still possible (name differences and translations, etc.). I do not like the "unpublished" parent concept for novels only ever published in magazines. Uzume 05:40, 4 April 2017 (UTC)

2017-04-01 downtime @7:10pm

The server will be unavailable between 7:10pm and 7:12pm server (US Eastern) time. Ahasuerus 22:53, 1 April 2017 (UTC)

The server is back up. Ahasuerus 23:32, 1 April 2017 (UTC)

Title merge bug with transliterated titles

I have been doing some title merges (for Japanese magazine editorial records making them "annualized") and I have noticed an issue related to transliterated titles. Instead of letting me pick which title record to take transliterated titles from, it automatically just forces me to take them all (which is not right and now I have to go back and edit the titles individually to remove these incorrect transliterated titles). Can this be fixed? Thank you, Uzume 14:16, 4 April 2017 (UTC)

It's not a bug, it's a feature :-) That's how all multiply occurring values like Web pages and transliterated names are handled when merging title/author/publisher records. If one or more of the multiply occurring values is in error, they need to be removed before or after the merge. Ahasuerus 15:27, 4 April 2017 (UTC)
That is not really good for titles. If I have titles "XXX" and "YYY" with transliterations, why would I want to choose one of those titles but have to take transliterations for both? Uzume 16:15, 4 April 2017 (UTC)
When merging multiply occurring values, numerous permutations are possible. Sometimes you may want to keep all of them. Other times you may want to keep the values from just one record. Or you may want to keep the values from 2 out of N records. Or you may want to keep selected values from record 2 and 3, but drop all of the values from record 1 as well as selected values from records 2 and 3.
It's possible to support all of these options by adding check-boxes for individual multiply occurring values, but it's a low priority. Ahasuerus 18:00, 4 April 2017 (UTC)
Transliteration is for what is now the title. So why would we want to keep a transliteration from an old now non-existing title? And when Magazines are concerned, you never want the individual issues transliterations... Annie 18:03, 4 April 2017 (UTC)
Let me use an example. Suppose I want to merge two Bulgarian title records. The titles are identical, "Лъвът"/"Лъвът". The first one has the following transliterations: L"v"vt and Luvuvt. The second one has Luvuvt and Lavavt. The way the software works, merging these titles will result in the following transliterated values: L"v"vt, Luvuvt and Lavavt. (The two occurrences of Luvuvt will become one.) Is this the desired behavior? Ahasuerus 18:14, 4 April 2017 (UTC)
That is fine, combining the different transliterations do not harm anything when the two titles match to start with. Not so much if you are merging Знаме, Януари 2016 and Знаме - 2016. When you merge that, the Знаме - 2016 will now have also a transliteration as Zname, Yanuari 2016. Maybe make a special exception for EDITOR records? Annie 18:28, 4 April 2017 (UTC)
It would appear that the underlying problem is not so much the title type but rather the fact that the about-to-be-merged titles are different. It's possible to change the software to let editors select one set of transliterated values when titles are different. However, there are various scenarios to consider. For example, suppose I want to merge title records A, B and C. They all have transliterated values associated with them. Records A and B have the same title while C's title is different. What would be the desired software behavior in this case? Also, I am concerned that introducing additional exceptions may make life even harder for new editors. Ahasuerus 19:36, 4 April 2017 (UTC)
Yes, but it is just more likely to happen with an EDITOR :) Maybe keep it easy - if at least one of the titles is different, force the editors to select the same way as they select a title. If there is a single title, then just combine all existing ones? Or even combine the transliteration with the title - if a title need to be selected, its transliteration comes with it automatically. Annie 20:35, 4 April 2017 (UTC)
Why not let the editor select such? If you have five titles to merge give me six choices. I can pick which one or have the complete merge (methinks it is overkill to allow partial merges, etc. though that could be accomplished with checkboxes). Uzume 21:27, 4 April 2017 (UTC)
After sleeping on it, I think the last approach makes the most sense. It's user-friendly and consistent. I have created FR 1028 which covers all multiply occurring fields, not just transliterations. Due to the way merges work under the hood, it won't be a trivial change, but it should be doable. Ahasuerus 15:47, 5 April 2017 (UTC)
To make matters works the submission record appears like it chose one of the title to take transliterations from when in fact they were all merged (when the submission is committed). Thank you, Uzume 14:22, 4 April 2017 (UTC)
I am not sure I understand. Could you please provide an example? Ahasuerus 15:27, 4 April 2017 (UTC)
Unfortunately, TitleMerge submissions become useless to look at after they are committed by a moderator. Uzume 16:15, 4 April 2017 (UTC)
Do you recall the sequence of events and the exhibited behavior? If you could describe it, I could try to recreate the problem on the development server. Ahasuerus 19:38, 4 April 2017 (UTC)
I just merged 2-3 Japanese editorial titles where each had multiple (2-4) existing transliterations. Since you are saying currently, complete merge of multi-value fields like transliterations is expected behavior, I am wondering how that is displayed upon submission (and in looking at the submission in the queue before moderation). Uzume 02:30, 5 April 2017 (UTC)
When merging multiply occurring fields, all cells use the background color for "retained" values. Ahasuerus 16:45, 5 April 2017 (UTC)

PLAY title type

Is it time to implement the PLAY title type yet? I just added half-a-dozen more of them today. (A cappella Zoo loved to publish mini-dramas.) --Vasha 15:02, 4 April 2017 (UTC)

No, I'm not sure that a consensus was reached. And, if we should restage the play, I'm still completely against this move. Hauck 15:06, 4 April 2017 (UTC)
I am sort of ambivalent, however, Hauck, why are you against it (I do not see it much different than POEM)? Uzume 15:16, 4 April 2017 (UTC)
Well, somewhere in the mists of the wiki, I gave some arguments. In a nutshell, I see the possible inflation of types as not desirable. Half-jokingly I proposed also HAIKU, SONG, FILK, CARTOON, TVSCRIPT (all types that we're presently listing under diverse categories) etc... My main objection to this complexification is the difficulty to convey such subtilities to new contributors (here I speak for myself as I'm regularly welcoming and guiding new users). I'm also also worried by the potential frictions that may arise with such detailed categories (I remember some heated debates on prose poems, are they SHORTSTORY or POEM?). Hauck 15:29, 4 April 2017 (UTC)
Yes, I agree that multiplication of types would be confusing and make it harder to assign things. (I can never remember what Locus & Contento mean by all those 20 abbreviations they use for categories.) That is one reason why, when adding PROSE POEM was proposed, I decided that I was against it. But I would make an exception for PLAY. That is a very clear and obvious category. There are a few cases which might, just possibly, be confusing, because the scenario for a movie is occasionally written like a short story. But if it says on it "a movie scenario", then it's a play. All right, admittedly sometimes you may have a story in dialogue form. But if it's not explicitly identified as a play, then it isn't one. --Vasha 15:47, 4 April 2017 (UTC)
I don't think that we have enough plays to justify a new type - especially because there are two types of plays (prose and verse) and there is a difference between a short (2 pages one) and a long (novella length one) - now keeping them with the other text styles, allows the distinction. I would not be opposed to a template that can be added to the notes that make it easier to show/find them. Annie 16:28, 4 April 2017 (UTC)
After hearing Hauck's argument, I agree we do not need a PLAY type. I would be open to a generic script type that could include plays as well as any other types of scripts we might run into and would be applicable under the rules of acquisition. Uzume 21:30, 4 April 2017 (UTC)
That was what I meant anyhow -- a title type covering prose and verse dramas, tv scripts, movie, radio, audio dramas, librettos. Last time the subject was discussed, the question of what to call it came up, and it was pointed out that locus indexes all those things under "play", so it might be good to use the same term. But "script" or "drama" would be OK. Vasha 21:42, 4 April 2017 (UTC)
I support "SCRIPT", as it would cover stage, television, radio/audio, and film scripts. ···日本穣 · 投稿 · Talk to Nihonjoe 21:55, 4 April 2017 (UTC)
And if the consensus is against a title type, a template in the notes would not be a bad idea. Inserting the words "This work is the script of a drama" maybe? That sounds somewhat odd but it's understandable. Vasha 22:07, 4 April 2017 (UTC)
And there would also be some plays that are of NOVEL length; and IMHO it wouldn't be desirable to separate this two. Stonecreek 07:20, 5 April 2017 (UTC)
Still opposed to the idea of SCRIPT (I've just thought about some pure SHORTFICTION texts that use the format of scripts without having been concieved to be filmed or aired, where should we put them?), the template is a good idea (or the simpler mention in the note field). Hauck 07:23, 5 April 2017 (UTC)
Right! Terry Bisson did a few of those things that are made solely out of the text of conversations. Stonecreek 07:30, 5 April 2017 (UTC)
That's why I said this category only applies to works that are definitely identified as the script to some sort of drama production. Anyone want to tweak the wording to that proposed template to make this more clear? (I M coming round to the idea that a template would be better than a title type.) Vasha 16:13, 5 April 2017 (UTC)
And who makes the decision? If it looks like a play, an editor will add it as a play even for those Bisson ones - and then a moderator will need to realize this is the case and fix. The less confusing we keep all that, the less likely we end up with everything being set in its own way :) Plus with a template, counting those will be easy and if we see a lot? Easily fixable :) Annie 16:31, 5 April 2017 (UTC)

(Unindent) How about {{SCRIPT}} to insert the words "This work is a script for a production." Vasha 16:51, 5 April 2017 (UTC)

NDL via ISBN

Can I request the "National Diet Library" be added to the "Other Sites" navbar with the URL: http://iss.ndl.go.jp/api/openurl?rft.isbn=%s&locale=en (where "%s" is the ISBN)? This will remove the need to link to the NDL in many notes. The URL works with any correct ISBN-10 or ISBN-13 (hyphenated or not). I should probably also consider requesting other significant Japanese places like Amazon JP (there are many missing Amazon sites now; you can see how to link to them at ISFDB:Book sources which I updated for ISBN access via the wiki; see it in action with this example ISBN 4-15-010175-2), CiNii, and Webcat Plus but I can save that for another time. Thank you, Uzume 23:16, 4 April 2017 (UTC)

A number of editors have expressed concern about the "Other Sites" section of the navigation bar getting too long and making it hard to scroll down to the "Editing Tools" section. We may want to address this issue before we add even more links to it.
For example, we could try to convert the bullet list to a drop-down list. The list would be similar to the drop-down list main search box. Once you select your Web site of choice and click "Go", the software will take you to that site. The only catch is that it may be difficult to make it appear in a new window.
Another approach would be to hide the "Other Sites" by default and display a small arrow in their place. When the arrow has been clicked, the "Other Sites" list will appear. Ahasuerus 00:06, 5 April 2017 (UTC)
We could have a set of buttons with drop-down lists, e.g., an Amazon button with a site selector drop-down to select which Amazon site. The buttons could be styled to look like links so they could be in a menu and less obtrusive. Though I am against using target="_blank" you can put that on forms to make them open in a new window upon selecting a submit button (but the same can be had with ctrl+click as on links). The more interesting advantage of using a form is that the form would essentially be a redirection form (you submit it and it redirects you to the site) and as such we could track the usage of such clicks as they round-trip through the server. I am not sure I like this idea but it has interesting implications. Uzume 12:12, 5 April 2017 (UTC)
I support adding the NDL, and I think the drop-down list is a good idea. Instead having to click "Go", what about just having the list be a set of links? That way, you can pass target="_blank" through as a part of the anchor tag, and that will tell it to open in a new tab or window (depending on the browser preferences of the user). ···日本穣 · 投稿 · Talk to Nihonjoe 01:26, 5 April 2017 (UTC)
Menus can be created via JavaScript and/or pure CSS. I am not sure how that may or may not affect text browsers (we do still support such right?). The list could also be made smaller via abbreviations and acronyms (not sure that is the best idea but those who use such links often would obviously know). My personal recommendation would be to use CSS to hide large sections in a menu that appears upon mouse over or keyboard focus (e.g., hover over ISBN and menu appears and click destination choice). But this speaks to the possibilities of a full interface redesign where we could do the same with many longer menus, etc. I am not a fan of forcing links to open in a new window (i.e., target _blank). In most browsers the same can be had via ctrl+click and lets the user choose. We could also use another strategy. Instead of directly supporting such via the web application we could link to the wiki's ISBN handling page. It would require another click on a separate page but the bonus is the list could be very long with little issue (and the linking DB updated via wiki semantics). Anyway, that is why (discussion is needed) I only requested the one NDL change...for now. Uzume 01:59, 5 April 2017 (UTC)
There is a pre-existing FR to add Amazon CN, ES, IT, IN and JP to the list of supported sites. I have been hesitant to implement it for the reasons outlined above, so it's not just NDL.
As far as the implementation side goes, as long as we use "pure CSS" menus, we should be fine. The problem with using JavaScript on bibliographic pages is not just text browsers, it's the millions of people who browse the internet with JavaScript disabled by default for security reasons. We get a fair amount of traffic from search engines like Google, so we need to ensure that random visitors get a coherent view of our data even if they have disabled JavaScript. Ahasuerus 04:57, 5 April 2017 (UTC)
JavaScript can work just fine (many large organizations are using it) but you are right it needs to degrade gracefully in situations where it is unavailable in order to maintain accessibility (so we should not be adding significant content to the page in AJAXy ways but we can implement non-significant user interface content). And while we are on the topic of accessibility, we might also want to consider not just JavaScript vs. no JavaScript but also things like mouse vs. keyboard (or even screen readers, etc.). Pure CSS menus can have issues for keyboard users (depending on how they are implemented). From what I have read there are a few solutions. The older way is to implement mouse hover menus in CSS and hide it from keyboard access making keyboard access go through a more cumbersome (but still available) set of links elsewhere (via links to anchors farther down the page or on a different page). The newer way is to use WAI-ARIA role hinting and degradable JavaScript. For our purposes, I think our best bet might be to make the menus be farther down the page (like they basically are now) and put in anchor links to those (for accessibility) and then use JavaScript to relocate the menus to popups over the anchor links with mouse hover (and possibly keyboard focus), etc. for ease of use for users with JavaScript (which includes all editors as JavaScript is required for editing I believe). I would need to find a good example somewhere. Uzume 11:16, 5 April 2017 (UTC)
If you are concerned with space right now, one thing can already be done. You can kill Shelfari which is gone as it merged with Goodreads (which does not need the strange capital "R" in the middle). Uzume 13:53, 6 April 2017 (UTC)

(unindent) As per the discussion above, I have made the following changes:

  • Removed Shelfari
  • Changed "GoodReads" to "Goodreads"
  • Added NDL
  • Added Amazon JP
  • Added the Library of Congress
  • Alphabetized the list of sites under My Preferences/My Web Site

Ahasuerus 19:56, 6 April 2017 (UTC)

Thanks! I really appreciate that (I believe those were manual table updates as memory serves). I have a few comments though. Why did you choose to use ISBN-10 linkage with LCCN and NDL? They do not require such and the ISBN space now includes more than just 978 (mostly just some French works but it could change more). I know why it was done for Amazon JP (and the rest of the Amazon sites) since we are linking to their detail pages via ASIN (which is a 10 digit string which happens to match ISBN-10 in the vast majority of cases) instead of their search results (which can sometimes differ if using an ISBN-13). Also as an FYI, the European Library may be the next of our "Other Site" links to get the axe as though their links for the time being still seems to work, they have publicly announced they have shut-down their service as of the end of last year (which so far methinks means they have stopped taking updates but I expect it will go entirely offline at some point). Our BiggerBooks.com linkage (http://www.biggerbooks.com/bk_detail.aspx?isbn=<ISBN-13>) seems to be broken but can be fixed using: http://www.biggerbooks.com/book/<ISBN-13>. Oh and you can probably now kill the text "At least one Amazon site needs to be selected since ISFDB links to Amazon-hosted images." (and the code that ensures that) from mywebsites preferences as we now have image host credit links under the images. Uzume 21:19, 6 April 2017 (UTC)
Good points, I have updated the links. Too bad about the European Library. There isn't much we can do about it aside from keeping an eye on things.
As far as Amazon goes, it was something that Al requested some years ago. I think it may have something to do with Amazon's linking requirements, but I am not sure. Ahasuerus 01:03, 7 April 2017 (UTC)
I imagine the linking requirements are likely met by the added image credit links we implemented sometime back. It does not make much sense to depends on ISBN links when not all such images refer to books that even have ISBNs. Thanks, again. Uzume 07:25, 7 April 2017 (UTC)
I am not 100% sure, but I think Amazon's requirements are more along the lines of "If you link to our image, then you should also link to the book-specific page so that the user could navigate to our site and potentially buy the book." Ahasuerus 13:27, 7 April 2017 (UTC)
If that is true, I image we are in violation for any of our pub records that use such images for publications before 1966 (when SBNs were first created). I did a few searches (is is hard to search for dates before a given date with the web interface; I did not grab the DB and run an SQL query) and there are quite a few (there are more than 300 such from the 1950s alone). Uzume 21:47, 11 April 2017 (UTC)
True, although they are probably not too worried about 40+ year old books. Once we have support for third party identifiers (including ASINs), we will be able to link pre-ISBN books as well. Ahasuerus 15:44, 13 April 2017 (UTC)

Cleanup: Cover art for nongenre books

Discussion moved to R&S. --Vasha 20:36, 5 April 2017 (UTC)

Elizabeth Goudge & Pamela Cleaver

An editor has proposed deleting the non-genre works of Elizabeth Goudge and Pamela Cleaver as both being below the threshold. Given the variety of opinions of that term and the fact that each author has more than a handful of genre works, I'm listing it here for community input. If there is no input within the next day or so, I will accept the deletions. -- JLaTondre (talk) 19:27, 5 April 2017 (UTC)

I think Goudge has enough works to be considered "above the threshold". I'm iffy on Cleaver. ···日本穣 · 投稿 · Talk to Nihonjoe 19:43, 5 April 2017 (UTC)
I'm not sure all of Goudge's works we have listed are truly genre (several of her collections don't look like it from descriptions). -- JLaTondre (talk) 20:53, 5 April 2017 (UTC)
I'm not acquainted with their works, but if Goudge's works aren't what they seem, they should be deleted, as should the nonfiction works by Cleaver. Stonecreek 20:58, 5 April 2017 (UTC)
FantasticFiction says of Goudge: "Elizabeth Goudge was an English author of romance novels, short stories and children's books." Notice that there is no mention of fantasy (so far as I know, she never wrote any SF). I have always thought of her as a borderline speculative fiction author, and that's certainly not what she's known for. SFE lists 6 of her books as genre; we list 34. I'm fairly confident that SFE is closer to the mark than we are.
Pamela Cleaver is a children's author, who teaches writing for children. She views herself as a writer of romances for adults, and spec fic for children. (Her non-fic book on writing for children includes suggestions for how to write science fiction for children.) On her home page "Short Stories" link, essentially all of the content is about her published science fiction. For some of us here, her focus on herself as primarily a spec fic writer would put her "above the threshold". I tend to think of "above the threshold" as being someone who is well-known to the SFF community, has been influential on other authors, or might have SFF fans who collect them. Cleaver would fail on all of these counts. Chavey 03:41, 6 April 2017 (UTC)
The books of Goudge's that the Encyclopedia of Fantasy lists are The White Witch for adults and The Little White Horse, Smoky-House, Henrietta's House, The Valley of Song, and Linnets and Valerians for children. From the descriptions of her other books I have been finding, that seems about right, with the addition of the children's animal fantasy Serena the Hen and the Christmas story The Well of the Star a.k.a. David the Shepherd Boy. However, in some of her books she made heavy use of superstitions and legends as thematic elements or as narrative interludes -- a prominent example being Island Magic where not only is island life vaguely wrapped in myth, but a character's premonition plays a significant role. If we want to include those borderline cases, it would be difficult to figure out which ones are far enough over the border. As for her shorter work, The Fairies' Baby and other stories seems to have consisted either mostly or entirely of fairy stories in the mode of Cicely Mary Barker. She probably wrote quite a few other fantasy short stories; it would be possible to go through this biography and identify some of them. Vasha 17:18, 8 April 2017 (UTC)
Based on responses, I have accepted the proposed deletions. I have also added a note to Goudge's summary page regarding the non-genre issue. Please edit that note if you have suggestions on better wording. -- JLaTondre (talk) 18:26, 8 April 2017 (UTC)

Is Hermann Hesse really genre?

I wouldn't consider him in that way and thus delete his non-genre works. The majority of his works seems to lay outside our boundaries, but I'll appreciate any input before taking any action. Thanks for your thoughts. Stonecreek 20:48, 5 April 2017 (UTC)

Well, he is in the Encyclopedia of Fantasy with several paragraphs of discussion. Clute dicusses Steppenwolf as a work of fantasy, and others of his novels as marginally fantasy. And The Glass Bead Game is a famous utopia or conte philosophique. --Vasha 21:23, 5 April 2017 (UTC)
I don't think of him as a genre author, but SFE3 says:
  • [Demian, Siddhartha and Der Steppenwolf] in which Jungian depth psychology, Indian mysticism and Weltschmerz are perhaps overpalatably combined; these and others of his novels can be read – unwisely – to emphasize any fantasy elements, for at their core they are meditations on transcendence. ... [collections] assemble from various sources much of his short work, mostly fantasy, though some parable-like sf venues are evoked, not unpalely.
so we have to be careful. I think it would be best to keep his borderline SF works, but it would also be helpful to add notes explaining what kinds of speculative elements they contain. Ahasuerus 21:25, 5 April 2017 (UTC)
Obviously, I should better have asked if he's above the threshold, and to decide if we should keep his non-genre works. I'm asking because I'm reading through a copy of Knulp, which 1) is a COLLECTION in the first place, and 2) has no speculative content so far (after two of the three items - I'm guessing there'll show up no more).
At least over here there are dozens of other COLLECTIONS, OMNIBUSES & ANTHOLOGIES that feature one or the other of his works. Would we really like to catalogue all of them, even if there's no spec. fic. in them? Stonecreek 03:58, 6 April 2017 (UTC)
Although The Glass Bead Game is clearly genre, that is AFAIK, the only truly genre book by Hesse. Steppenwolf and Siddhartha are spiritual journeys where the visions the characters see are primarily spiritual visions, not the kind we associate with fantasy. In some cases, such as Craig Strete's "A Sunday Visit with Great-Grandfather", the spiritual vision is clearly meant by the author to be fantasy. But that's not Hesse's intent in these novels. It's fair to call them borderline genre, but ONE true genre book doesn't make Hesse a genre writer. Chavey 04:12, 6 April 2017 (UTC)
Thanks, Darrah! That's congruent with what the various secondary sources state. As for his shortfiction: quite a long time ago, I read through his collected stories, and I remember that there were only a few fairy tales, but the vast majority of his shortfiction was clearly non-genre. Stonecreek 15:14, 6 April 2017 (UTC)

Additional cleanup reports made available to all editors - 2017-04-05

Even more cleanup reports have been made available to all editors. Enjoy! Ahasuerus 21:14, 5 April 2017 (UTC)

"Other Sites" changes

I have made some experimental changes to the "Other Sites" section based on our discussion the other day. The links are now hidden by default. Users can access them by moving the cursor to the words "Other Sites", which have a big down arrow next to them. The rest of the functionality, including the ability to select and deselect sites on the My Web Sites page, remains the same.

Looking at the results, it occurs to me that we could move the list next to the "ISBN" field. The only reason we originally added it to the navigation bar was that we didn't want to clutter the main area. However, now that the list is hidden by default, it should no longer be a problem. We'll just display the words "Other Sites" and an arrow to let the user know that more information is available. Ahasuerus 00:00, 7 April 2017 (UTC)

Those are some interesting changes. Even if the external ISBN based links disappeared for some users (e.g., non-JavaScript capable ones), I would content they are not significant site content but just ease of use content (valuable but not significant). On the other hand, if links between pubs and titles etc. were not accessible by all users (regardless of their browser capabilities), I would consider the site broken (because they are more than just valuable they significantly define what we are about). Uzume 07:34, 7 April 2017 (UTC)
I should clarify that the changes are CSS-based and make no use of JavaScript. Even if you disable JavaScript for isfdb.org, your browser should display the page correctly -- as long as it supports CSS 2. Ahasuerus 13:24, 7 April 2017 (UTC)
I like the change. However, the links of the opened "Other Sites" menu now all have a bold font and the menu looks quite crammed as a result. I think it'd look better if they had no bold font-weight like the links in the other sections do. Jens Hitspacebar 18:56, 7 April 2017 (UTC)
Oh, right. Sorry, I didn't notice the bolding and the bullet change. It should be fixed now. Ahasuerus 19:42, 7 April 2017 (UTC)
This thing works as a charm on things where you can hover (aka my laptop) but there is no way to open them from my IPhone where hovering is a bit... impossible AFAIK (and I suspect other touchscreen may have similar issues). Annie 19:46, 7 April 2017 (UTC)
Hovering works, at least on my Fairphone (Android 4). Just tap on "Other sites" and the section opens. Close it by tapping on an area outside the "Other sites" section. Jens Hitspacebar 19:57, 7 April 2017 (UTC)
Well - I did not post without trying :) Not the same on Iphone 5s apparently :) Single tap does nothing; double tap is zoom in/out. No way to open the menu. Annie 20:01, 7 April 2017 (UTC)
(after edit conflict)Oh. I didn't occur to me that touchscreen devices didn't support hovering. I have installed a fix which is supposed to help, but I can't test it since I don't have an internet-capable touchscreen device at the moment. Could you please give it another shot to see if it worked? Ahasuerus 20:02, 7 April 2017 (UTC)
Still does not work here. Unfortunately I have the Iphone 5s only here so cannot try on another one Annie 20:10, 7 April 2017 (UTC)
Maybe it's a browser (Safari) problem and a different browser might help? I'm using Firefox on my phone but also just tried Android's stock browser (which is Google Chrome I think) and it works there too. Jens Hitspacebar 20:15, 7 April 2017 (UTC)
Apparently Apple has a special way of handling hovering. I have installed yet another fix -- hopefully Annie will be able to test it later today. Ahasuerus 20:33, 7 April 2017 (UTC)
Yeah, it is the built-in Safari. Still does not work on it though. :( Wonder if it is not because it is an older phone (although its OS is upgraded to 10.2.1 - I am seeing an upgrade available though so let me try to upgrade that and add another browser eventually and then we can keep troubleshooting. Will take awhile though - I am on my way to Europe in a few hours and planes are not a good place to play with your phone ;) Annie 21:01, 7 April 2017 (UTC)
Bummer. Oh well, one patch at a time... Ahasuerus 21:14, 7 April 2017 (UTC)
No hurry but to align with our templates we could also add the British Library to the national libraries we link to. See details here. Uzume 18:52, 12 April 2017 (UTC)

Smashword images

I image the answer will be "no" but for some reason I feel the need to throw it out there and ask this anyway. We have rights to deep link images from certain sites as enumerated here: Template:PublicationFields:ImageURL. Among them is Amazon. They grant such a right to their affiliates of which ISFDB is one. Smashwords seems to link it cover images from the DNS space of cloudfront.net. Can we use such image URLs since apparently cloudfront.net belongs to Amazon (as evidenced by this set of whois records)? It would certainly make adding the indie ebooks Smashwords publishes easier to add (they seem to allocate ISBNs to their ebooks unlike Amazon Kindle). Uzume 07:49, 7 April 2017 (UTC)

Cloudfront is a content delivery service sold by Amazon. Smashwords still owns the content, not Amazon, and they pay Amazon by the amount of data accessed. While a minor cost, Smashwords would still end up footing the bill if we linked to them. -- JLaTondre (talk) 10:59, 7 April 2017 (UTC)
Yes, I did find Amazon CloudFront. Uzume 13:12, 8 April 2017 (UTC)

Cleanup reports - performance

We have a lot of nightly cleanup reports. They are kicked off at 1am server time and take around 20-30 minutes to complete. While the reports are running, the system is sporadically unresponsive.

Not all reports are created equal. Here is how long each one takes to compile on the development server:

  • Report 1 took 9.875000 seconds to compile
  • Report 2 took 9.329000 seconds to compile
  • Report 3 took 6.177000 seconds to compile
  • Report 4 took 0.390000 seconds to compile
  • Report 5 took 0.515000 seconds to compile
  • Report 6 took 0.156000 seconds to compile
  • Report 7 took 0.484000 seconds to compile
  • Report 8 took 3.401000 seconds to compile
  • Report 9 took 0.639000 seconds to compile
  • Report 11 took 1.794000 seconds to compile
  • Report 12 took 0.078000 seconds to compile
  • Report 13 took 0.078000 seconds to compile
  • Report 14 took 2.403000 seconds to compile
  • Report 15 took 1.513000 seconds to compile
  • Report 16 took 17.893000 seconds to compile
  • Report 17 took 0.764000 seconds to compile
  • Report 18 took 0.734000 seconds to compile
  • Report 19 took 0.390000 seconds to compile
  • Report 20 took 3.385000 seconds to compile
  • Report 21 took 1.435000 seconds to compile
  • Report 22 took 0.094000 seconds to compile
  • Report 23 took 0.093000 seconds to compile
  • Report 24 took 0.421000 seconds to compile
  • Report 25 took 0.000000 seconds to compile
  • Report 26 took 0.000000 seconds to compile
  • Report 27 took 0.094000 seconds to compile
  • Report 28 took 0.094000 seconds to compile
  • Report 29 took 1.045000 seconds to compile
  • Report 30 took 0.281000 seconds to compile
  • Report 31 took 0.359000 seconds to compile
  • Report 32 took 4.461000 seconds to compile
  • Report 33 took 21.310000 seconds to compile
  • Report 34 took 11.045000 seconds to compile
  • Report 35 took 0.234000 seconds to compile
  • Report 36 took 1.154000 seconds to compile
  • Report 37 took 0.764000 seconds to compile
  • Report 38 took 22.989000 seconds to compile
  • Report 39 took 0.218000 seconds to compile
  • Report 40 took 2.077000 seconds to compile
  • Report 41 took 1.326000 seconds to compile
  • Report 42 took 3.120000 seconds to compile
  • Report 43 took 0.296000 seconds to compile
  • Report 45 took 3.292000 seconds to compile
  • Report 46 took 0.755000 seconds to compile
  • Report 47 took 43.652000 seconds to compile
  • Report 48 took 3.246000 seconds to compile
  • Report 49 took 1.591000 seconds to compile
  • Report 50 took 1.274000 seconds to compile
  • Report 51 took 0.203000 seconds to compile
  • Report 52 took 35.721000 seconds to compile
  • Report 53 took 0.093000 seconds to compile
  • Report 54 took 8.672000 seconds to compile
  • Report 55 took 0.843000 seconds to compile
  • Report 56 took 0.265000 seconds to compile
  • Report 57 took 0.234000 seconds to compile
  • Report 58 took 1.903000 seconds to compile
  • Report 59 took 1.872000 seconds to compile
  • Report 60 took 1.888000 seconds to compile
  • Report 61 took 1.887000 seconds to compile
  • Report 62 took 0.749000 seconds to compile
  • Report 63 took 3.389000 seconds to compile
  • Report 64 took 1.389000 seconds to compile
  • Report 65 took 0.579000 seconds to compile
  • Report 66 took 0.141000 seconds to compile
  • Report 67 took 0.670000 seconds to compile
  • Report 68 took 3.885000 seconds to compile
  • Report 69 took 38.446000 seconds to compile
  • Report 70 took 12.942000 seconds to compile
  • Report 71 took 0.702000 seconds to compile
  • Report 72 took 0.218000 seconds to compile
  • Report 73 took 0.016000 seconds to compile
  • Report 74 took 0.842000 seconds to compile
  • Report 75 took 0.266000 seconds to compile
  • Report 76 took 0.015000 seconds to compile
  • Report 77 took 0.000000 seconds to compile
  • Report 78 took 0.078000 seconds to compile
  • Report 79 took 3.370000 seconds to compile
  • Report 80 took 20.749000 seconds to compile
  • Report 81 took 0.031000 seconds to compile
  • Report 82 took 0.000000 seconds to compile
  • Report 83 took 0.421000 seconds to compile
  • Report 84 took 226.592000 seconds to compile
  • Report 85 took 0.115000 seconds to compile
  • Report 86 took 2.371000 seconds to compile
  • Report 87 took 5.320000 seconds to compile
  • Report 88 took 34.535000 seconds to compile
  • Report 89 took 0.655000 seconds to compile
  • Report 90 took 0.000000 seconds to compile
  • Report 91 took 0.718000 seconds to compile
  • Report 92 took 4.883000 seconds to compile
  • Report 93 took 11.871000 seconds to compile
  • Report 94 took 1.092000 seconds to compile
  • Report 95 took 1.077000 seconds to compile
  • Report 96 took 0.000000 seconds to compile
  • Report 97 took 4.691000 seconds to compile
  • Report 98 took 0.062000 seconds to compile
  • Report 99 took 36.972000 seconds to compile
  • Report 100 took 0.374000 seconds to compile
  • Report 101 took 0.265000 seconds to compile
  • Report 103 took 163.226000 seconds to compile
  • Report 104 took 0.000000 seconds to compile
  • Report 105 took 0.117000 seconds to compile
  • Report 106 took 0.004000 seconds to compile
  • Report 107 took 5.184000 seconds to compile
  • Report 108 took 0.108000 seconds to compile
  • Report 109 took 0.216000 seconds to compile
  • Report 110 took 0.002000 seconds to compile
  • Report 111 took 16.063000 seconds to compile
  • Report 112 took 0.289000 seconds to compile
  • Report 113 took 0.019000 seconds to compile
  • Report 115 took 0.681000 seconds to compile
  • Report 116 took 0.000000 seconds to compile
  • Report 119 took 0.413000 seconds to compile
  • Report 120 took 0.000000 seconds to compile
  • Report 121 took 0.016000 seconds to compile
  • Report 122 took 0.015000 seconds to compile
  • Report 123 took 0.546000 seconds to compile
  • Report 124 took 1.014000 seconds to compile
  • Report 125 took 1.014000 seconds to compile
  • Report 129 took 0.999000 seconds to compile
  • Report 126 took 1.014000 seconds to compile
  • Report 127 took 1.029000 seconds to compile
  • Report 128 took 0.999000 seconds to compile
  • Report 130 took 1.061000 seconds to compile
  • Report 131 took 0.998000 seconds to compile
  • Report 132 took 0.998000 seconds to compile
  • Report 133 took 0.998000 seconds to compile
  • Report 134 took 0.999000 seconds to compile
  • Report 135 took 0.998000 seconds to compile
  • Report 136 took 1.014000 seconds to compile
  • Report 137 took 1.030000 seconds to compile
  • Report 138 took 1.077000 seconds to compile
  • Report 139 took 1.092000 seconds to compile
  • Report 140 took 1.076000 seconds to compile
  • Report 141 took 1.934000 seconds to compile
  • Report 142 took 1.139000 seconds to compile
  • Report 143 took 1.778000 seconds to compile
  • Report 144 took 1.077000 seconds to compile
  • Report 145 took 0.640000 seconds to compile
  • Report 146 took 19.422000 seconds to compile
  • Report 147 took 0.203000 seconds to compile
  • Report 148 took 19.266000 seconds to compile
  • Report 149 took 19.235000 seconds to compile
  • Report 150 took 19.281000 seconds to compile
  • Report 151 took 27.362000 seconds to compile
  • Report 152 took 19.204000 seconds to compile
  • Report 153 took 19.344000 seconds to compile
  • Report 154 took 19.282000 seconds to compile
  • Report 155 took 19.234000 seconds to compile
  • Report 156 took 19.328000 seconds to compile
  • Report 157 took 19.298000 seconds to compile
  • Report 158 took 19.360000 seconds to compile
  • Report 159 took 19.297000 seconds to compile
  • Report 160 took 19.266000 seconds to compile
  • Report 161 took 8.315000 seconds to compile
  • Report 162 took 19.532000 seconds to compile
  • Report 163 took 19.500000 seconds to compile
  • Report 164 took 19.453000 seconds to compile
  • Report 165 took 19.609000 seconds to compile
  • Report 166 took 19.469000 seconds to compile
  • Report 167 took 29.827000 seconds to compile
  • Report 168 took 2.995000 seconds to compile
  • Report 169 took 0.687000 seconds to compile
  • Report 170 took 0.671000 seconds to compile
  • Report 171 took 0.686000 seconds to compile
  • Report 172 took 0.687000 seconds to compile
  • Report 173 took 0.686000 seconds to compile
  • Report 174 took 0.686000 seconds to compile
  • Report 175 took 0.671000 seconds to compile
  • Report 176 took 0.686000 seconds to compile
  • Report 177 took 0.686000 seconds to compile
  • Report 178 took 0.687000 seconds to compile
  • Report 179 took 0.686000 seconds to compile
  • Report 180 took 0.687000 seconds to compile
  • Report 181 took 0.671000 seconds to compile
  • Report 182 took 0.686000 seconds to compile
  • Report 183 took 0.718000 seconds to compile
  • Report 184 took 0.827000 seconds to compile
  • Report 185 took 0.764000 seconds to compile
  • Report 186 took 1.310000 seconds to compile
  • Report 187 took 0.952000 seconds to compile
  • Report 188 took 1.482000 seconds to compile
  • Report 189 took 0.094000 seconds to compile
  • Report 190 took 0.015000 seconds to compile
  • Report 191 took 4.602000 seconds to compile
  • Report 192 took 1.139000 seconds to compile
  • Report 193 took 23.541000 seconds to compile
  • Report 194 took 0.624000 seconds to compile
  • Report 195 took 0.733000 seconds to compile
  • Report 196 took 3.416000 seconds to compile
  • Report 197 took 3.417000 seconds to compile
  • Report 198 took 0.312000 seconds to compile
  • Report 200 took 4.333000 seconds to compile
  • Report 201 took 0.009000 seconds to compile
  • Report 202 took 0.016000 seconds to compile
  • Report 204 took 7.272000 seconds to compile
  • Report 205 took 0.001000 seconds to compile

One thing to keep in mind is that not all reports are run in the numeric order, e.g. report 180 may be run after report 200.

Here are some things that we could do to minimize the impact of these reports on the system:

  1. Optimize the reports to take less time to run. Although it may be possible to improve some reports significantly, it will take a fair amount of development time to tweak the code.
  2. Eliminate some reports. Once we wrap up the Wiki migration project and break the "lexical match" links between the database and the Wiki, we should be able to disable a few dozen reports.
  3. Move the most resource-intensive reports to a weekly schedule. The problem with this approach is that some logically connected reports, e.g. certain reports that facilitate the Wiki migration project, will end up on different schedules.
  4. Change the frequency of the "nightly job" to run once a week or perhaps twice a week.

My current thinking is that we could start with approach #4, then work on #1. #2 will happen a few months down the road once the Wiki migration projects has been completed.

Does this sound like a reasonable approach? If so, should we run the reports weekly or twice a week? Ahasuerus 18:02, 7 April 2017 (UTC)

Is there a way for the report to be run on the development server and just posted on the live one (update data, run report, post back on the live site)? Annie 19:23, 7 April 2017 (UTC)
It's technically possible, but, unfortunately, it's not feasible given how the development server works. Ahasuerus 19:47, 7 April 2017 (UTC)
Too bad :( It was just an idea. Annie 20:05, 7 April 2017 (UTC)
If not - then the above approach sounds like the most logical one. Weekly should be fine I think - Sunday is not a very busy day from what I had seen in terms of edits so it may be a good day for them :) Or you can run some statistics and find a calm time. Annie 19:23, 7 April 2017 (UTC)
6 PM US eastern time? Americans are making or eating dinner, Europeans are going to bed. --Vasha 23:08, 7 April 2017 (UTC)
Currently I experience issues with the site daily at around 6:30 am US/Pacific (currently daylight time so UTC-7) for about five minutes. Uzume 02:24, 8 April 2017 (UTC)
That's when the daily backups run. As you said, they take under 5 minutes, which isn't too bad. The nightly job takes over 20 minutes and is getting to be a nuisance. Ahasuerus 02:31, 8 April 2017 (UTC)
What about spacing them out? Even over a couple of days? Since many are small, that should reduce the perceived impact. If it goes to weekly, can the clean-up report page be changed so that each report button shows the remaining number of issues (and perhaps even hide the button if zero left)? That would be much more convenient than having to click each one to see if they have been resolved yet or not. -- JLaTondre (talk) 12:24, 8 April 2017 (UTC)
I was thinking of having a message at the top of the reports that say something like "X hours until report is regenerated" or similar. This would become useful if the reports are spaced out and/or run at different times (like some weekly, etc.). Uzume 13:08, 8 April 2017 (UTC)
If you send me pointers/source for the long-running reports, I'd be happy to take a look at them. --MartyD 12:27, 8 April 2017 (UTC)
Great! The majority of cleanup reports are in this module. Transliterations-related reports are in this module. Each query has a comment like "Report 85: Non-Latin authors with Latin characters in legal names", so it should be reasonably straightforward (hopefully.) BTW, I suspect that the main culprit is "and not exists". Ahasuerus 14:33, 8 April 2017 (UTC)

(unindent) I am happy to report that thanks to Marty's mad SQL skills 4 of the worst offenders have been brought to heel! :-) Ahasuerus 03:27, 16 April 2017 (UTC)

Great work: nightly_wiki.py nightly_update.py Uzume 19:33, 18 April 2017 (UTC)

Problem with Template:Author Image Data

It doesn't link correctly to Japanese names (such as 村田蓮爾). See here and try the link in the template. It will send you to this page, which is an invalid page. Somehow, it's not passing the correct escape characters for the kanji to Template:A. If the database could be adjusted to allow UTF-8 characters in the URL, that would correct the problem as it wouldn't need to use escape characters. There may be other ways to fix it, too. ···日本穣 · 投稿 · Talk to Nihonjoe 04:59, 8 April 2017 (UTC)

This is a problem in the Wikimedia software (we use a very old version) on the wiki side; not in the database. I have updated Template:Author Image Data to have a NameId field that can be provided in addition to the Name. If present, it will use that to link which bypasses the problem. I updated Image:Range Murata.jpg as an example. -- JLaTondre (talk) 12:39, 8 April 2017 (UTC)
This is an issue with Template:A (which Template:Author Image Data uses) and the fact that it is using anchorencode and not urlencode (see mw:Help:Magic words). We could use urlencode but as JLaTondre stated we are using a very old version of MediaWiki (see Special:Version) and ea currently requires spaces in names to be converted to underscores (not pluses as our old version of urlencode supports). A further issue is that the wiki uses UTF-8 while the DB application still uses ISO-8859-1. To properly link to the DB with an author name string would requires one to first XML encode the Unicode and then URL encode. Currently there is no viable option available. Probably the best solution would upgrade MediaWiki (so we can use urlencode with WIKI argument (so spaces are converted to underscores) and upgrade the DB application to serve content in UTF-8. Until then we continue to use the anchorencode hack for ASCII linking and can work around the issue for Unicode via author record ID linking instead. Uzume 13:58, 8 April 2017 (UTC)
There is also a possibility with updating the interwikimap and using interwiki linking syntax (which should do "WIKI" style urlencoding with spaces as underscores even in this older MediaWiki version) but we would still have to handle the Unicode issue. Currently the database tables have not been updated to handle Unicode (in any encoding) and we are using XML entities all the way into the database itself so the DB actually contains &#26449;&#30000;&#34030;&#29246; for 村田蓮爾 and the DB application just depends on the browser to handle the XML entity encoding (displaying and in sending via forms). We could serve UTF-8 HTML in the web application with a large amount of conversion into and out of the DB or do a single huge conversion and change the tables and all the table data to UTF-8. Uzume 14:31, 8 April 2017 (UTC)
Yeah, we really need to figure out how to update to the most recent version of the Wiki software. Too bad Al doesn't really do anything anymore. ···日本穣 · 投稿 · Talk to Nihonjoe 18:40, 9 April 2017 (UTC)
I have looked into this before and would be happy to do the work but I do not have access to the server to deploy it even if I get to the point where I have tested it extensively locally first. I believe we would have to make incremental jumps as there have been a number of changes along the way. The first minor jump would be to move to 1.12.0 (and get off the 1.12.0rc1). Next, I would likely target the last 1.12: 1.12.4. Then I can start looking at what it takes to move to considerably newer versions. Uzume 03:48, 10 April 2017 (UTC)
Well, it's up to 1.28.1 as of just a couple days ago, so we've got a ways to go. ···日本穣 · 投稿 · Talk to Nihonjoe 05:32, 10 April 2017 (UTC)
Yes, but I researched it once and I believe with a little planning and testing, some larger jumps can be safely made. Some will certainly require some tweaking before we consider taking them on however. FYI: in recent times they have made more releases so the number has moved more (not all releases are equally large in changes). Uzume 21:22, 11 April 2017 (UTC)
Well, maybe you and Ahasuerus can work together to do some testing and whatnot. I know a little, but I've never done it on a system with this much entered into it. ···日本穣 · 投稿 · Talk to Nihonjoe 21:28, 11 April 2017 (UTC)

Primary verifications -- work in progress

Please note that I am in the middle of redoing primary verifications. The first few patches should have no impact on the way the data is displayed, but if you see anything unusual (aka a bug), please let me know. Ahasuerus 00:12, 9 April 2017 (UTC)

The demise of SFF Net

As some of you probably know, SFF Net shut down on March 31. Our author pages have some 150+ links to the author bios which used to be hosted by SFF Net and are now defunct. We will need to re-link them to the last known snapshot taken by the Wayback Machine. For example, this page is a snapshot of Kevin O'Donnell, Jr.'s biography. Looking for volunteers! Ahasuerus 02:49, 10 April 2017 (UTC)

I can do it after this week. Some of them may have found new hosts for their sites, too, so each one would need to be researched to make sure. ···日本穣 · 投稿 · Talk to Nihonjoe 05:34, 10 April 2017 (UTC)
That is definitely true. I already updated some (even replacing them in cases where they obviously did move to different hosting; the ones that are now dead are somewhat of a bigger problem) but I am not sure I made much a dent in the list. Uzume 21:24, 11 April 2017 (UTC)

Variant Reviews

I recently noticed author pages do a strange things to variant review titles when the reviewed authors are cited differently but the reviewing authors are cited the same. Here is an example: T1956242 and T1282608 on A123788 (both are reviewed by "Edward Cox" and review the title "The City of a Thousand Gods" but in one the authors are "Marge B. Simon and Malcolm Deeley" and in the other "Marge Simon and Malcolm Deeley"). You will note that T1281964 looks good showing the review as having been reprinted. Uzume 13:58, 12 April 2017 (UTC)

Since Marge B. Simon is a pseudonym of Marge Simon, shouldn't the VT direction of the two reviews be reversed? Ahasuerus 15:12, 17 April 2017 (UTC)
It looks like someone merged them (which is probably not really a problem since it only changes the how the review credits the author or the reviewed work and not the actual reviewer), however the display in T1281964 is a little strange listing the review and that it is reprinted and then also listing the reprint below that. Uzume 17:41, 17 April 2017 (UTC)
Yes, it looks like a bug. I have created Bug 654 to record the problem and will take a closer look later. Thanks! Ahasuerus 18:03, 17 April 2017 (UTC)

Defining "Publisher" in a self-publishing world

I believe there has been some discussion on what constitutes a publisher in the past and I thought I would bring it up in light of self-publishing becoming more common due to the proliferation of on-demand printers, ebook distributors, etc. (e.g., CreateSpace, Kindle Direct Publishing Smashwords/Smashwords, etc.) Many of these seem to masquerader as publishers but what really is a publisher? In my mind, a publisher is just a literary publicist which happens to have a somewhat special place in that they are historically listed on the copyright page and thus can be used to identify publications. Part of this process is the definition of a literary product or edition and allocation of an ISBN (a standardized literary product/edition identifier).

I see CreateSpace (an Amazon company) as an on-demand printer (or on-demand fabricator; remember they started with DVDs as "CustomFlix"). Kindle Direct Publishing (though they have "publishing" in their name) does not directly publish anything (though they do distribute) as they are more of an online publisher's toolkit and actual publication is directed by the publishers that use them (much as a traditional publisher can choose among printers). Smashwords is similar (although they seem to require an ISBN vs. KDP allocating an ASIN).

I would like to suggest updates to Template:PublicationFields:Publisher to clarify self-publishing and exclude known on-demand printers, ebook distributors, etc. such as CreateSpace and Smashwords (and perhaps we can work on emptying CreateSpace and Smashwords). I would also go so far as to say I believe that in general (there may be some exceptional cases) we should not have "CreateSpace" or "Smashwords" in the publisher name at all (though a publisher comment that a specific publisher seems to publish exclusively via their services seems appropriate). Unless these are actually related imprints, this means renaming publishers such as Peculiar Productions / CreateSpace, Pink Fox Publications & CreateSpace and The Chakat's Den / CreateSpace and merging publishers such as Chimericana Books / Smashwords and Triskell Press and Smashwords. We could also consider our publisher records that contain the words "Amazon" or "Kindle" (though a brief look tells me that world might be a bit more complex). Thank you for your feedback, Uzume 18:43, 12 April 2017 (UTC)

Discussion on this started awhile back at Rules_and_standards_discussions#What counts as a publication for an e-book?, but no clear resolution. -- JLaTondre (talk) 19:35, 12 April 2017 (UTC)
I believe that one outcome of that discussion was to agree that CreateSpace should not be used as a publisher name. Converting existing ones will be quite a project though. And it hasn't really been agreed what to convert them to. I've been seeing some cases of the publisher space in such cases being left blank; that may not be a good idea because it doesn't make clear whether the blank means "self-published" or "unknown". As for using the author's name, that's clearly the thing to do if Joe Blow has (like some people do) set up a company named "Joe Blow, LLC". But do we want to be able to distinguish between such cases and people who are just self-publishing without a company? Vasha 20:56, 12 April 2017 (UTC)
Re: it doesn't make clear whether the blank means "self-published" or "unknown", that's what Fixer does at this time. There is just no way of telling without using Look Inside and even that is not always definitive because it only shows some of the pages. Even in the best case scenario, i.e. when all relevant pages are visible, a book can be clearly self-published yet the publisher is unknown. It's been a royal pain... Ahasuerus 23:24, 12 April 2017 (UTC)
Well, I guess the blank does mean "unknown" or "not certain what to say", then. I suppose I would rather have blanks that can maybe be filled in at some point than have the space filled by something wrong or misleading. --Vasha 23:31, 12 April 2017 (UTC)
I agree. I would rather see a field unpopulated than with some special "unknown" value. That said, I agree with "uncredited" (when it is known and it can even be a variant if we later learn who really did it; "unknown" would not make any sense in such a context). Uzume 23:49, 12 April 2017 (UTC)
If we're going to try to "define" a publisher, I'd like to suggest that we try for a definition that applies to how fanzines were published as well. The most common scenario is the common "self-publishing" model, where the editor made up the masters, stuck them in a machine, and turned the crank themselves. Sometimes they gave themselves and their living room a fancy name (which I think we should list). Sometimes they had a friend run off all the copies for them (occasionally, mailing the masters to a friend in another state, who took over the work). Sometimes it was a clubzine, so they took the masters to the club machine, and other club members helped duplicate, collate, staple, etc., and hence the club is often the "publisher". But in many of these cases, the "publisher" of record is often doing nothing more than the self-published "publisher": The fanzine editor has done all the work except for the duplication, and the "publisher" has done nothing but making copies of the thing they were handed. Yet that club, or that friend with a Gestetner, is still listed as the publisher. Chavey 05:17, 13 April 2017 (UTC)

I think it is not correct to insert the author's name and leave the publisher field blank, when there is a known entity that does some parts of the publisher's business. It's still different from self-publishing, when you do ALL of a publisher's tasks. Christian Stonecreek 02:09, 25 June 2017 (EDT)

I go by whatever my source states and document the source. I'm not sure why there's a desire to change ISFDB's rules & standards with regard to Template:PublicationFields:Publisher. I believe the goal is to make the process as objective as possible. We should not be guessing at how much work a publisher does and if we thus deem them to be a publisher or not.
Even in the dead tree world there are a wide range of services available from the publisher of record. Some of them were just printers or manufacturers. They shipped boxes of books to you. You did the marketing, sales, and distribution. Another model was you paid the publisher but they kept the boxes of books marketing, sales, and distribution. Yet another model was "on speculation" where there was no payment other than perhaps $1 to make it a valid contract. The publisher keeps the boxes of books and handles marketing, sales, and distribution. This is the model that modern ebook publishers use. Finally, there's the model where the publisher pays the author in advance, pays them a smaller percentage until the publisher's up-front costs are covered by book sales, and then pays a higher percentage.
To complicate things - while I mention "boxes of books" that normally is not the case within a publisher. Publishers often store unbound sheets and do the folding, cutting, and binding immediately prior to boxing and shipping. Sometimes publishers fold and cut and then store signatures. Related to this is that book jackets or pb/tp book covers are also printed and stored as sheets until needed. Often the presses for the covers were not in the same building (or even in the same country) as the book presses. There's quite a bit of USA tax law that's specific to manufacturing and publishing that covers how to value the parts of a product, how they can be written off, etc. I imagine the tax law in other countries is equally complicated.
With all of these, we go by whatever is stated in the publication. If nothing is stated we enter the name of the editor/author and add a pub-note explaining the issue. I noticed that the Publisher field rules are silent on the use of secondary sources. The rules for fields such as the Date mention secondary sources. I suspect at times people have used, and hopefully documented the use of, secondary sources for the publisher field even when doing a primary entry or verification. --Marc Kupper|talk 13:56, 26 June 2017 (EDT)
I see at least two complicating factors when debating whether printers like CreateSpace and Lulu, or ebook producers like Smashwords and Amazon Digital Services, do enough work to be listed in the Publisher field.
1) Sometimes the author of a book produced through one of these services gives themself a publisher name. Smashwords and Kobo list this publisher ("Silly Name Press" appears in the Publisher space), but Amazon insists on putting in CreateSpace or Amazon Digital Services no matter what. Should we be putting in the production company only when there is no other publisher name -- and you need to use Look Inside to find the other publisher if you're relying on Amazon.
2) If we list ebook producers, there will be multiple editions: one for Amazon, and at least one other if it's not Amazon exclusive, from Smashwords or other producers, listed on Kobo sometimes, not always...
I haven't yet made a comprehensive survey of Kobo's practices, but I see that they often list the author's name where Amazon has ADS. I already consult Kobo to find if an ebook has an ISBN. I think I am going to start citing them as an authority for the publisher name. I don't see a problem with that as long as I put it in notes ("publisher name according to Kobo; Amazon gives the publisher as Amazon Digital Services"). However, I'd be combining the Amazon and non-Amazon records into one, that way. I can see an argument in favor of keeping them separate. I don't like the fact that we are soon going to be importing Amazon ebooks automatically, because their online information omits data like publisher name and ISBN even when it is given inside the book-- I would rather have the data for the one "starting" edition be imported from somewhere more informative. --Vasha 15:44, 26 June 2017 (EDT)
Amazon stores all ISBNs, including e-book ISBNs, in its many databases. They just choose not to display them for e-books for reasons unknown. Fixer has a way of retrieving e-ISBNs based on ASINs, so all of our Fixer-entered records already include ISBNs for ISBN-enabled e-books.
The plan was to allow Fixer to enter e-books that truly do not have an ISBN as well. However, given the Amazon changes that I mentioned earlier today, it's not clear what's going to happen with Fixer. Ahasuerus 19:04, 26 June 2017 (EDT)
@Vasha When I add something to ISFDB based on the Amazon Look Inside I always add a pub-note explaining what it was I got from the Amazon Look Inside. I also normally provide the full URL to the source when it's on-line if the URL is not already part of the record and the date that I extracted the information. If you can get what looks like exactly the same ebook text (the same cover, same title page, same copyright page, etc.) from two or more sources then we'd have one record on ISFDB. If the publication is different then there's a different pub-record for each version. With that, I suspect we don't need separate records for people who distribute the text via various ebook file or data transfer protocol formats. There is a complication in that authors often do not submit books on the same day to the various ebook services. At present that means a separate pub note.
Fixer can still report on what it finds. If the publisher name is "Amazon Digital Services" then Fixer would use that for the publisher name and to include a pub-note that Amazon reported the publisher as "Amazon Digital Services" and that the publication's Look Inside needs to be reviewed to get the actual publisher name.
BTW, if the Amazon API reports on if a work is in Amazon's KDP Select program then that should be reported. Those stories are exclusive to Amazon and should not be showing up on sites such as Smashwords.
The main complication is accidental duplicate records are created. Someone looks at Amazon and adds a book as "Amazon Digital Services". Someone else looks, but also checks the Look Inside and adds a record for "My Big Press". At present we do not have the ability to merge verifications meaning once the duplication is discovered there's extra work chasing down verifiers and having them re-verify the remaining record. --Marc Kupper|talk 06:14, 27 June 2017 (EDT)

Note templates and abbreviations

It has been suggested that the new note templates be aligned on what is displayed. Currently we display a mix of abbreviations/acronyms and full length name/descriptions. One option that has been discussed is to use abbreviations/acronyms with full length name/descriptions in title attributes to create tooltip popups. Regardless whether we decide to go with the technical solution to implement both or not, we should standardize the naming. Here is a grouped list of current templates:

LinkTemplate Label
Internal A
Publisher
External ASIN ASIN
ASIN-UK ASIN-UK
BL British Library
BNB British National Bibliography
BNF Bibliothèque nationale de France
DNB Deutsche Nationalbibliothek
FantLab-authorFantLab author
FantLab-pubFantLab publication
FantLab-title FantLab title
JNB Japanese National Bibliography
OCLC OCLC
LCCN LCCN
NDL National Diet Library
SFBG-pub SFBG publication
SFBG-publisher SFBG publisher
SFBG-title SFBG title
Specific Internal Bleiler1 Science-Fiction: The Gernsback Years by Everett F. Bleiler and Richard J. Bleiler, 1998
Bleiler78 The Checklist of Science-Fiction and Supernatural Fiction by E. F. Bleiler, 1978
Clute/Grant The Encyclopedia of Fantasy, eds. John Clute and John Grant, 1997
Clute/Nicholls The Encyclopedia of Science Fiction, 2nd edition, eds. John Clute and Peter Nicholls, 1993
Currey Science Fiction and Fantasy Authors: A Bibliography of First Printings of Their Fiction and Selected Nonfiction by L. W. Currey, 1979
Miller/Contento Science Fiction, Fantasy, & Weird Fiction Magazine Index (1890-2007) by Stephen T. Miller and William G. Contento
Reginald1 Science Fiction and Fantasy Literature: A Checklist, 1700-1974 by Robert Reginald, 1979
Reginald3 Science Fiction and Fantasy Literature 1975 - 1991 by Robert Reginald, 1992
Tuck The Encyclopedia of Science Fiction and Fantasy through 1968 by Donald H. Tuck, 1974-1982
Specific External Contento Index to Science Fiction Anthologies and Collections, Combined Edition, William G. Contento
FantLab FantLab
Locus1 The Locus Index to Science Fiction
SFBG SFBG
SFE3 SFE3
None Tr Translated by

Notice things are far from balanced with some being labelled extremely tersely and other extremely verbose.

If we decide to adopt the tooltip approach, we could either add a new field (to support both label and popup title) or instead directly use the template name as the label and expand and use the current label at the popup title. The former requires extra code. The latter is problematic for some existing templates (though we could ignore templates without labels some are still an issue, e.g., "Tr" which is not really useful as a label).

By the way, why do we have an SFE3 and not an SFE3-entry? We could perhaps also do with a corollary FE (and FE-entry?) for linking to the Digital Edition of the Encyclopedia of Fantasy at: http://sf-encyclopedia.uk/fe.php

Thank you, Uzume 21:18, 12 April 2017 (UTC)

I am afraid I was so busy implementing template tool tips that I missed this section when it was posted 1.5 hours ago. I hope the current implementation addresses the issues that you raised in the second to last paragraph. I am not sure what you mean by "not an SFE3-entry". Ahasuerus 23:05, 12 April 2017 (UTC)
I was referring to something like:
  • 'SFE3-entry': ('http://www.sf-encyclopedia.com/entry/%s', 'SFE3'),
  • {{SFE3-entry|norman_john}}
to refer to the SFE3 "Norman, John" entry. I shall have to examine and update table based on the new changes. Uzume 23:42, 12 April 2017 (UTC)
Oh, I see. Yes, I agree that it would be desirable. The only catch is that SFE3's linking values are not the same as their display values, e.g. in this case it's "norman_john" vs. "John Norman". I'll have to revamp the template code to support this type of dichotomy. 00:32, 13 April 2017 (UTC)
Yes, if one wants to have a different display value for the relative URL (or identifier vs. the label and tooltip). I am not saying that is necessary in this case but it could be useful (and adding that kind of support would allow internal linking using record numbers while displaying names or titles, etc. too vs. how we currently link with A and Publisher using string match/searching). Uzume 06:37, 13 April 2017 (UTC)

Template tool tips

Templates have been changed to support tool tips. For example, if you hover over "SFE3" on Chris Foss's Summary page, you will see the full name used by the SFE3 project. If you hover over the templates on this page, you will see what "OCLC" and "LCCN" stand for. Some of the longer displayed template names have been changed to template codes.

It would be nice to make it clear that these tool tips exist, but I am not sure what the best way to do it would be. Ahasuerus 22:59, 12 April 2017 (UTC)

Are you referring to something akin to how "class" "hint" (for cursor selection) and the question mark icon are employed for transliterations? Some style could be applied. Uzume 06:43, 13 April 2017 (UTC)
That's right. I just haven't been able to come up with a unobtrusive yet clear way to indicate that the links are "hoverable". Ahasuerus 15:45, 13 April 2017 (UTC)
Maybe a little CSS like:
abbr a:link, abbr a:visited {text-decoration-line: underline; text-decoration-style: dotted;}
which would mimic most browsers abbr default styling, with a
abbr a:hover, abbr a:active {text-decoration-line: underline; text-decoration-style: solid;}
to indicate it's still a link, although users will still get the cursor change on hover as well. Albinoflea 17:43, 13 April 2017 (UTC)
Thanks, I'll experiment later today. Ahasuerus 18:14, 13 April 2017 (UTC)
I noticed your markup code is a bit strange. Using the examples you mentioned above I see things like
  • <abbr title="Third Edition of the Encyclopedia of Science Fiction"><a href="http://www.sf-encyclopedia.com/">SFE3</a></abbr>
  • <abbr title="Library of Congress Control Number"><a href="https://lccn.loc.gov/2014658071">LCCN 2014658071</a></abbr>
  • <abbr title="WorldCat/Online Computer Library Center"><a href="http://www.worldcat.org/oclc/806016149">OCLC 806016149</a></abbr>
This seems inverted. Should it not be something more like:
  • <a href="http://www.sf-encyclopedia.com/"><abbr title="Third Edition of the Encyclopedia of Science Fiction">SFE3</abbr></a>
  • <a href="https://lccn.loc.gov/2014658071"><abbr title="Library of Congress Control Number">LCCN</abbr> 2014658071</a>
  • <a href="http://www.worldcat.org/oclc/806016149"><abbr title="WorldCat/Online Computer Library Center">OCLC</abbr> 806016149</a>
With the current method, you are putting everything in an "a" tag inside of an "abbr" tag. If you are putting everything in, it almost begs the question why bother with the "abbr" tag (not to mention the identifier is not part of the actual abbreviation is it?)? On the other hand, if you put the "abbr" tag just around the abbreviated label that makes more sense (you could also link only the identifier but I am not sure that is for the best; besides in some cases like "SFE3" there isn't one). Uzume 23:13, 13 April 2017 (UTC)
As for style, why not something like abbr[title] {cursor:help} abbr[title]:after {content: url('/question_mark_icon.gif')}? Uzume 23:13, 13 April 2017 (UTC)
The reason I wrapped the whole link in the "attr" tag is that I wanted the mouse-over functionality to be available regardless of which part of the link the mouse hovered over. However, you are right that it defeats the original purpose of the "attr" tag. It's probably better to put the hover-over/"title" functionality inside of a "span". Ahasuerus 00:22, 14 April 2017 (UTC)
If you want the tooltip on the entire thing why not put it directly on the "a" tag link (no need to add an extra "span" tag)? That said, the tooltip and "title" attribute are really only applicable to the abbreviation (and thus the "abbr" tag). Uzume 00:46, 14 April 2017 (UTC)
Albinoflea's suggestion (dotted underlining for templates) has been implemented. I'll need to take a closer look at "abbr". Ahasuerus 18:17, 16 April 2017 (UTC)

British Library added to the list of "other sites"

British Library has been added to the list of "other sites" that we link to using ISBNs. Ahasuerus 23:19, 12 April 2017 (UTC)

I noticed this seems to link via ISBN-10 and not ISBN-13 (which the URL supports). Thank you, Uzume 23:43, 12 April 2017 (UTC)
FYI, the BnF link is broken and needs to be fixed to http://catalogue.bnf.fr/rechercher.do?motRecherche=%s. Uzume 12:59, 14 April 2017 (UTC)
Thanks, I will take a look once I deploy the "unlimited primary verifications" patch. Ahasuerus 03:57, 15 April 2017 (UTC)
BLIC and BnF have been fixed. Open Library had to be adjusted as well due to certain software changes. If you find anything unusual, please let me know. Ahasuerus 16:55, 16 April 2017 (UTC)

Why is it allowed to vote for variants/translations?

I wonder why it is allowed to vote for variants/translations. Shouldn't a vote be given for the work and not for each variant separately? Example: Cixin Liu's 三体 has gotten one vote and the English translation The Three-Body Problem has gotten two votes. Shouldn't casting votes be either disallowed for variants and centralized in the main parent title, or if allowed be aggregated into one single result for all variants of a title? Jens Hitspacebar 21:09, 14 April 2017 (UTC)

I can see someone reading one variation/translation and voting for that one, especially if they haven't read another one. Perhaps the database could combine all votes for the main work and all variations to create a new user rating? "Combined User Rating", perhaps? It would be interesting to see how different variations rated compared to the original work and the combined user rating. ···日本穣 · 投稿 · Talk to Nihonjoe 22:44, 14 April 2017 (UTC)
That's a good idea. Keep the votes separated per title as they are now but calculate an additional "Combined User Rating" across main parent title and its variant titles. Jens Hitspacebar 08:12, 16 April 2017 (UTC)
It's certainly doable. This may also be an opportunity to take another look at the way we display votes. As of Saturday morning we had:
  • 1,317,500 titles
  • 9,555 titles with votes
  • 382 titles with 5 or more votes
Considering the fact that we don't display the "User Rating" value unless we have 5+ titles on file, our voting data is largely hidden. Perhaps it would be better to always display what we have. The original concern was that it would give undue prominence to unrepresentative votes, but it could be mitigated by displaying the number of people who have voted for the displayed title. Something like:
  • User Rating: 8.65 (17 votes)
  • User Rating: 5.00 (1 vote)
 ? Ahasuerus 17:57, 17 April 2017 (UTC)
I like this idea. ···日本穣 · 投稿 · Talk to Nihonjoe 22:44, 17 April 2017 (UTC)
I like it too. Jens Hitspacebar 12:22, 18 April 2017 (UTC)
OK, FR 1049, "Make votes more visible and informative", has been created. Ahasuerus 16:21, 27 April 2017 (UTC)

Translator of Cornelia Funke's "Dragon Rider"

In verifying my copy of this book, I noted an interesting issue about the translator. In my American first edition, the copyright page says: "Original English translation copyright © by Oliver Georg Latsch", but then says "This translation by Anthea Bell copyright © 2004 by The Chicken House". I wondered if there were an earlier publication translated by Latsch, but WorldCat says no, the only earlier edition they know, the British edition, WorldCat says was also translated by Bell. As it turns out, there's an uncorrected proof copy currently for sale on Abebooks, and the credit says "Translated by Anthea Bell and Oliver Latsch". I'm guessing they didn't like Latsch's translation, hence they hired Bell, and I further suspect that the Latsch translation was never published. Consequently, I've promoted Anthea Bell from being listed as the translator of my publication to being the translator for this title record, where I've documented this issue. I mention this both because it's somewhat interesting, and just in case anyone knows a publication in this title (of which there are a dozen) that was NOT translated by Anthea Bell. Chavey 23:09, 15 April 2017 (UTC)

Server downtime - 2017-04-15 @8pm

The server will be unavailable between 8pm and 8:05pm server time. A new patch will be installed. The patch will change the way primary verifications work and allow an unlimited number of primary verifiers. Help will be updated later tonight. Ahasuerus 23:51, 15 April 2017 (UTC)

Done. If you run into any issues, please post your findings here. Ahasuerus 00:04, 16 April 2017 (UTC)
Great. It should be noted there are many (more than 500 by my quick estimate) pub notes that need to be updated that refer to a primary verification by number (e.g., "PV1", "PV2", "PV3", "PV4", "PV5" and "PVT" or "PV(T)"). Thank you, Uzume 14:34, 16 April 2017 (UTC)

sort by author in search

It would be nice to have the option to sort by author in advanced title & publication search. Ideally sorted by last name, first name. Vasha 10:25, 16 April 2017 (UTC)

I have considered adding it, but how should the sort work for titles/publications with multiple authors? Ahasuerus 12:58, 16 April 2017 (UTC)
First ones by Author A, then ones by Author A + Author B, then A+C, then A+C+D, then A+D.....
I'm not sure how complicated the sorting can get. Is it possible to put the authors within a publication in alphabetical order so as to make sure not to have some works listed as by A+B and some as by B+A? Vasha 14:53, 16 April 2017 (UTC)
I know that the more complicated you make it, the longer the running time. I would actually be happy with something ultra-simple that only approximately groups same authors together. --Vasha 18:57, 16 April 2017 (UTC)
Understood. I'll see what I can do. Ahasuerus 20:49, 16 April 2017 (UTC)

"Last User Edit/Verification" field added

The list of primary verifications displayed on publication pages has been enhanced to display the "Last User Edit/Verification" date for each primary verifier. Hopefully the new field will make it easier to determine which verifiers are currently active.

Ideally, the new field would also include each verifier's "last Wiki activity" date, but I haven't been able to find a quick and reliable way to find it in the Wiki database. Ahasuerus 20:48, 16 April 2017 (UTC)

In theory that should not be that hard (and should be similar to how the new messages on the user talk page works as that just compares the last activity on the talk page to the last user's activity). Uzume 00:53, 17 April 2017 (UTC)
I am certainly open to suggestions! :-) At first I thought that "user_touched" in "mw_user" would work, but no such luck. Ahasuerus 01:26, 17 April 2017 (UTC)
Try mw_revision. Select max(rev_timestamp) from mw_revision where rev_user = xxx. --MartyD 01:44, 17 April 2017 (UTC)
Ah, yes, much better. <installs a fix> Thanks! Ahasuerus 02:28, 17 April 2017 (UTC)
I just noticed this ... and was rather startled. As the dates are immediately to the right of people's verifications I assumed it was the last date those people edited/verified the pub record I was looking it. I was confused as I was looking at a record I had not touched in ten years and yet it said I'd last edited/verified it on 2017-06-04. I then worked on another pub record and looked at the first again, and was surprised to see the date for me was now 2017-06-08.
It also means that robots such as Google checking for changes will need to be re-indexing pub record continuously as most of them are verified by at least one active user. That can be a nightmare of someone is archiving each version of a page as the robot will think something on the page has changed when that's not the case. One suggestion is to not show this data if the person is not signed in. Anonymous users do not need to see the system wide "Last User Edit/Verification" dates on every publication record.
An improvement is to move "Last User Edit/Verification" information to a page that can also return <meta name="robots" content="noindex"> as the page should not be indexed by search engines. It could also include the various tags that hint that a page should not be cached. Links to "user verification details" pages should include rel="nofollow". This would allow us to make the "user verification details" messages more informative such as to include how many days (or months/years) ago the person was last active, the list of verifications by each user, etc. Another way to do this is to generate the dates using client side Javascript though I suspect some robots are now running a Javascript engine to get a view that's closer to what's visible to end-users.
"User verification details" can either be a separate cgi script or coded within pl.cgi as +uver that would work much like +c and +f. This would also cut down on the server load as it appears the existing dates are expensive to compute. At present I believe they are computed for each verifier every time a pub record is displayed.
Ideally, we would have a way to mark users as inactive. If someone dies for example, we don't need to wait out of a year of idle time. --Marc Kupper|talk 22:36, 8 June 2017 (UTC)
OK, I have changed the column header to "Last User Activity Date" -- see this pub. Hopefully it clarifies the intent. Ahasuerus 23:52, 10 June 2017 (UTC)
Thank you. I believe it's still quite mysterious from the POV of someone unfamiliar with ISFDB. It's something that's only of value to someone intending to query or notify the verifiers of something. That's why I was suggesting that the data be moved to a +uver version of the page. The rare person that intends to notify the verifiers about something cares about the date while everyone else doesn't. --Marc Kupper|talk 06:29, 14 June 2017 (UTC)
I agree that casual ISFDB users generally do not care about verifiers' "Last activity dates". However, it seems harmless and the data retrieval overhead is low. Ahasuerus 15:07, 15 June 2017 (UTC)
Semi-related, can the links be to the user-talk pages? --Marc Kupper|talk 06:29, 14 June 2017 (UTC)
At this time our bibliographic pages link to User pages while moderator-only pages link to User pages as well as to Talk pages. Before we make any changes, we may want to have a Community Portal discussion re: the desired behavior. Ahasuerus 15:07, 15 June 2017 (UTC)
As far as the issue of robots goes, how do they handle similar Web pages with dynamically generated data like date/time stamps? Ahasuerus 23:52, 10 June 2017 (UTC)
In thinking about it, Google seems to be crawling and re-loading all the pages on a site. I base that on a local WordPress blog where I sometimes want to find a post I sort of remember and so use site:... Google always returns dozens of irrelevant hits as the site has a snippet sidebar and Google is indexing that along with the main contents. Thus on that site all of the pages are changing constantly as they all have the most recent posts/comments in the sidebar.
If someone wants to reduce the server load caused by robots it seems their only option is to maintain page-updated time stamps. Some organizations do this with a reverse-proxy server. --Marc Kupper|talk 06:29, 14 June 2017 (UTC)

(unindent) What is the timezone for the date on "Last User Activity Date"? It seems to be different from what the whole server is using - see The Stone House foe example. I just did the verification and it is still 6/19 according to the timestamp. The Last User activity shows the 20th - even though based on the verifications timestamp, there are some more minutes from the 19th left. So one of the two dates is in a wrong timezone. Annie 03:27, 20 June 2017 (UTC)

The software checks the last date/time stamp of the following user activities:
  • primary verifications
  • secondary verifications
  • submissions
  • Wiki posts
I think the Wiki uses UTC, which presumably accounts for the discrepancy. It's not a big deal since we just need a ballpark estimate, but it's better to have it fixed. Thanks for identifying the issue! Ahasuerus 04:54, 20 June 2017 (UTC)
Aha - that explains the thing - it is the Wiki then. And yeah - it is in a different zone. Knowing that makes sense now. Thanks for the explanation. :) I agree - not a big deal - it was just... surprising :) Annie 05:49, 20 June 2017 (UTC)
Bug 675 has been created. Ahasuerus 15:30, 20 June 2017 (UTC)
I think I got it. If you see anything odd, please let me know. Ahasuerus 17:51, 20 June 2017 (EDT)

Nightly cleanup reports -- update

Here are the new timings. Only reports that took more than 1 second to complete are included:

Report 1 took 9.488000 seconds to compile
Report 2 took 9.001000 seconds to compile
Report 3 took 6.322000 seconds to compile
Report 8 took 3.619000 seconds to compile
Report 11 took 1.014000 seconds to compile
Report 14 took 2.366000 seconds to compile
Report 15 took 1.446000 seconds to compile
Report 16 took 16.035000 seconds to compile
Report 20 took 3.408000 seconds to compile
Report 21 took 1.481000 seconds to compile
Report 29 took 1.063000 seconds to compile
Report 32 took 4.533000 seconds to compile
Report 33 took 21.543000 seconds to compile
Report 34 took 10.743000 seconds to compile
Report 36 took 1.148000 seconds to compile
Report 38 took 21.580000 seconds to compile
Report 40 took 1.983000 seconds to compile
Report 41 took 1.290000 seconds to compile
Report 42 took 3.037000 seconds to compile
Report 45 took 3.136000 seconds to compile
Report 47 took 26.933000 seconds to compile
Report 48 took 3.195000 seconds to compile
Report 49 took 1.614000 seconds to compile
Report 50 took 1.295000 seconds to compile
Report 52 took 34.622000 seconds to compile
Report 54 took 8.457000 seconds to compile
Report 58 took 1.857000 seconds to compile
Report 59 took 1.828000 seconds to compile
Report 60 took 1.836000 seconds to compile
Report 61 took 1.847000 seconds to compile
Report 63 took 3.329000 seconds to compile
Report 64 took 1.366000 seconds to compile
Report 68 took 4.362000 seconds to compile
Report 69 took 44.199000 seconds to compile
Report 70 took 14.796000 seconds to compile
Report 79 took 3.217000 seconds to compile
Report 80 took 19.965000 seconds to compile
Report 86 took 1.103000 seconds to compile
Report 87 took 5.158000 seconds to compile
Report 88 took 3.617000 seconds to compile
Report 92 took 3.200000 seconds to compile
Report 93 took 11.676000 seconds to compile
Report 94 took 1.106000 seconds to compile
Report 95 took 1.077000 seconds to compile
Report 97 took 4.512000 seconds to compile
Report 99 took 35.509000 seconds to compile
Report 107 took 4.769000 seconds to compile
Report 111 took 14.773000 seconds to compile
Report 127 took 5.989000 seconds to compile
Report 137 took 7.227000 seconds to compile
Report 143 took 1.791000 seconds to compile
Report 144 took 1.058000 seconds to compile
Report 151 took 37.451000 seconds to compile
Report 161 took 5.103000 seconds to compile
Report 167 took 28.999000 seconds to compile
Report 168 took 3.050000 seconds to compile
Report 188 took 1.449000 seconds to compile
Report 191 took 4.427000 seconds to compile
Report 192 took 1.047000 seconds to compile
Report 193 took 23.072000 seconds to compile
Report 196 took 3.369000 seconds to compile
Report 197 took 3.375000 seconds to compile
Report 200 took 3.870000 seconds to compile
Report 204 took 6.525000 seconds to compile

Ahasuerus 13:47, 18 April 2017 (UTC)

It seems like you highlighted all the ones over 20 seconds. If I add all the highlighted ones up we are now down to 269 seconds (under 4.5 minutes). This is down from 721 seconds (12 minutes)—more than a 2.5× speedup. Of course that is a very rough estimate as I did not count the excluded 20 seconds or less of each report not highlighted (which can add up with many reports) but it shows that even with just a little targeted work serious progress can be made (of course as we improve things there are diminishing returns). Uzume 19:45, 18 April 2017 (UTC)
One thing to keep in mind is that the cleanup reports are not the only thing that the nightly process does. It also rotates the banner and rebuilds the "stats" reports. The "stats" reports take a fair amount of time to rebuild; they too are on my hit list. Ahasuerus 19:58, 18 April 2017 (UTC)
Yes, I know. I was just looking at that code. Do we have times on the tasks from nightly_os_files.py (I am guessing no)? Uzume 21:38, 18 April 2017 (UTC)
Not at the moment, but I plan to add some checkpoints. Ahasuerus 21:58, 18 April 2017 (UTC)
In terms of the reports, however, the next biggest issue is Report 69. I looked at this and I imagine it is related to the 250+ "like" matching clauses (joined by "or") in the query caused by "badUnicodePatternMatch" (the dictionary returned by "unicode_translation" has that many entries). Uzume 21:38, 18 April 2017 (UTC)
That's right. When I limit the report to just one "bad" Unicode character, it takes less than a second. The more characters I add, the longer it takes. Ahasuerus 22:02, 18 April 2017 (UTC)
I am wondering if switching to single (long) "regexp" pattern would be better. Uzume 21:38, 18 April 2017 (UTC)
It's certainly something to look into. If you happen to have the proposed regexp handy, I can test it on the development server. Ahasuerus 22:00, 18 April 2017 (UTC)
It would have to be built from the dict like we currently are. I have not tested this code at all but I imagine something like:
def badUnicodePatternMatch(field_name):
        return "{field} regexp binary '{regexp}'".format(field=field_name, regexp="|".join(unicode_translation()))
Uzume 00:09, 19 April 2017 (UTC)
I am afraid "format" was added in Python 2.6 while we use Python 2.5. Ahasuerus 23:41, 18 April 2017 (UTC)
Easily fixed:
def badUnicodePatternMatch(field_name):
        return "%s regexp binary '%s'" % (field_name, "|".join(unicode_translation()))
Thanks. Alas, using regexp instead of LIKE makes performance much worse :-( Ahasuerus 01:13, 19 April 2017 (UTC)
Why don't we upgrade? I believe the latest 2.7.13 is compatible (3.x of course is not). Uzume 00:09, 19 April 2017 (UTC)
There are many components that I would like to upgrade, but we'll need Al for that. He is the only one who has full server access and can roll things back in case an upgrade messes things up. I am hesitant to change Python, MySQL and MediaWiki as long as we don't have that level of access.
As far as compatibility goes, at one point I tried running the ISFDB software under Python 2.6, but it immediately ran into problems. I am sure they are fixable, but it's a lower priority than MySQL and MediaWiki. Ahasuerus 01:21, 19 April 2017 (UTC)
I thought I got that working once. I will have to try that again once I get my development system back up. Uzume 01:52, 19 April 2017 (UTC)
I have an idea for this one. I just haven't gotten to it yet. I think it needs to be split up to do something easy/cheap to whittle down the set, then do the more complex matching against that smaller set. --MartyD 01:50, 19 April 2017 (UTC)
Ah, you mean something like looking for &#768; instead of looking for that combined with letters in 12 different ways (A&#768;, E&#768;, I&#768;, N&#768;, O&#768;, U&#768;, a&#768;, e&#768;, i&#768;, n&#768;, o&#768;, u&#768;). And use that same sort of logic on the whole set and then whittle that down after that for complete matches once the set is already smaller. That makes sense. I might try to process the keys somehow first to get a set of patterns that will reduce the set first before running the entire thing. Uzume 02:06, 19 April 2017 (UTC)
Well, it takes less than a second to retrieve the 12K title records with "#&" in the title (select title_id,title_title from titles where title_title like '%&#%';). We could then use Python to hunt for "bad" Unicode characters, which should take only a few milliseconds. Ahasuerus 02:35, 19 April 2017 (UTC)

(unindent) Perhaps but methinks we can find what you are looking for (plus a few other wrong cases) by just doing this:

def badUnicodePatternMatch(field_name):
        # All of the keys are either a single XML entity mapped character or a single simple character followed by single XML entity mapped character
        # We only care about the single XML entity mapped character so we remove the lead character when it is not an ampersand
        # And then we remove duplicates by pushing the list into a (frozen) set
        # This substantially reduces the number of patterns to search for
        unicode_patterns = frozenset(key if (key[0] == '&') else key[1:] for key in unicode_translation())
        return ' or '.join("%s like binary '%%%s%%'" % (field_name, pattern) for pattern in unicode_patterns)

And this does not require changing the query to search inside of a search and still reduces the number of patterns from 248 to 44 (a reduction of more than 5.5×). It will also find other aberrant combinations like a combining character after a character that has no business being combined with because it just finds all combining characters used in the map (regardless of what it might be combined with). Uzume 03:41, 19 April 2017 (UTC)

The updated query finished in 6.24 sec, which is a very good improvement. I'll have to take a closer look to see what the new logic does with combining characters. Ahasuerus 03:53, 19 April 2017 (UTC)
The new patterns will for example find any occurrence of "&#768;" and not just the combinations in the table (so it will catch "B&#768;", etc.). The set of search patterns are used for reports 65 though 70 so it should reduce some other reports as well. Perhaps we need some reports of bad and suspect Unicode sequences on transliterations too? Another thing to look at is if each of those fields is indexed. For the problem report number 69 this is true (so indexing won't yield anything better), however, according to the schema documentation these could perhaps be improved: 65 by indexing publisher_name in publishers and 66 by indexing pub_series_name in pub_series. However, according to the reported times these are apparently not currently an issue. Uzume 04:32, 19 April 2017 (UTC)
I don't know that an in-depth discussion of coding-related issues is appropriate for the Community Portal, --MartyD 10:58, 19 April 2017 (UTC)
We should probably move technical discussions to another page. Development has been mostly inactive for the last few years, but it (or one of its sub-pages) could be resurrected. Ahasuerus 11:53, 19 April 2017 (UTC)
but one comment on the above: Reducing the number of patterns would help but will not produce the biggest win. The basic performance with those reports and badUnicodePatternMatch is it leads to scanning all of the titles multiple times (once per pattern), and that remains a fixed cost. The original performs 248 x 1.317M = 326.62M comparisons. Reducing to 44 does help, but is still 57.95M comparisons. And the cost grows significantly (+248 or +44 per title) as titles are added. The trick is to minimize the comparisons against that full set when we know the candidate set is very small. Using the titles one as an example, I would try something like:
select title_id from titles where title_title like '%&#%' and (... the "or" comparisons ...)
If MySQL's optimizer does the right thing, it should do one scan for the first pattern (so 1.3M comparisons) and then one comparison for each member of the set and for each specific pattern (so 248 x 11.2K = 2.78M or 44 x 11.2K = 0.49M). As it is functionally equivalent to the existing code, I would try it and see what the results are, then tackle reduction of the patterns if necessary. --MartyD 10:58, 19 April 2017 (UTC)
Yes, I was planning to look at the "and (...)" approach but I noticed we were already using far too many extraneous patterns to begin with. Ahasuerus's comment about doing it partially in Python (looking for "&#" in the title in SQL first) underscored the need for that but databases are designed to handle data efficiently so I figured it would be better to focus upon the SQL and I was already embroiled in writing the Python to boil set of patterns down. Uzume 12:48, 19 April 2017 (UTC)
1.7 sec on the development server and 2.3 sec on the production server! Great job, folks! :-) Ahasuerus 11:04, 19 April 2017 (UTC)
One thing I don't understand about the patterns is whether, say, all occurrences of "&#768;" are problematic or just the enumerated ones (B&#768;, etc). If it's all of them, then the trick of ignoring the letter-modified portion of the character to find the sequence is definitely a good approach. --MartyD 10:58, 19 April 2017 (UTC)
Combining Diacritical Marks are used by Unicode to allow entering accented characters as a combination of two characters. For example, consider this ISFDB title. The characters "è̂" and "à̂" are actually combinations of two separate characters: "è"/"^" and "à"/"^".
That is fine for display purposes, but what happens when Unicode has another character which is identical to the two combined characters? Inevitably, some users will enter it one way and some users will enter it the other way. The two versions will be displayed the same way, but searching will be broken because a search will find just half the matching entries.
That is why we have a translation table (unicode_translation) which normalizes "combining diacritics" at data entry time. The cleanup reports are just another way to make sure than nothing got past the translation table.
Uzume's approach is interesting because it also finds occurrences of combining diacritics which we currently do not trap. Something to consider. Ahasuerus 11:49, 19 April 2017 (UTC)
We can very easily do both with:
def badUnicodePatternMatch(field_name):
        # Reduce the set of bad Unicode to search for by finding all "combining diacritic" combinations, not just the ones we already know how to replace
        # All of the keys are either a single XML entity mapped character or a single simple character followed by single XML entity mapped character
        # We only care about the single XML entity mapped character so we remove the lead character when it is not an ampersand
        # And then we remove duplicates by pushing the list into a (frozen) set to substantially reduce the number of patterns to search for
        bad_unicode_set = frozenset(key if (key[0] == '&') else key[1:] for key in unicode_translation())
        bad_unicode_sql_patterns = " or ".join("%s like binary '%%%s%%'" % (field_name, bad_unicode) for bad_unicode in bad_unicode_set)
        # All the patterns contain XML entity mapped characters so optimize the SQL to find those (and throw away everything else) first
        return "%s like binary '%%&#%%;%%' and ( %s )" % (field_name, bad_unicode_sql_patterns)
Uzume 13:21, 19 April 2017 (UTC)

Banner rotation

I doubt the nightly banner rotate is much of a performance bottleneck for the nightly run, however, I thought I would point out it does impact performance (perhaps slightly) via a change in client bandwidth. It would probably make more sense to have every page with the banner add a nightly_banner.css and just update the CSS nightly instead of updating CurrentBanner and IsfdbBanner.jpg. The client browsers can then cache the images and only has to get tiny CSS file update. Of course we could also use CSS sprites and JavaScript (and then only those users would see the banner rotate) and could rotate it every hour or something then as there is no cost and users would see the rotate even without refreshing any pages. It it not a priority though. Uzume 22:12, 18 April 2017 (UTC)

I like this idea. ···日本穣 · 投稿 · Talk to Nihonjoe 18:39, 27 April 2017 (UTC)
Upon reflection I don't think the current system, which requires the nightly job to rotate banners, is optimal. We could modify the software to display the right banner based on the day of the month. Ahasuerus 19:12, 27 April 2017 (UTC)
How many banners are there? Is there a gallery we could look at? ···日本穣 · 投稿 · Talk to Nihonjoe 20:26, 27 April 2017 (UTC)
There are 10 banners. There is no way to view them as a gallery although it would be easy to implement. You can review them by accessing www.isfdb.org/IsfdbBannerX.jpg , where X is one of the following numbers: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Ahasuerus 20:42, 27 April 2017 (UTC)

Advanced Search changes - 2017-04-18

The software behind the Advanced Search has been revamped. For the most part the functionality remains the same. However, the new code will make it easier to add new ways of sorting the data like the recently requested ability to sort titles and pubs by author.

For now, Advanced Title Search and Advanced Publication Search have been enhanced to sort titles and publications by title and then by date transparently. For example, if you do a search on "title is exactly Heaven" using the default title sort, the results will also be sorted by title date.

Due to the scale of the changes, the likelihood of new bugs is high. If you see anything unusual, please report your findings here. Ahasuerus 21:41, 18 April 2017 (UTC)

ISFDB logo now clickable

Clicking the ISFDB logo in the navigation bar on the left now takes you to the ISFDB home page. Ahasuerus 00:04, 19 April 2017 (UTC)

Similarly, clicking the composite image displayed at the top of ISFDB pages now takes you to the home page. Ahasuerus 00:46, 19 April 2017 (UTC)
Why is the logo different on the home page (where it looks like a card catalog) than elsewhere (where it looks like a catalog card)? Uzume 00:28, 19 April 2017 (UTC)
I don't really know. I think that's how Al did it back in the day. Since he has an arts background and I am colorblind, I rarely question his design choices. Ahasuerus 00:46, 19 April 2017 (UTC)

And I wonder why we are using a GIF for that (I am sure it is historic). Uzume 00:28, 19 April 2017 (UTC)

Most likely! Ahasuerus 00:46, 19 April 2017 (UTC)
There's this one, too. I think it would be useful to have a link on the wiki side that took you to the actual main page of the site (http://www.isfdb.org/) instead of the wiki main page (http://www.isfdb.org/wiki/index.php/Main_Page). Can we add such a link to the Navigation menu? I'm not sure what to call it, unless we have a "Main Page" link and a "Wiki Main Page" link. ···日本穣 · 投稿 · Talk to Nihonjoe 16:43, 19 April 2017 (UTC)
I have added a link to the main ISFDB page, renamed the link which takes you to the main Wiki page and dropped "Random Page". How does it look? Ahasuerus 18:30, 19 April 2017 (UTC)
Awesome. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 04:09, 20 April 2017 (UTC)
Oh no!—the death knell of Random page. I know I won't miss it (you can still pull it out of Special pages if you do). In terms of looks do not forget the various skins, e.g.:
If you like one you can of course permanently select one from your Preferences "Skin" tab (and do not forget each also has a print version too). Uzume 18:52, 20 April 2017 (UTC)
This has been one of my pet peeves forever... Thanks for adding these links! Albinoflea 22:30, 19 April 2017 (UTC)
One pet peeve at a time :-) Ahasuerus 23:03, 19 April 2017 (UTC)

Advanced Author Search - e-mail

Advanced Author Search has been changed to allow searching for e-mail addresses. For example, you can search on e-mail address contains sff.net if you want to find all authors whose e-mail addresses are invalid now that SFF.net has shut down. Ahasuerus 18:58, 19 April 2017 (UTC)

Perhaps it would be good to simply run a job to remove all of those? They are invalid now, and most of them probably don't have a public replacement. Email is not a terribly important entry, anyway (at least IMHO). ···日本穣 · 投稿 · Talk to Nihonjoe 16:38, 20 April 2017 (UTC)
Some of them have public replacements, e.g. as per Lawrence Watt-Evans's Web page, his public e-mail address is misenchantedpress@gmail.com. Ahasuerus 16:46, 20 April 2017 (UTC)
I have replaced several of them already. I do not think a mass deletions is warranted. Uzume 18:04, 20 April 2017 (UTC)

Tangent

Is Tangent left out for some reason or does it qualify by the rules of acquisition? It should be noted there seems to be two such serials: the first was a short lived two issue fanzine in 1965 and the other a regular semipro printed SF/F review magazine (but has published some fiction) from 1975 until 1977. The latter restarted as a print magazine in July 1993 and went to web in 1997. More information is available:

  • the earlier fanzine:
    • GC Tangent 1965 (the earlier fanzine published by British Science Fiction Association and edited by Roger G. Peyton)
  • the later semipro review magazine:

It seems like at least the latter should be "in" to me (but the earlier also published some interesting fiction). Thank you for your feedback, Uzume 19:29, 20 April 2017 (UTC)

"Watching" an author page

One of the annoyances with moving all of our author bios over to the database is the loss, AFAIK, of the ability to "watch" a bio page. I could do that so long as the bios were on the Wiki side, so I could see if someone added some new information about an author, or deleted information I had put there. My understanding is that there is no way to do that with the author pages any more. Too bad. Chavey 03:38, 21 April 2017 (UTC)

That could be a cool feature. Basically, have it allow each user to create a list of authors they want to keep an eye on, including those they haven't verified. If it could also notify once a day of any new items for that author, or any changes to the author page, that would be even more awesome. That would keep it from being too resource intensive. ···日本穣 · 投稿 · Talk to Nihonjoe 19:46, 21 April 2017 (UTC)

Ahasuerus downtime -- 2017-04-21

Ahasuerus says that he may be unavailable until Sunday afternoon "for technical reasons". Humans, what can you expect?... Fixer 20:23, 21 April 2017 (UTC)

In Lands That Never Were

Hi -- I just heard from Gordon Van Gelder that [3] this is misattributed; he says it was actually published by Thunder's Mouth Press, not Four Walls and Eight Windows. I see Amazon has Thunder's Mouth Press. I don't want to submit a correction as it's been primary verified. If anyone has a copy or knows how to find a way to verify it online, would they correct it? I'm hoping to run the F&SF article on the main page of Wikipedia, and would like the bibliographic data to be as accurate as possible. Thanks for any help. Mike Christie (talk) 22:11, 21 April 2017 (UTC)

Did you ask Don Erikson? ···日本穣 · 投稿 · Talk to Nihonjoe 00:03, 22 April 2017 (UTC)
I have now; thanks for the nudge -- I should have done that first. Mike Christie (talk) 00:31, 26 April 2017 (UTC)

Title page enhancements

The "Publications" section of the Title page has been enhanced. If a title has translations or other variants associated with it, the "Publications" section will now display the following line:

  • Displaying all variants and translations • Do not display translations • Do not display variants or translations

Clicking "Do not display translations" or "Do not display variants or translations" will re-display the title page with the requested subset of publications as well as the option to "Display all variants and translations".

The new logic is still experimental. Let's give it a day or two and see if additional tweaks may be desirable. Ahasuerus 23:57, 23 April 2017 (UTC)

Nice enhancement! Chavey 05:35, 25 April 2017 (UTC)

When did transliteration become romanization?

Although transliterations support was added to many record types (I am still looking forward to them for title/work series), apparently all the help specifies this is only for use with romanizations (i.e., Latin transliterations): Template:TitleFields:TransliteratedTitle, Template:AuthorFields:TransLegalName, Template:PublisherFields:TransliteratedPublisherName. When did this become romanization only? Are there plans/interest in supporting other transliterations? Should I take this to rules and standards discussions or make a feature request to support such things? On a related note, if we really only support romanizations why are these fields labelled as transliterations; wouldn't it be better to correctly label them as romanization fields? Thank you, Uzume 15:51, 24 April 2017 (UTC)

In practice, they had been used also for Kana transliterations of Kanji titles/authors (for Japanese)... The help may be lagging behind on this though. Annie 16:15, 24 April 2017 (UTC)
It should be stated that the Japanese use of kana for collating, indexing and searching (used to equate different kanji forms) is not within the realm of transliteration but withing the realm of transcription (Transcription (linguistics) which is a very different thing). For example "山", a character meaning "mountain" (see wiktionary) can be pronounced different ways and cannot be transliterated, however, it can be transcribed in a context were the pronunciation is determined (e.g., to "yama", "san", etc.). Uzume 19:58, 24 April 2017 (UTC)
The use of the recently added "transliterated" fields has been largely limited to romanizations from the beginning. Certain other, less common, cases are also supported as listed on the Template:AuthorFields:TransLegalName page:
  • The author's working language uses multiple scripts. For example, Japanese names are usually written using kanji, but in certain cases hiragana or katakana are used. In Serbian, both Latin and Cyrillic are used, although the Cyrillic alphabet is considered primary. In Azerbaijani, the Perso-Arabic script, Cyrillic and Latin have all been used at different points.
  • The author has lived in different countries which use different alphabets/scripts. For example, Alexander Lomm lived in Russia as well as Czechoslovakia and wrote SF in Russian and Czech. Since we list Russian as his working language, the Russian form of his legal name, "Кличка, Вацлав", is currently entered in the Legal Name field. The Czech version of the name, "Klička, Václav", and the fully Romanized version, "Klichka, Vatslav", appear in the Transliterated Legal Name field.
Ahasuerus 16:20, 24 April 2017 (UTC)
Well I ask this because I recently had a moderator remove Japanese kana and Cyrillic transliterations (I had added) of an English working language author record (both canonical and legal names) and I was quoted the help that these are romanizations. This author does have translated publications in languages supporting those scripts. I think that is wrong and want to know how I can change that so we do support such transliterations (either rule-wise or technically). Should we be removing valid transliterations in certain scripts and why? This is not really clear based on the help vs. the above discussion so I wanted to clarify this and ask for getting support of having other transliterations that apparently are not mentioned in the current help. "In practice" and "use of the recently added 'transliterated' fields has been largely limited to" do not really spell "policy". Uzume 16:37, 24 April 2017 (UTC)
Cyrillic transliterations are tricky because of the different alphabets that had been created through the years. Which one would you use - the Russian, the Bulgarian, the Serbian, another one? A non-used by anyone subset that contains only letters that all of them share? The name as used in Russia/Bulgaria/Serbia/elsewhere (as opposed to an actual transliteration)? So if we open the door for these, we run the risk of making this a lot bigger project that it needs to be - if a Russian edition even shows up, there will be a variant. And no - I am not mistaking translation and transliteration - it is just that a "Cyrillic" alphabet/script actually do not exist on its own - it is a collection of scripts based on a common ancestor (pretty much like the Latin ones - but I do not see anyone arguing that an English name needs to be transliterated into the Polish variant for example as well)- and the transliteration by definition is conversion between scripts :) Annie 16:58, 24 April 2017 (UTC)
This is not much different than the case for Chinese characters used by various Chinese languages/dialects, Japanese, and (traditional) Korean and is one of the reasons Unicode differentiates between scripts and writing systems despite things like Han unification (there are some Japanese created kanji called kokuji). Also similarly there are numerous romanization systems for Japanese (not to mention other historic kana systems). Some of the more modern and common ones include: Hepburn romanization, Nihon-shiki romanization, and Kunrei-shiki romanization. I believe the case is similar for Korean (last I checked). So the "door" is already open with regards to romanizations because even from a single language there are many. Leaving the door open for romanizations but not other transliterations seems unbalanced and ill thought out. Uzume 17:51, 24 April 2017 (UTC)
The issue was considered back when transliteration support was first implemented. The decision to use romanization was a conscious one. Ahasuerus 18:28, 24 April 2017 (UTC)
If so I propose we correctly label the fields as romanizations and not transliterations since apparently that is consciously decided upon and correctly labeling them would be clearer. Uzume 19:06, 24 April 2017 (UTC)
As indicated in the previously mentioned Template:AuthorFields:TransLegalName, the field is used for more than romanizations. Ahasuerus 19:38, 24 April 2017 (UTC)
Template:AuthorFields:TransLegalName mentions Chinese romanizations and kana for Japanese but I would submit one cannot transliterate such scripts, however, they can be transcribed within a context and a proscribed pronunciation. It is easy to get transliteration and transcription confused. One can have romanization within transcription too, e.g. Yale romanization of Mandarin vs. Yale romanization of Cantonese. Clearly these use mostly the same Chinese script but different pronunciations. Uzume 19:58, 24 April 2017 (UTC)
I don't know enough about this issue to tell whether it affects what we are doing here. I'll ask Linguist to stop by. Ahasuerus 21:17, 24 April 2017 (UTC)
(resolving an edit conflict) The different romanizations still use Latin that people recognize. The Macedonian/Serbian letter Џ will be less readable for a Bulgarian/Russian user than the Latin variant. Same with the Bulgarian/Russian Я or Щ or the Russian/Belorussian Э - a Macedonian/Serbian will be better served by the Latin than having a "Cyrillic" transliteration with letters they do not know. There isn't a common way to move to Cyrillic. The Chinese example above is not relevant here - it is the reverse of what we are talking about. The difference between your examples and the Cyrillic one is that you are talking about multiple scripts in one language; the Cyrillic family is the opposite case. Unless if you are proposing that we need to add a transliteration in ALL Cyrillic languages/alphabets, it is just not practical. Or do you propose we transliterate into each Chinese dialect? Annie 18:43, 24 April 2017 (UTC)
I was never suggesting a move to Cyrillic just supporting Cyrillic transliterations where appropriate (and "where appropriate" is a different language exercise as you point out). I already suggested we should have some rules about which transliterations we support but I would like it to include more than just a blanket "romanization" (for romanization, e.g, see ALA-LC romanization). Uzume 19:06, 24 April 2017 (UTC)
As indicated in Template:AuthorFields:TransLegalName, we don't have a preferred romanization system. The field can be used for any and all romanizations:
  • [enter additional values in the new fields ... if...] There are multiple competing Romanization schemes for the author's working language. For example, there are multiple Romanization conventions for the Chinese language, including Gwoyeu Romatzyh, Wade-Giles, and Pinyin.
This was done by design. We had a number of discussions of romanization systems in 2012-2014 and decided against picking just one. Hence the ability to add multiple romanizations. Ahasuerus 19:41, 24 April 2017 (UTC)
Agreed, however, can I create my own home grown romanization scheme tomorrow and submit data for it? I was not proposing standardizing on just one but standardizing nonetheless. Uzume 20:00, 24 April 2017 (UTC)
I don't think this has been a problem so far, but it may be useful to expand Help to say something like "If you are not sure how to transliterate a letter or a word, you can consult the list of common transliteration schemes [somewhere]". Ahasuerus 21:05, 24 April 2017 (UTC)

(after edit conflict)

Let's consider the case of Lois McMaster Bujold, a widely translated author. Her works have appeared as by:
  • ロイス・マクマスター・ビジョルド
  • 洛伊斯.比约德
  • 路易丝·麦克马斯特·比约德
  • Loiz Mekmaster Bižol
  • לויס מקמסטר בוז`ולד
  • 洛伊絲.莫瑪絲特.布喬
  • Лоис Макмастер Буджолд
  • Lois M Bujoldová
  • Лоис Макмастър Бюджолд
  • Λόις Μακμάστερ Μπυζόλντ
and that's just the names that we have on file. We list these name as pseudonyms, but we don't enter them in the Transliterated Name field of the main author record. I don't see the value in adding them there: we would be looking at 156,000 author names times a few dozen alphabets, syllabaries, etc times multiple transliteration systems per alphabet. That's millions upon millions of values.
The reason why the "transliterated" fields were added in the first place was to help our users read/pronounce names and titles written in languages/alphabets/etc that they are not familiar with. Latin is the universally recognized alphabet and is sufficient for our purposes.
We have also overloaded this field for certain corner cases -- see above -- since we didn't have another place to put this information.
As far as the issue of clarity goes, I think that the current policy is explained clearly in the Help section that I quoted above. If there are any ambiguities, we can clarify them. Ahasuerus 17:03, 24 April 2017 (UTC)
I would suggestion "our purposes" could be better for helping our users out if we supported users of other languages. If we had Japanese kana transliterations that would help (and perhaps get more of them here) them read and pronounce Latin, Cyrillic and other text we use here. And the case could be made for Cyrillic language users, etc. Uzume 17:55, 24 April 2017 (UTC)
Japanese, Chinese, Russian etc children learn Latin characters when they study math, physics, chemistry and other subjects. It's a universally recognized alphabet at this point. Ahasuerus 18:26, 24 April 2017 (UTC)
Would the addition of other transliterations help some Japanese/Chinese/etc users use the site? Probably, although we would first need to add user preferences for transliterations and add "alphabet" labels to transliterations. Would the benefit justify the amount of time that it would take to change the software and to enter the requisite data? I don't think it would, at least not given our current resources and other priorities. Ahasuerus 18:32, 24 April 2017 (UTC)
Many from other countries do learn Latin scripts but certainly not all. I would say our user base is still highly European and North American concentric (where Latin-scripted languages are extremely predominate), however, I am not sure that we should consider that a feature and future goal of our project. I am not opposed to "eventually not now" answers. I realize we have priorities and limited resources (we have many FRs that are not being addressed yet too). The point is, I believe we should consider going that way (even if it is currently only an idealized future). Uzume 18:54, 24 April 2017 (UTC)
Well, there are thousands of things that would be "nice to have". Some of them have been proposed at various points in time, e. g. making the user interface and Help available in multiple languages. They may or may not be difficult to implement depending on the preferred design. I don't think repeated discussions of these issues (this one was debated extensively in 2012-2016) are worth our bandwidth. There may come a time when our resources and our list of priorities change, at which point we will revisit them. Ahasuerus 19:52, 24 April 2017 (UTC)

Let's not forget that this field is not only used for romanizations of non-latin alphabets, but also for providing alternate diacriticless versions of latin-alphabet titles that contain non-Latin-1 characters. That is useful, not for readability purposes (titles with the diacritics stripped off are the opposite of readable), but rather so that they can be found if people type them into the search box without diacritics. So that's already a deviation from the stated purpose of having the romanizations.

I think, maybe, it would be good to have a separate field for alternate titles -- this would not be displayed in mouseover, or visible to users, but it would be used for variants that might be searched for. This would be useful not only for allowing people to pull up Vonarburg's L'œil de la nuit whether they search for "oeil" or "œil", but could also be used for titles that have ampersands in them (who can remember whether to search for "and" or "&"?) and so much more. --Vasha 03:32, 25 April 2017 (UTC)

Filing collections and anthologies as non-genre

I do think that a title should be marked non-genre if the majorities of the titles collected in it are non-genre; is that assumption right? I'm asking because it seems that some titles we file as genre will likely turn out as non-genre if we apply this rule. In particular, I'm thinking of Vonnegut's Welcome to the Monkey House, which I reread right now, having made about the half of it (the first reading was long ago). I have the impression that it is regarded as a genre collection by many, but most items seem not to be in that category. Stonecreek 19:14, 24 April 2017 (UTC)

I am on board with labeling container titles as non-genre when the majority of the published contained titles are non-genre. Uzume 19:26, 24 April 2017 (UTC)

Maurice Level

A lot of Maurice Level's stories need the English translations connected to their French titles. I've done a bit of checking online but the information available is sketchy. What I did discover, though, is that it sounds as if most of the stories are non-speculative. Is there anyone here interested in working on that, tidying up the page and figuring out which are the speculative stories? --Vasha 19:04, 26 April 2017 (UTC)

Christopher Nuttall vs. Christopher G. Nuttall

If memory serves, a few weeks ago someone volunteered to work on reversing suboptimal pseudonyms and asked for more targets. I knew we had some, but I couldn't think of anything at the time. Well, here is one: Christopher Nuttall vs. Christopher G. Nuttall. Ahasuerus 19:03, 28 April 2017 (UTC)

So basically, Christopher G. Nuttall should be the main name? ···日本穣 · 投稿 · Talk to Nihonjoe 19:31, 28 April 2017 (UTC)
That's right. At this point most of his books have been published as by "Christopher G. Nuttall". Ahasuerus 19:42, 28 April 2017 (UTC)
I submitted a removal of the pseudonym relation. What do you think the quickest order of the next steps would be? I don't want to make more work than necessary. ···日本穣 · 投稿 · Talk to Nihonjoe 19:39, 28 April 2017 (UTC)
Unfortunately, there is no magic bullet at this time. Every existing VT needs to be broken and new ones need to be set up. Ahasuerus 19:43, 28 April 2017 (UTC)
There are three stages, and you have to wait for approval between each one:
1A. Break old pseudonym relationship; create the new one; transfer author info from the old canonical name to the new one.
1B. Go into the pseudonym page (Christopher G. Nuttall), click "view all titles by this pseudonym", open all those records, and break the variant relationship.
2. After approval of those changes, open all the records that remain on the Christopher Nuttall page. If a record has a publication, make it a variant of a new CGN title. If it has no publications, change the name on it to CGN.
3. Once no records remain on the CN page, go to the CGN page and use "Check for Duplicate Titles" to merge all duplicates.
This is not the method that necessitates the fewest approvals, but it is the easiest, most streamlined and automatic procedure. Additional steps would have to be added if there were translations, for example, but Nuttall is a simple case. --Vasha 19:54, 28 April 2017 (UTC)
Yes, that's pretty much it. I have approved the pseudonym removal submission, so we can proceed with the next step. Ahasuerus 23:44, 28 April 2017 (UTC)
Okay, I've removed all the variants and moved the info from one name to the other. Have to wait for those to be approved before I can revariant. ···日本穣 · 投稿 · Talk to Nihonjoe 02:28, 29 April 2017 (UTC)
I've now varianted all the titles which were credited only to Christopher Nuttall. Once those are done, all the ones left on Christopher Nuttall should be deleted since they will be titles without any pubs under them (since the titles have never been published under just "Christopher Nuttall" (without the "G."). There's no need to change the author and then merge them as that would be an additional step. I can do that once the currently-waiting items are approved. ···日本穣 · 投稿 · Talk to Nihonjoe 17:55, 29 April 2017 (UTC)
Actually, those titles that were formerly at the top level on the CN page may contain information that the CGN titles you unvarianted from them don't-- series, length, links to reviews, notes, etc. So don't delete them. You could check for differences and transfer the info over, but the change-name-and-merge procedure is simpler. --Vasha 18:51, 29 April 2017 (UTC)
The series information has already been transferred. I've submitted all the author changes. ···日本穣 · 投稿 · Talk to Nihonjoe 19:15, 29 April 2017 (UTC)
Okay, I think all of the final submissions are done. Just waiting for them to be approved so I can verify. ···日本穣 · 投稿 · Talk to Nihonjoe 19:34, 29 April 2017 (UTC)

Alex Irvine

Also, we may want to reverse Alex Irvine and Alexander C. Irvine. Ahasuerus 23:44, 28 April 2017 (UTC)

I've done step one for Alex Irvine. --Vasha 08:02, 29 April 2017 (UTC)
And it's all finished. --Vasha 19:53, 29 April 2017 (UTC)

Title Merge tweaks

The Title Merge page has been tweaked. If you are merging 2+ titles and at least one of them has a "Yes/No" flag (juvenile, non-genre, etc) set to "Yes", the page that lets you select the desired value will default to "Yes" instead of the first displayed value. Ahasuerus 19:32, 29 April 2017 (UTC)

The Geek's Guide to the Galaxy

Lightspeed often publishes interviews conducted as part of the podcast The Geek's Guide to the Galaxy. We have some of these in the database credited to The Geek's Guide to the Galaxy and some to David Barr Kirtley or to Kirtley and John Joseph Adams. Should these be linked as pseudonyms, and if so, how? --Vasha 00:02, 4 May 2017 (UTC)

Too bad it's not specfic...

...because it would be totally awesome to have this on the site. ···日本穣 · 投稿 · Talk to Nihonjoe 23:03, 5 May 2017 (UTC)

The cover makes it look like it could be a time slip romance, but apparently not... Ahasuerus 00:29, 6 May 2017 (UTC)
Yup. It's finger-lickin' good! ···日本穣 · 投稿 · Talk to Nihonjoe 19:08, 8 May 2017 (UTC)
It's been licked off amazon.com. I think it happened today as the most recent review is on June 13th.[4] When amazon deletes or disables a title it keeps the product image and reviews. --Marc Kupper|talk 06:52, 14 June 2017 (UTC)
Sad! I may know where a copy is if you ever want to read it. ;) ···日本穣 · 投稿 · Talk to Nihonjoe 07:24, 20 June 2017 (UTC)

New cleanup reports: invalid/unsupported HTML in notes and synopsis

We have 10 new cleanup reports which look for invalid and/or unsupported HTML in record notes and title synopses. The data will become available on Tuesday morning (server time.) I expect that the reports will find 1,500+ records with problematic HTML.

The vast majority of the records that the new reports will find contain malformed HTML: misspelled tags, improperly closed angle brackets, non-existent HTML tags, etc. In addition, they look for other types of HTML issues. The fact that we have been allowing arbitrary HTML in notes is a serious security issue which could potentially cause massive problems. For this reason the new reports also identify records with HTML tags and attributes which will be disallowed in the near future, including embedded style sheets and divs.

Here are the HTML tags that the reports currently allow. Tags without attributes:

  • abbr
  • b
  • blockquote
  • caption
  • center
  • cite
  • del
  • em
  • h1
  • h2
  • h3
  • i
  • li
  • ol
  • p
  • pre
  • q
  • span
  • strong
  • sub
  • sup
  • table
  • tbody
  • td
  • th
  • tr
  • u
  • ul

Self-closing tags:

  • br
  • hr
  • p

Tags with attributes:

  • a
  • table
  • td
  • th
  • tr

These lists will be tweaked and further restricted as we get ready to implement automatic HTML sanitizing at submission creation time. Ahasuerus 17:57, 8 May 2017 (UTC)

I'm making progress on them but sometimes the correction of errors is above my knowledge of HTML and its subtleties (after all I'm a COBOL guy). My feeling, apart that there are some frequent offenders with the same mistakes (probably a cut & paste effect) is that too much sophistication in the layout of the notes is never a good idea. Hauck 08:42, 9 May 2017 (UTC)
I went through 281 of these last night -- all but the publication records. There are 31 records remaining in the "naughty" lists, most of which are due to tags that I think you should accept (the comment tag and the font tag) or else tags that are acceptable but which you're rejecting because it's using attributes of that tag (lists with "type=" and "start=" to be precise). There's another batch of title recs that have embedded images, which seems like they should be admitted, possibly with limitations. And then there's one "meta" tagged record ("The Kiss"), whose purpose you may recognize when you see it. I don't know if that tag should be on the "good" list, or if there should be an "exceptions" list, or if there's some other solution. Chavey 13:00, 9 May 2017 (UTC)
I agree with Hauck on the issue of "too much sophistication" in the html being a problem, not a strength. Some of the records we had would have been impossible for a non-html expert to be able to edit or update, or even to read in the code format. Several of those in the title records I had to simplify dramatically, e.g. some (div with inset margins) tags being replaced with blockquotes. Chavey 13:04, 9 May 2017 (UTC)
Thanks for working on these! I have further tweaked the Title list, which is now down to 18 records. The first 6 are OL's with various "start"/"type" values, which can be easily replaced with HTML blockquotes. The 11 that follow are all img's, which can't be allowed because they leave us wide open to XSS attacks -- see XSS Filter Evasion Cheat Sheet for a very long list of possible attacks. We'll have to replace them with regular HTML links. The last one is the "meta" tag that you mentioned.
As a general rule, tags without attributes ("b", "i", etc) are easy to sanitize. As long as we have a white list of allowed tags, we can sanitize the input in three simple steps:
  • Convert all tags on the white list from <tag> to [[tag]]
  • Convert all special HTML characters like < and > to their HTML versions
  • Convert the white listed tags back to the HTML format
On the other hand, attributes are much more difficult to deal with and require complicated handling, which is time-consuming and error-prone. We will have to support valid href's for "a" tags, but otherwise attributes will not be supported. It will affect a couple dozen notes which use fancy tables, but it's a small price to pay.
HTML comments are another huge security hole and need to be disallowed.
As far as the "meta" tag goes, the easiest way to implement this functionality is to add a new template. We can call it something like "norobots" and it will be converted to HTML at display time. Ahasuerus 15:50, 9 May 2017 (UTC)
This looks like the right approach to me. I had changed a bunch of > and < characters to their HTML versions in the cleanup I did. I had done a couple of blockquote and "by hand" start/type attributes, but I left some of the easier ones in just in case those were options that could be more generally allowed. (I replaced all the related attributes for the "li" tag.) Doesn't surprise me that they can be abused, or that images could be, but I'm a little surprised to hear that comment tags can be as well. (I was thinking about using them to put in sources in some of the author notes. It would be a nice way to make them available without cluttering up the 'standard' presentation.) I like your idea about the "norobots" tag. I don't know whether the approach I used will actually work. When I've used norobots before, it's always been in the header, where meta tags normally live. Google hasn't re-indexed isfdb since I added that tag, so I don't know yet if it will work. But if it does, a template would be a natural way to implement it. Chavey 23:03, 10 May 2017 (UTC)
How about putting them after the {{BREAK}} template? All notes support the break syntax that sends whatever is after it to a new page. That way they are easily visible if you click on the link but do not clutter the main page. Annie 23:07, 10 May 2017 (UTC)
Excellent solution! I still don't think about the BREAK command enough.
I just completed all of the "titles with unsupported HTML", except for the image tag. I'll leave that for Ahasuerus to think about :-) . The most complicated of the lists was Asimov's The Road to Infinity, which I converted into a table. I think it managed to capture the original fairly well, without all of the extra blank lines before and after each sub-heading. I'll try to begin work on "Publications" tomorrow. Chavey 00:35, 11 May 2017 (UTC)
That's right, separating core information from our sources was one of the intended uses of the BREAK template.
As far as "fancy tables" go, it may be possible to preserve some of the existing functionality by creating one or more templates which will be converted to HTML at display time. Granted, as of Saturday morning, we had relatively few table-related tags with attributes: 16 "tables", 2 "trs", and 24 "tds". Still, if the cleanup effort finds that it's something that we can benefit from in the future, we can look for commonalities which could be used to create templates. Ahasuerus 01:17, 11 May 2017 (UTC)

(unindent) Re: the proposed {{NOROBOTS}} template. I have poked around and there are some issues associated with putting "meta" tags into the body of an HTML page. The core problem is that they are invalid under our (4.01) version of HTML, although they are apparently valid under HTML5. The resulting HTML may be displayed differently by different browsers, although user testing suggests that major browsers shouldn't be confused by it.

I am more concerned by the fact that Google looks for HEAD-only HTML tags in the BODY section and assumes that their presence may be indicative of the site having been spammed/hijacked -- see this post in which Matt Cutts, the head of Google's Webspam team, talks about some unexpected "meta" tags in the BODY section. Ahasuerus 18:00, 11 May 2017 (UTC)

It looks to me like they're only looking flagging certain specific meta flags as indicative of a hacked website, so I suspect we'll be okay. Since the "big" search engines will all be interpreting HTML5, I'm guessing there won't be any real problems with that, but I'm not sure how to be confident about that.
I finished off most of the publication fixes, leaving only the lists with attribute tags (which I can finish off tomorrow). There's one more 'font=red' tag, and I don't remember how we dealt with that in the title recs. Altogether, there are now 42 complaints. There also seem to be two spurious complaints (mismatched angle brackets and double quotes) that have cropped up on both reports without linking to any records. It might be that someone has beat me to correcting them on both days or, as I know has happened before, something odd is getting reported that isn't really there. Chavey 09:49, 12 May 2017 (UTC)
All done! The links in this cleanup report now show no complaints. The two editors whose work was most affected by these constraints have been informed about the changes made. (And one has already responded 'That's fine".) One other editor very regularly used a non-standard style of tag that's not in the html documentation, but was accepted by most browsers (putting a "/" *after* tags like ul, i, a, etc. to start the tag). There were literally *hundreds* of those to be changed. But I only looked at the dating of a couple of them, all of which were from a few years ago, so I didn't bother to tell that editor about the problem. We can wait to see if more such records crop up later before we mention it to him. Chavey 20:07, 13 May 2017 (UTC)
Many thanks! In the meantime, I have tweaked the logic some more and will deploy the new version in a day or two. It should find a few dozen bad "href"s. Ahasuerus 20:36, 13 May 2017 (UTC)

"Universe" and routine sequels by other hands

Is there a criterion for the creation of a superseries, commonly named "Universe", such as Neverland Universe and Oz Universe?

Those two are sprawling. Within the Neverland Universe subseries Peter Pan, and the Oz Universe subseries Oz, we do include some Sequels by Other Hands (Encyclopedia of Fantasy entry by John Clute), both authorized and unauthorized.

Those examples suggest to me that series Five Children Universe is a mistake, for what seem to be rather straightforward latterday sequels to three E. Nesbit novels.

  1. Cresswell 1994: "A sequel to E. Nesbit's children's classic, "Five Children and It". After the original five children went home, the Psammead ("It") was feeling lonely, but then a new family of children was sent to the White House."
  2. Saunders 2014: "A sequel to E. Nesbit "Five Children" books set during World War I."

--Pwendt|talk 22:59, 10 May 2017 (UTC)

I'd say that if there is more than one series in the same universe/story-universe (or a series and individual books), it is a Universe and we need a series to show the connection. How would you call the "Five Children Universe" one? The sequels are not properly part of the series (so adding them to the series itself is a bad idea) and not connecting them makes no sense. Annie 23:03, 10 May 2017 (UTC)
I think having an overarching series is good in this case. Since they are unofficial sequels, especially. ···日本穣 · 投稿 · Talk to Nihonjoe 23:08, 10 May 2017 (UTC)
I am not sure what the consensus is as of 2017, but in the past a number of editors felt strongly that "sequels by other hands" should not be put in the same series as the original books. Not all of our series are handled that way, e.g. consider The Swiss Family Robinson or Pym, but many are. Ahasuerus 01:23, 11 May 2017 (UTC)
Yeah, that's basically what I was saying (I think):
  • Overarching series
    • Main series
      • 1 Main book volume 1
      • 2 Main book volume 2
      • 3 Main book volume 3
    • Followup series by other authors
      • 1 Followup volume 1
      • 2 Followup volume 2
I suppose the numbering could be continued if the publisher indicates they are direct sequels (so instead of the Followup series starting over with 1, 2, it would be 4, 5). ···日本穣 · 投稿 · Talk to Nihonjoe 07:28, 20 June 2017 (UTC)

Shakespeare and his co-authors

Is there a criterion for booking William Shakespeare as a formal co-author of some latterday work. Do we, should we, indeed rely on the title page alone, as the manual suggests, or sometimes book a Shakespeare as a co-author "regardless"? (to be continued)

Recently I broke some relationships as variants --that is, freed some children-- of our 5 Shakespeare dramas (as SHORTFICTION, perhaps an exception to the general scope of the database). And added 3 such works to the existing series A Midsummer Night's Dream, which previously contained the Shakespeare drama and Carter short story alone.

Is there consensus that we should use series in this way? Certainly it will make the Shakespeare summary bibliography easier for visitors to read, and easier for editors to improve. If generally approved in advance, I will create shortfiction series led by the 4 other Shakespeare dramas (Hamlet, Macbeth, The Tempest, The Winter's Tale).

--Pwendt|talk 23:32, 10 May 2017 (UTC)

There's no exception or special criterion that I'm aware of. However, I'm not sure what you are asking about and so can't say if there is consensus or not.
Did you mean Shakespeare's Stories for Young Readers which is a publication of Shakespeare stories as told by E. Nesbit? The titles of the Shakespeare works have "(abridged)" appended to them as a way of denoting that these are not the same text as the Shakespeare versions of the works. The stories are co-authored as being by William Shakespeare and E. Nesbit. I wished that publication record had included a note about what's on the title page. There's an Amazon Look Inside available but it does not include the title page. I suspect the title page only has "E. Nesbit" as that's what's on the front cover. The publication record should also have had a note about that within the publication the stories are only preceded by it's title and do not credit the authors. However, it's clear from the author's introduction that she revised the Shakespeare stories so that they could be enjoyed by children.
Another way that publication could have been handled is to only credit William Shakespeare and to variant title the stories from E. Nesbit to William Shakespeare. That has the benefit of "Used These Alternate Names: William Shakespeare" appearing at the top of E. Nesbit's bibliography and "Used As Alternate Name By: E. Nesbit" appearing at the top of William Shakespeare's bibliography.
Still another way that publication could have been handled is to not credit Shakespeare at all on the theory that they are E. Nesbit stories patterned after Shakespeare works. The downside is it means someone looking at Shakespeare's bibliography would not know that the Nesbit version was available.
Had I entered this publication I likely would have used the variant title route as Nesbit and Shakespeare were not co-authors. The author's introduction makes it clear she intended to re-tell the stories for a young audience. The publication only credits Nesbit as the author on the cover. I'll assume the title page credit is the same. The individual stories do not have author credits which gives us some leeway when it comes to what we enter the publication record. --Marc Kupper|talk 18:30, 11 May 2017 (UTC)

Primary Verifications - new section is hard to spot

It used to be that Verifications was a single section with the primary verifications color coded in green. I suspect the color coding did not work well for color-blind people but it did make it easy to tell at a glance if a publication was primary-verified.

We now have two sections, "Primary Verifications", and "Secondary Verifications." Numerous times recently I've glanced at a page and thought "there are no primary verifications" as visually it's very similar to the Contents section that's above it. The PVs are now colored gray. Can we add some color coding to the Primary Verifications section? I'd be happy with the green that you used to use but am posting the request here to see if others have ideas. --Marc Kupper|talk 07:18, 11 May 2017 (UTC)

Sorry, my bad. I didn't think of the impact on the color scheme when redoing verifications because... well, I guess because I generally don't think in terms of colors. I'll happily change all primary verifications to green or anything else that our color-enabled editors may like! Ahasuerus 14:12, 11 May 2017 (UTC)
I think that green should be fine - and consistent with the table below it. Annie 16:19, 11 May 2017 (UTC)
Done! Ahasuerus 17:12, 11 May 2017 (UTC)
Did you loose the white background on purpose? See here for example. Annie 17:28, 11 May 2017 (UTC)
It seems to look OK under Firefox and Chrome. The HTML also validates when run against the W3 validator. Are you using a different browser? What kind of problem are seeing? Ahasuerus 17:47, 11 May 2017 (UTC)
Browser issue apparently - now it looks fine. The two verifications sections were missing their white boxes around them on Firefox 53.0.2 for some reason and even Ctrl+F5 could not get them to show... Sorry for bothering you :) Annie 17:53, 11 May 2017 (UTC)
No worries. The patch also fixed a pre-existing CSS bug; perhaps Firefox was confused until something forced it to flush one of its caches. Ahasuerus 18:03, 11 May 2017 (UTC)
Thank you. At first I was confused as the sections did not have a white background but think you made a change in the last few minutes and a sample of test cases looks good.
  • [5] Standard pub with no primary verifications.
  • [6] Standard pub with primary verifications.
  • [7] Chapbook pub with small Contents section and no primary verifications.
  • [8] Chapbook pub with small Contents section and with primary verifications.
--Marc Kupper|talk 19:44, 11 May 2017 (UTC)

ASIN-UK

I am currently working on adding support for "third party identifiers" like ASINs and LCCNs. It would appear that different Amazon stores (UK, FR, JP, etc) can use different ASINs for most of their products. However, e-books appear to be an exception: the same ASINs are apparently used by all of their stores. If true, it means that we probably won't need separate country-specific ASIN IDs (or Notes templates) once we add support for third party identifiers. The display will look something like:

ASIN B06Y1L3VRG: CA, DE, FR, JP, UK, US
COPAC 11111: Web view, XML view
LCCN: 98765 [note the different display format since there is only 1 link]
OCLC: 12345

Does this look about right? Or are there e-books that different Amazon stores list under different ASINs? Ahasuerus 18:41, 11 May 2017 (UTC)

There had been historically e-books available only in some of the regions and in this case the links won't work for the places where it is not valid. I am not sure if these days there is a case where the same ebook has different ASINs but I will check some of my older books tonight and see if I can discover something... Annie 19:02, 11 May 2017 (UTC)
I am sure some ASIN links won't work in some countries. On the other hand, the only alternative that I can think of would be to enter one ASIN per country. That would result in an awful lot of duplicate ASINs. It would also require a great deal of manual work in order to check what's available at each country-specific store. And, of course, Amazon can make a subset of the "missing" ASINs available at any time. Ahasuerus 19:25, 11 May 2017 (UTC)
True. Quick check on my earliest e-books show that the ASINs are the same on both UK and US... so I suspect we can assume so unless if someone knows something else. Annie 19:45, 11 May 2017 (UTC)
It would make things a little more complicated, but the duplicate ASINs could be addressed by having the database compare submissions of new ASINs against those already in use for the publication. If they are the same, it could auto-merge the duplicate ASINs (or create a cleanup report that would allow mods or other users to verify they are indeed the same product). This would help prevent the duplicate ASIN bloat you describe. ···日本穣 · 投稿 · Talk to Nihonjoe 19:47, 11 May 2017 (UTC)
I'm not sure what the issue is. We have long allowed for duplicate ISBNs. More recently logic was added to the moderator-approval page to highlight that an ISBN or ASIN was duplicated. Most of the time the warning is spurious as the record being added is for a different printing than existing records. People are encouraged to specify the sources for information that they add to a publication record. Ideally, they are already saying if the source was amazon.com, amazon.ca, amazon.co.uk, etc. rather than just "Amazon". If the notes say the source was amazon.com and someone later discovers that ASIN or ISBN is a 404 on another site then it's not on that site. Do we really need to add logic to note that at this date and time from this source IP address a particular URL did not work? Not all ISBNs are on all the Amazon sites. ASINs are no different.
BTW, is isfdb's logic aware of the ISBNs that only work on www.bn.com? Amazon uses ASINs for e-books. Barnes and Noble uses what look like ISBNs for e-books but they only work on www.bn.com and are a 404 on Amazon's sites. I can't recall the prefix though. --Marc Kupper|talk 20:38, 11 May 2017 (UTC)
The folks at B&N use what they call "BN IDs", which are 13 digits long and kind of look like ISBN-13s. They even used to call them "ISBN-13"s on their product pages. However, their IDs start with 294, which would make them invalid if they were ISBNs. B&N no longer calls them "ISBN-13s", e.g. see the "BN ID" field on this page. Ahasuerus 20:54, 11 May 2017 (UTC)
I guess the next question is whether we want to add "BN IDs" to the list of supported third party identifiers. At the moment, it includes the following types of identifiers:
  • ASIN
  • BL
  • BNB
  • BNF
  • COPAC
  • DNB
  • FantLab
  • Goodreads
  • JNB
  • LCCN
  • NDL
  • OCLC
  • Open Library
  • SFBG
but we can certainly add more. Ahasuerus 21:00, 11 May 2017 (UTC)
Thank you for the BN ID explanation and link to an example. I'm confused by some of those as they are web sites that index what's available elsewhere but do not directly provide copies of a story for reading. While I can see citing one of them as a source of information about a story I would never expect to enter their ID number in ISFDB's ISBN/Catalog # field for a publication record. What field is this thread about? --Marc Kupper|talk 21:28, 11 May 2017 (UTC)
It's about the new group of fields that I am currently working on, "external identifiers" -- see this FR. Adding software support for them will enable robots like Fixer to submit ISBN-less e-books. In addition, moving these identifiers from notes to a separate field will enable users to search the database by OCLC/LCCN/etc number as well as other Good Things :-) Ahasuerus 21:49, 11 May 2017 (UTC)

Does Wattpad count as being published?

I know we've decided that a story appearing on the author's own website does not mean it has been published and is not listed as a first appearance date in the database. Should appearing on Wattpad be treated the same? It seems similar to me. --Vasha 20:25, 11 May 2017 (UTC)

I see several troublesome aspects:
  • Many but not all authors use handles rather than names.
  • I could not see publication dates for the stories.
  • It's not clear if authors are allowed to revise or delete a story. Once we link to something would that content be stable?
Offhand though - there's not much to distinguish a wattpad.com from an amazon.com from someone putting something on their own web site. I believe Amazon.com now allows people to set a price of $0.00. It used to be the minimum was $0.99 but recently I was adding an author to isfdb and saw that some of her ebook editions were priced at $0.00. --Marc Kupper|talk 21:06, 11 May 2017 (UTC)
Sometimes $0.00 prices are promotional, but there are tricks that can force Amazon to set them to $0.00 permanently -- see this post. Ahasuerus 21:23, 11 May 2017 (UTC)
If you can buy it/get it, it is published in my book. I still think that we need to revisit the rules around what is allowed in terms of web content considering the changing realities of books publishing and maybe instead of checking sites one by one, we need to reopen that discussion... :) Annie 21:31, 11 May 2017 (UTC)

Sluggish times?

Hello, it seems that the db is particularly slow this morning. A general problem? Hauck 07:30, 12 May 2017 (UTC)

Sluggish for me as well for the last at least 30 minutes. Annie 07:41, 12 May 2017 (UTC)
We haven't had significant performance problems in a while, so I don't have any of my performance profiling tools enabled. If the issue persists, I can enable them and see when the spikes occur. Ahasuerus 14:05, 12 May 2017 (UTC)
It went back to normal in about a hour. Hauck 15:11, 12 May 2017 (UTC)

External identifiers - preliminary cleanup

As indicated earlier, I am currently working on adding support for external identifiers (OCLCs, LCCNs, ASINs, etc.) I need to clean up certain things behind the scenes before I can add the new functionality.

I plan to deploy a few minor patches over the next day or two. They won't change anything that editors normally see, but it's possible that they may introduce new bugs. If you see anything unusual, please post your findings here. Ahasuerus 22:55, 12 May 2017 (UTC)

Seiun Awards for foreign works

For Best Translated Long Story and Best Translated Short Story, the entries were put in before we had Japanese support for titles. Should I go through and change the awards to point to the Japanese title since it was the Japanese translation that got the award? This may involve me creating a new variant entry in some cases, but that's fine. ···日本穣 · 投稿 · Talk to Nihonjoe 05:50, 13 May 2017 (UTC)

The software fully supports awards given to variant titles/translations, so I don't think there should be a problem. Ahasuerus 16:54, 15 May 2017 (UTC)
Not really a problem, but: this can result in a mix of original and translated titles on the author's award page. Example: Wolfgang Jeschke's awards, see The Cusanus Game and Das Cusanus-Spiel there. That was one of the reasons why I linked the award records for Best Foreign Work of the Kurd-Laßwitz-Preis to the original title and not to the translation. Another reason was, that the award is given for the work, not the translation. However, a disadvantage of the way I did it is that the titles on the award year page become a mixture of languages, for example German, Chinese and English on the 2017 Kurd-Laßwitz-Preis page (see Cixin Liu's entry there), which deviates from the way award nominations and results are usually published (using the titles of the translations) by the award committees. Maybe the software could add a "translation of TITLE X" for award records where applicable. Jens Hitspacebar 17:42, 15 May 2017 (UTC)
In this case, the award is specifically for the translation. Hence, "Best Translated X". ···日本穣 · 投稿 · Talk to Nihonjoe 17:55, 17 May 2017 (UTC)
Not sure what you want to say. That's what I was talking about as well above, in the sense of "award given to an author for his work which got translated into the award's language" (and not in the sense of "award given to the translator for the quality of the translation"). Or do you mean something different? :) Jens Hitspacebar 18:53, 17 May 2017 (UTC)
I'm saying the award is given to the translated work, not to the original work. As I've never attended one of their ceremonies, I don't know if they specifically call out the translator for doing a good job. ···日本穣 · 投稿 · Talk to Nihonjoe 19:27, 17 May 2017 (UTC)
Interesting points. Let me take a closer look at the software... Ahasuerus 18:32, 15 May 2017 (UTC)
(after sleeping on it) Since title awards can be linked to VTs, I think the parent data should be displayed on all award pages, including the author-specific Award Bibliography page and the Award Year page. For translation, it will say "translation of ..." while other types of variants will say "variant of ...", which is how the Content section of the Publication page handles it. Ahasuerus 17:37, 16 May 2017 (UTC)
Sounds good. Jens Hitspacebar 08:47, 17 May 2017 (UTC)
OK, FR 1058 has been created. Once I finish implementing "external identifiers", I will take care of the minor FRs and bugs that have been piling up for the last few weeks. Ahasuerus 15:47, 17 May 2017 (UTC)
Sounds good. I will work on switching the awards over to the correct titles. ···日本穣 · 投稿 · Talk to Nihonjoe 17:55, 17 May 2017 (UTC)

Unsupported HTML in notes/synospes - Bad HREFs

The recently implemented "invalid/unsupported HTML" reports have been updated to look for malformed HREFs. The data (25 pubs and titles) will become available tomorrow morning. Ahasuerus 16:35, 15 May 2017 (UTC)

List "Variant Title" and "Translation" on author's page separately?

Just a minor thing, but with the title pages now having variant titles split up nicely into the "Variant Titles", "Translations", "Serializations" and "Serialized Translations" sections, shouldn't the same distinction be made on the author's page to make it all more consistent? It'd first list all "Variant Title" records of the parent title as bullet points, then all "Translation" title records etc. Jens Hitspacebar 21:11, 15 May 2017 (UTC)

Well, the Title page is a little different. With a table, we can have up to 4 columns and then we sort individual VTs chronologically within each column. I suspect that a similar layout would look very busy on the Summary Bibliography page, so we are stuck with the current list layout.
We could split VT lists into 4 sections (VTs, translations, etc), but then we would lose the chronological ordering. If we decided to do that, I think the best section order would be:
  • Variant Titles
  • Translations
  • Serializations
  • Serialized Translations
Ahasuerus 18:22, 16 May 2017 (UTC)
Yes, that was what I had in mind: keep the bullet lists but organize them into the sections you mentioned above, one below the other. Changing the whole layout to a table-based would be over the top. Another, even simpler solution which keeps chronological ordering would be to just replace the "Variant Title" label with "Translation" where applicable and only split "Serializations" into "Serializations" and "Serialized Translations" ("Serializations" is already a separate section with its own chronological ordering anyway, therefore splitting it into two sections shouldn't be such a big change for users). I think the latter solution is sufficient because it'd implement what my main concern was: label it as a translation and not as a variant title if it is a translation. Jens Hitspacebar 08:45, 17 May 2017 (UTC)
Sounds good. FR 1059 has been created. Ahasuerus 18:14, 17 May 2017 (UTC)

Publication summary display issue

The standard practice is to have the date of a variant title be the first date that title+author name combination appeared. That's fine, except it leads to the following issue: On publication summary pages, contents items that differ from the canonical title in title and date have (variant of CanonicalTitle Year) appended to them; contents items that differ in only author and date have [as by Pseudonym] appended to them. So the original publication year is not displayed in the latter case. It would be nice to come up with a way of having the original year there. I'll admit I can't think of one at the moment. --Vasha 22:15, 19 May 2017 (UTC)

Further thought: We are already treating translations and variant titles differently for many purposes. So maybe in a table of contents things should be displayed like this: VariantTitle • (OriginalYear) • length by CanonicalAuthor (variant of CanonicalTitle) [as by Pseudonym] OR in the case of translations, TransTitle • (TransYear) • length by CanonicalAuthor (translation of CanonicalTitle OriginalYear) [as by Pseudonym]. So for variant titles in the original language, the date of the variant wouldn't be displayed here, but I generally find it confusing having it in tables of contents anyway. --Vasha 22:27, 19 May 2017 (UTC)

Doctor Who Audio plays/novels/audiobooks

Are the Big Finish Doctor Who audio plays/novels/audiobooks (call them whatever you want - different places call them differently) eligible for inclusion? We already have a few in Original Audiobooks (Big Finish). I do not think that they are any different from their BBC counterparts (which are an extension of the printed series and as such eligible I think -- we have a few of them over here and the only difference between these and the printed series is the formats... I think that they should be eligible but wanted a confirmation before I get this series sorted out and get a lot more of these added. Thanks! Annie 20:14, 22 May 2017 (UTC)

And if the decision is that they are not eligible, some of those above will need to be zapped :) Annie 21:13, 22 May 2017 (UTC)
That Original Audiobooks (Big Finish) series looks like a mixture of some eligible and some non-eligible. The title of the series is misleading because some of those included are full cast audio dramas, not books, such as The Land of the Dead, The Apocalypse Element and the 'Gallifrey' titles. These don't qualify for inclusion and I remember removing about 50 of the audio dramas a few years ago after some discussion (including the first two above titles that have since reappeared). As for the rest, I expect titles that are indicated as "read by..." on their covers are eligible for inclusion as audiobooks, but it will also be worth checking if there is an actual book that serves as the source for the audio. I'll happily remove the dramas. PeteYoung 13:25, 24 May 2017 (UTC)
Let's remove the dramas then. :) There are some of the audio books that were never published as printed books but are novels actually and are part of official novels lists (don't get me started - confusing does not start covering it). I guess I will need to go and look through all those disks and figure out which ones are what. Single reader - eligible; multiple readers - ineligible (except when it is a multi-cast reading of an existing text). Is this what we are converging to? I am happy with this distinction. Annie 16:50, 24 May 2017 (UTC)

If Magazine now online

Of interest to many editors here, the complete run of If is now available for free in multiple formats at the Internet Archive. You're welcome. PeteYoung 12:39, 24 May 2017 (UTC)]

Oooooh. ···日本穣 · 投稿 · Talk to Nihonjoe 19:30, 24 May 2017 (UTC)
Hey, I was planning to sleep some time this decade... :) Annie 19:51, 24 May 2017 (UTC)

Server downtime at 5pm - External Identifiers are finally here

The server will be down for patching between 5:00pm and 5:10pm server time. The patch will add support for publication-specific external identifiers. Patch notes to follow. Ahasuerus 20:37, 24 May 2017 (UTC)

The server is back up. Patch notes:

The following types of identifiers will be supported out of the box (more can be added if needed):

+----------------------+----------------------------------------+
| ASIN                 | Amazon Standard Identification Number  |
| BL                   | The British Library                    |
| BN                   | Barnes and Noble                       |
| BNB                  | The British National Bibliography      |
| BNF                  | Bibliothèque nationale de France       |
| COPAC                | UK/Irish union catalog                 |
| DNB                  | Deutsche Nationalbibliothek            |
| FantLab              | Laboratoria Fantastiki                 |
| Goodreads            | Goodreads social cataloging site       |
| JNB                  | The Japanese National Bibliography     |
| LCCN                 | Library of Congress Control Number     |
| NDL                  | National Diet Library                  |
| OCLC/WorldCat        | Online Computer Library Center         |
| Open Library         | Open Library                           |
| SFBG                 | Catalog of books published in Bulgaria |
+----------------------+----------------------------------------+

Any number of identifiers can be entered when adding, cloning or editing a publication. Once associated with a publication, an external identifier appears on the Publication page and is automatically linked to the associated third party Web page. A publication can have more than one identifier of the same type, e.g. it can link to 2+ OCLC records. The functionality is similar to templates, but it’s more advanced because a single identifier can link to multiple Web pages. For example, an ASIN identifier links to the 6 supported Amazon stores while COPAC identifiers link to two different versions of the COPAC data: human readable and XML-based. You can still use linking templates in notes, but it’s best to limit their use to third party records which are not directly related to the publication, e.g. "The skeleton OCLC record {{OCLC|123456}} may be related."

The vast majority of ASINs (over 46,000) have been automatically migrated from Notes to the new field. A few thousand will have to be migrated manually. There is a new cleanup report to facilitate the process. The first iteration of the report is limited to the most straightforward cases; the data (146 pubs) will become available tomorrow morning. OCLC, LCCN, BL and other record ID haven’t been touched yet, but I’ll try to automate the conversion process to the extent possible.

A new Advanced Search, “Publication Search by External Identifier”, is now available. It does exactly what you would expect it to do. We may want to discuss making external identifiers available on certain other pages, e.g. in the standard “publication table” used by the Title page, Advanced Publication search, etc.

As always, a major patch may contain bugs or break something unrelated. If you notice anything unusual, please post your findings here.

Once the dust settles, I will create a cleanup report to look for e-books without ISBNs, ASINs or BN IDs. I will also enhance Fixer’s logic to check external identifiers when creating new submissions. Once everything is in place, Fixer will be able to create submissions for ISBN-less publications, something that he has been unable to do up until now.

Here is an example of what a publication record with multiple external IDs looks like. Ahasuerus 21:07, 24 May 2017 (UTC)

Compatibility Issues

The "add external identifier button" is not working. --Vasha 00:48, 25 May 2017 (UTC)

Have you tried a full page reload (Control-F5 in most browsers)? Ahasuerus 01:04, 25 May 2017 (UTC)
Fully quit and restarted the browser, still nothing. I am using Firefox. --Vasha 01:15, 25 May 2017 (UTC)
And... it does work in Safari. So why not Firefox? I have rather a lot of plugins... can list them for you if you need --Vasha 01:21, 25 May 2017 (UTC)
It's working under Firefox on my side, so I suspect that you are right and that there is something about your installation that causes a conflict. What's odd is that the added JavaScript code is similar to the code responsible for all the other "Add ..." buttons. Hm. Could you check the console? It's under Tools, Web Developer, Web Console. Once the console area appears at the bottom of the page, please click on the "JS" mini-tab and check the "Error" and "Warning" boxes, then reload (Control-F5) the page to see if anything shows up in the console area. TIA! Ahasuerus 01:53, 25 May 2017 (UTC)
I removed an extension called Awesome Timestamp and it's working now. Whew. --Vasha 14:24, 25 May 2017 (UTC)
Glad the mystery has been solved! Sorry about the conflict, but, unfortunately, there is not much we can do about add-on conflicts on the software side... Ahasuerus 20:50, 25 May 2017 (UTC)

Lowercase 'x' in ISBNs

Another odd thing (in Safari, this time). I entered the ISBN "014008374x" with a lowercase x and got the message "Catalog number is missing etc." but when I entered it uppercase "014008374X" it was accepted. I'm pretty sure that didn't use to happen. --Vasha 01:47, 25 May 2017 (UTC)

I think I encountered this scenario a couple of months ago. It will go away when we add a separate field for catalog IDs. Ahasuerus 01:56, 25 May 2017 (UTC)

Tweaking the Format and Location of External Identifiers

What about pushing the external identifiers below the notes field? With some exceptions (ex. LCCN), they normally aren't information from the pub itself & they seem pretty prominent in the example. Maybe it just because it's a change to what I'm used to, but I find them distracting in the current location. -- JLaTondre (talk) 22:05, 24 May 2017 (UTC)

Sure, we can do that. Let's wait 24 hours and see if there are any other suggestions. Ahasuerus 23:05, 24 May 2017 (UTC)
I love these. But I agree - they should move somewhere else - they clutter the page a bit. I wonder if a separate container (like the ones for the two types of notifications) won't make sense here... Annie 23:09, 24 May 2017 (UTC)
I also find this "mass" of (for me) uninteresting and likely-to-grow-bigger (with all the public libraries of the world) data particularly annoying. IMHO this should go either on a separate page or way out of sight ;-). Just think of users (like me) that sometimes consult the db on a smartphone with a quite small screen. Hauck 06:51, 25 May 2017 (UTC)

(unindent)I agree with moving the External IDs further down -- I'd rather have more important information near the top. I'm also concerned about the appearance for records that have, say 8 OCLC numbers. With books from before ISBNs, it's not uncommon for the WorldCat system to be unable to link related OCLC records (via the "View all formats" option). Hence there are times when I've included as many as 8 OCLC records for one book. When they're just in a comma separated list, that's not ugly; but when each one gets their own line, it just looks ugly. Chavey 14:11, 25 May 2017 (UTC)

It sounds like we have consensus re: moving external identifiers to the bottom of the section. Let me see if I can do it real quick. I will also try to change the logic to display multiple OCLC/etc records on the same line. Ahasuerus 15:43, 25 May 2017 (UTC)
OK, External IDs have been moved below the Note field. Ahasuerus 20:51, 25 May 2017 (UTC)
The display software has been changed to show all IDs of the same type on one line. For example, this pub has 4 OCLC records associated with it and they all appear on the same line. Ahasuerus 01:56, 26 May 2017 (UTC)
Excellent! Chavey 05:15, 26 May 2017 (UTC)

Catalog IDs

Is there a plan for getting publisher catalog codes out of the ISBN field (the one field=one data principle of database design)? --Vasha 00:39, 25 May 2017 (UTC)

Yes, indeed! As the Roadmap says:
  • Move catalog IDs to a separate field, which will help with pubs that have both an ISBN and a catalog ID, e.g. book club publications.
Ahasuerus 01:05, 25 May 2017 (UTC)
There isn't a 'BCE' option in the drop-down menu. --~ Bill, Bluesman 18:10, 25 May 2017 (UTC)
'BCE' as in "Book Club edition"? Ahasuerus 19:44, 25 May 2017 (UTC)
Yes, though if there's going to be a new field for Catalog #s it doesn't really matter. Saw that note above after posting. Hopefully that new field will be directly below the ISBN field as pre such #s it's the identifier of the record in most cases. --~ Bill, Bluesman 22:12, 25 May 2017 (UTC)
That's the plan! Ahasuerus 22:17, 25 May 2017 (UTC)

Help Updates

Can we add the table from above to here? Annie 23:52, 24 May 2017 (UTC)

Done. Ahasuerus 00:36, 25 May 2017 (UTC)
In addition, the mouseover Help has been updated to display full identifier type names when hovering over drop-down lists. Ahasuerus 18:06, 26 May 2017 (UTC)

BNB links

Can you check your code for the link for BNB? If you click on the link for it from the new fields (use your own example above), it leads to an empty result page in BNB. The code is correct - once you do a search for it, it find it - but that initial search seems to be sending some additional details wrong. Annie 20:01, 25 May 2017 (UTC)

The link was working yesterday, but today all BNB IDs and templates are broken. Let's wait a day or two to see if it's a temporary glitch. If not, we'll switch to the full "search" URL.
That's the nice things about IDs and templates -- when a site changes its URL schema, all we need to do on our side is change the ID/template definition instead of updating hundreds or thousands of Note fields. Ahasuerus 22:21, 25 May 2017 (UTC)
Ah, so the first time I decided to check a BNB link and their server is broken. I guess I should stay away from any other server I had not tried yet :) Thanks for checking :) Annie 22:40, 25 May 2017 (UTC)
Still glitching the same way... Annie 01:21, 1 June 2017 (UTC)
Looks like it's time to start looking for a new URL pattern! Ahasuerus 16:59, 1 June 2017 (UTC)
this seems to behave properly... And this is the shortest that works.. So something in between them maybe? Or just the really short last one? Annie 19:10, 1 June 2017 (UTC)
Confirmed and changed. Thanks! Ahasuerus 22:22, 1 June 2017 (UTC)

LCCN link bug

I've also entered a bug about a problem in interpreting certain uncommon LCCN numbers. Chavey 14:11, 25 May 2017 (UTC)

Fixed. The LCCN associated with the affected pub has been changed to "a 48004732". Ahasuerus 17:01, 26 May 2017 (UTC)

Barnes & Noble IDs

Here is a project for anyone interested. We have 170 publications with "294" ISBNs. Almost all of them were published by "Barnes & Noble Books / Google Books".

294 "ISBNs" are not real ISBNs, they are Barnes & Noble's proprietary IDs. They look like they may be EANs, but they are definitely not ISBNs. We'll need to move them from the ISBN field to the "External IDs" field. Ahasuerus 23:10, 24 May 2017 (UTC)

I tried one and I like it. Awesome addition! ···日本穣 · 投稿 · Talk to Nihonjoe 23:22, 24 May 2017 (UTC)
I'll get them cleared this weekend - had been slowly chipping at them when I have a minute. Annie 18:45, 26 May 2017 (UTC)
Thanks! Once they are gone, we'll be able to close FR 1055. Ahasuerus 21:55, 26 May 2017 (UTC)
Or a bit longer -- all done now. As soon as the pending ones are approved, all should be done. One legitimate ISBN in the search (a ISBN-10 that really starts with 294) :) Annie 18:14, 30 May 2017 (UTC)
FR closed. Thanks! Ahasuerus 21:35, 30 May 2017 (UTC)

"ASIN-UK" removed

Now that we know that different Amazon Web sites use the same ASINs (books only), I have removed "ASIN-UK" from the list of supported templates. All pre-existing occurrences of the template have been replaced with external IDs. Ahasuerus 23:11, 25 May 2017 (UTC)

Functionality to add to "Make Variant"

Hello, is it possible to add the "Check for duplicate title" button in the result screen of "Create New Parent Title" (Option 2) of "Make This Title a Variant Title or Pseudonymous Work". This function would check duplicate for the new Title/Author couple and will avoid the task of manually checking if this new combination is already present in the db (if it's not already been done by the "Check for Duplicate Titles" option, a phase that I usually forget). Thanks. Hauck 08:36, 26 May 2017 (UTC)

Hey, great idea. --Vasha 12:26, 26 May 2017 (UTC)
Done. "Check Parent Title for Duplicates" is now available on the two Make Variant post-approval pages. I am not sure how useful it will be when processing Make Variant submissions which link to existing titles, but it's harmless and there is something to be said for consistency. Ahasuerus 15:30, 26 May 2017 (UTC)
Thanks. Hauck 16:49, 26 May 2017 (UTC)

ASIN Cleanup Part 2

A batch of 900 pubs with ASINs in notes has been converted to use external identifiers. The ASIN cleanup report has been modified and will find 193 additional pubs tomorrow morning. Ahasuerus 03:13, 28 May 2017 (UTC)

The ASIN cleanup report has been tweaked to find additional (ca. 200) ASINs.
Also, a new cleanup report has been created and deployed. It looks for publications with explicit BL/BLIC URL in notes. Please note that BL URLs have changed in the last 12 months, so some links may not work when you click them. Ahasuerus 04:14, 30 May 2017 (UTC)
The ASIN cleanup report has been expanded to cover the vast majority (around 2,300) of ASIN links. The data will become available tomorrow morning. Ahasuerus 19:02, 31 May 2017 (UTC)
There are ~60 left which either need some more careful looking into before moving (I am slowly working on those) or will never get cleaned because the links are not to the actual books but there for either comparison or notes. So whenever you have a chance to convert this report to one that allows ignoring (so we can get the second type out of it) and add whatever else is lagging that may be ASIN and need a look at, feel free to. :) Annie 18:28, 16 June 2017 (UTC)
Thanks, working on it... Ahasuerus 20:45, 16 June 2017 (UTC)
The two Amazon-specific cleanup reports have been modified to support "ignoring" records by moderators. Ahasuerus 21:24, 16 June 2017 (UTC)
The BLIC report is as clean as it will get (after the pending are accepted). We need an ignore option though -- there are some non-BLIC links to the British Library. Annie 07:09, 18 June 2017 (UTC)
Most of which were put in there by me, I think. (All the Shakespeare stuff.) Sorry about that. (But I got a Shakespeare professor to verify it for me, and he thought I did a good job :-) .) Chavey 02:09, 20 June 2017 (UTC)
Oh, I think the links are exactly where they need to be and very helpful. They just got in the crossfire of me trying to clean the report :) That's why I am asking for an ignore option - there are legitimate cases where we will want to connect to the British library without it being a BL or a BNB number. Annie 02:17, 20 June 2017 (UTC)
Done! Ahasuerus 02:59, 20 June 2017 (UTC)

(unindent) And one more group of ASIN related cleanup ASIN as catalog ID. Most of them do not have the word ASIN anywhere so they are not on the new report. No need for a report or tweaking the old one - there aren't that many of them but just adding them here so they do not get forgotten. That makes me wonder if we do not have other external links stuck as catalog IDs though... Annie 19:13, 21 June 2017 (EDT)

Nongenre stories: attempt at a roundup

So the background is I was going through an anthology of classic horror stories, and wondering why so many famous ones, which have no speculative elements, are not marked non-genre in this database. Hypothesis: originally added because they were in books of horror stories; never changed because it's too much trouble to get opinions from the many, many primary verifiers involved.

But maybe it's time to revisit them. I put together a list of a dozen that I think are non-genre, and sent a message to 26 primary verifiers. See the post and conversation on MLB's page, the person who suggested making this a board conversation instead. And contributed several more suggestions (only the first 12 are by me), commenting "I... don’t count dreams, fugue states, and hallucinations as fantasy elements, but others do, and I’m not gonna argue with them.".

Thus... folks, please chime in and give your reasons if you think these stories should NOT be marked non-genre. And, contribute other stories with many verifications that need the same discussion. --Vasha 00:41, 31 May 2017 (UTC)

Genre? No. SYNOPSIS: Faulkner visits a low gambling house where he wins a large amount. One of the hosts serves him coffee and urges him to not try going home with his money through the streets at night while he is drunk, instead offering him a bed for the night. Faulkner lies on the large four-poster bed, feels sick, and is wide awake. He sees the canopy slowly and quietly descending. Scrambling out from under, he sees the canopy being lowered by a screw. He realizes that the coffee was drugged but a too-strong dose of the drug had the opposite effect, and that he was supposed to be smothered and disappear without reporting that he had been robbed. NOTE: Non-genre, because it is a suspenseful mystery story with no hint of the supernatural. Mostly found in anthologies of mystery, suspense, and terror rather than supernatural anthologies.
Genre? No. SYNOPSIS: Looking back from 1930, the narrator recollects his the townsfolk of Jefferson, Mississippi, recollect glimpses of the life of Emily Grierson, an elderly woman from the antebellum Southern aristocracy. After the war, she and her father continued to live as if in the past; he didn't let her marry. After her father's death, she became romantic with a Northerner newly come to town, but he said he was not the marrying type and eventually disappeared. The rest of the story chronicles Emily's archaic, aristocratic, and increasingly reclusive behavior; the full extent of her eccentricity only becomes clear after her death. NOTE: Non-genre, since it is a very gothic and gruesome story, but not at all supernatural; it belongs to the non-speculative side of the gothic genre.
  • Leiningen Versus the Ants (Carl Stephenson) Army ants (exaggerated so they're like none seen in nature, but not intended to be speculative I don't think) nearly overwhelm the hero. As MLB says, a prototype of "nature gone berserk" stories
Added the following note: "Borderline speculative: The ant invasion is very much larger than life, and even though the story is written as if it is an account of ordinary life in Brazil, it is the prototype of many more obviously speculative man-versus-nature stories." --Vasha 02:33, 1 June 2017 (UTC)
Genre? No. SYNOPSIS: The narrator is a selling a brand of relish door-to-door and sharing living quarters with an Oxford scholar. One of his customers is suspected of murder, but the investigation is at a standstill because no body can be found. The salesman gathers information and the scholar ratiocinates, figuring out how the corpse was hidden. NOTE: Non-genre. This is one of the crime stories that Dunsany published in Ellery Queen's Mystery Magazine. Although its gruesome final revelation gives it a place in horror anthologies, it is thoroughly non-fantastic.
  • Taboo (Geoffrey Household) An eastern European community is terrorized by a series of murders. The mystery is solved, but the psychological effects linger on. Non-supernatural
Genre? Yes. SYNOPSIS: The psychiatrist Shiravieff is talking about the bad effects of not expressing one's emotions, and illustrates his thoughts with a story about a very restrained Englishman named Vaughan and an experience which Shiravieff and Vaughan shared a decade earlier. They met while vacationing in a remote Eastern European town. Local people had been disappearing. Talk in the inn turned to stories of werewolves. Vaughan's wife had ideas about humans who behave like werewolves. Shiravieff and Vaughan, both hunters, decided to keep watch in the forest and trap the killer-- one walking up and down in plain sight, the other in hiding, keeping a gun trained in readiness. Their plan succeeded but the solution of the mystery did not make matters any less disturbing. Vaughan's refusal to admit that he was disturbed had lingering psychological consequences. NOTE: A genre story, in spite of not being outright supernatural. A story of fear and the hidden workings of the mind. The monster in the story is human but there is a lot of discussion about evil and what might make a person behave like a werewolf.
Genre? No. SYNOPSIS: By chance the big-game hunter Rainsford has been stranded on an island owned by another hunter, General Zaroff. Zaroff reveals that he has grown bored with hunting even the most dangerous animals, and wants to match wits with human prey. Rainsford is to be pitted against him in a desperate effort to survive. NOTE: This non-speculative adventure story does not fall within the SFF genre, strange though its premise is; however, it has influenced numerous speculative adventure stories.
Genre? No. SYNOPSIS: Mary Maloney bashes in her husband's head with a frozen leg of lamb, then puts it in the oven to roast. She prepares her alibi with care and summons the police. They don't suspect this very housewifely and heavily pregnant woman. After they've fruitlessly searched for the intruder and wondered what his weapon could have been, she invites them to eat the lamb, so it won't go to waste. NOTE: Non-genre, since it is a conte cruel with a startling nature but nothing at all fantastic about it.
Genre? No. SYNOPSIS: The artist Raut is visiting Horrocks, the manager of an ironworks in Stoke-on-Trent, while surveying the artistic possibilities of the industrial landscape; he starts an affair with the manager's wife. Horrocks takes Raut on a tour, and shows him a highly advanced blast furnace before pushing him into it. NOTE: This non-genre story is a depiction of life in industrial towns, ironically juxtaposing the aesthetic, progressive, and murderous aspects of the technology of its day.
Genre? No. SYNOPSIS: A man climbs onto his balcony to retrieve a piece of paper (which contains work he's been neglecting the rest of his life for, in search of a promotion), and gets inadvertently trapped on the ledge. NOTE: Non-genre, because it is a non-speculative suspense story (and not horror, because it has a positive message and a happy ending).
Genre? No. SYNOPSIS: Mr and Mrs Bunting desperately need money; fortunately, Mr. Sleuth, the man who offers to rent a room from them, seems a perfect gentleman. He does have some eccentric habits, though, such as secrecy and late-night expeditions. As a serial killer terrorizes the city, Mrs Bunting begins to wonder if their lodger is the killer. What should the Buntings do? Risk losing their income by talking to the police, especially when they're not sure? NOTE: A non-speculative mystery-suspense story.
Genre? No. SYNOPSIS: On a rainy night, a shepherd is hosting a jolly gathering at his house. One after another, two strangers arrive and are welcomed in. A third stranger arrives, sees the other two, and quickly backs out the door. Then the news arrives that a prisoner who was to be hanged the next day has escaped from prison. The reader will suspect that the escapee is one of the three men, but which? and will he get away? NOTE: A realist story with a touch of mystery and suspense about it.
Genre? Yes. SYNOPSIS: The narrator tells of mixing up a poison from instructions he found in an old book, a poison that neither washing nor heat can destroy. With it he intends to commit murders that will be absolutely untraceable because not even he will be able to follow the path from his hands to the victim. NOTE: Although this is a crime story, it falls within genre due to the speculative nature of the murder weapon.
Genre? Yes. SYNOPSIS: During the American Civil War soldiers are executing a prisoner. They put a rope around his neck and push him off a bridge. The rope breaks and he makes his way across country to his home. Just as he gets there, he dies-- his escape has all been a last-second imagining as he's falling. NOTE: Barely genre; but as a morbid story of strange mental experiences, included in a number of genre anthologies.
Genre? Yes. SYNOPSIS: The narrator lives with an old man whose eye he finds terrifying. He kills the old man and hides him under the floorboards. The police arrive and he chats with them amicably, but is disturbed by hearing a heartbeat... NOTE: Genre in spite of being questionably supernatural; first because of its disorienting nightmare qualities, and second because it's in a lot of genre anthologies.
Genre? Yes. SYNOPSIS: The narrator Montresor meets Fortunato, who greets him with friendship, not suspecting that Montresor has sworn vengeance on his old friend for an offense he doesn't name. Montresor invites Fortunato to come into his cellars and taste a cask of fine sherry; however, he has other plans in mind than drinking. NOTE: Although there is nothing supernatural in this story, it belongs to the genre both because it is told from the point of view of a madman and because it has been included within fantastic literature by long tradition.
  • Psycho (Robert Bloch) More madness.
Genre? Yes. SYNOPSIS: The novel begins at the Bates Motel, where Mary Crane, on the run after impulsively stealing from her employer, overhears Norman Bates arguing with his mother, who berates him and screams "I'll kill her!" Soon, Mary is dead; Norman finds her corpse and believes his mother killed her, and that he must protect her by disposing of the evidence. Mary's sister, her boyfriend, and a private investigator search for her; one encounters Mrs. Bates and is killed, the others learn that Mrs. Bates has been dead for years. A ghost? No, the killer will turn out to be human. NOTE: Borderline genre, since stories of strange psychopathologies usually lie within the genre's realm.
Genre? Yes. SYNOPSIS: Madeleine de Scudery is an elderly single woman noted as a poet, and a friend of King Louis and his mistress Madame de Maintenon (she is actually a historical figure). She finds herself enveloped in a murky affair when rich men begin to be killed in the street and robbed of fancy jewelry. The killer sends her a present of jewelry; then the jeweler Cardillac is stabbed to death. Cardillac's assistant Olivier is accused of his murder and begs Mlle. de Scudery for help. She manages to disentangle matters by talking to people, convincing them to tell their stories and to behave reasonably. NOTE: Although this story has essentially no speculative elements, apart from the rather peculiar mindset of the killer, and is not especially horrific either, it nonetheless belongs to fantastic literature simply by virtue of having been included in it for a long time, alongside Hoffmann's more clearly fantastic tales.
Genre? Yes. SYNOPSIS: A prisoner of the Inquisition is subjected to slowly encroaching death, first in the form of a pendulum which he manages to escape, and then in the form of a pit. NOTE: Genre, both because this supposedly historical story has the effect of a strange nightmare, and because it is found in many genre anthologies.
    • and The Torture by Hope by Villiers de l'Isle-Adam. Tortures by the Inquisition. Perhaps fanciful, not fantastic.
Genre? No. SYNOPSIS: In Aragon, in previous centuries, the Jew Aser Abarbanel falls into the hands of the Inquisition. He is tortured daily. Then he is to be executed. Hours before the execution, he finds his cell door open and the guards gone. He escapes... only to be recaptured, and realize that he was never really free, and was only being subjected to the terrible torture of hope. NOTE: Non-genre, in spite of the fact that it has an epigraph from "The Pit and the Pendulum;" it belongs to the class of non-speculative contes cruels (stories with nasty final twists).
  • Vendetta (Guy de Maupassant) An elderly woman gets revenge on the killer of her son by training a dog to kill him. Nongenre IMO. I added the following note: "Almost but not quite fantastic, in that it is a horror story that aims at a strange and shocking effect."
So do you simply want them marked as non-genre, or do you want them deleted? I got the impression that many of these got a pass because their authors are influential in their genre. --Auric 12:04, 31 May 2017 (UTC)
Full disclosure: I am not enthusiastic about a zealous pursuit of, and defense again, non-genre. I don't like the removals of small numbers of non-genre pieces from anthologies/collections of largely genre work, and I have inclusionist tendencies -- if whether a piece should be "in" or "out" is debatable, I'd prefer to see it be "in". That said, I'd much rather see things marked non-genre than deleted outright. But I think some of the items cited above may fall into a gray area, where either "fantastic" or "supernatural" is debatable. For example, Poe's "The Tell-Tale Heart" could be considered to have a supernatural element. Any one of us might not think so, but another one of us may. Since that assessment is subjective (neither site can "prove" its claim), the debate could be enless. So following my inclusionist tendencies, I'd err on the side of considering "genre" anything that someone else deems to be such. I don't think there is much value to be gained from debating large numbers of individual titles, so I'd be inclined not to worry about how they're classified. --MartyD 12:20, 31 May 2017 (UTC)
I do agree with you that borderline works belong here. When in doubt, keep it in. And I myself like to include stories of the "weird" that create unease out of things that are ordinary on the surface, and stories of odd mental experiences, and suchlike borderline cases.
However, the first twelve on that list (with the possible exception of "Leiningen Versus the Ants") are not borderline at all. I don't see the value in having a database supposedly devoted to spec fic containing masses of crime and adventure stories. In particular, it does harm to author bibliographies-- if someone comes here to find out what science fiction Jack London wrote, and the page contains 40 stories, most of them non-genre and not marked, that isn't helpful.
It's inevitable that people will not necessarily know which contents of an anthology are speculative when they enter it. That's why I favor marking the stories n-g instead of removing them, at least in the case of older ones that will come up again in a new anthology sometimes.
In summary, I would very much like to see authors like Wilkie Collins, Roald Dahl, etc. etc. have their speculative stories distinguished and featured as the central purpose of this database. That's going to mean taking another look at existing records. I don't intend to systematically hunt for them, but when I see a story that needs to be relegated to the bottom half of the page, I will speak up. Vasha 13:27, 31 May 2017 (UTC)
The way I see it, our first priority is accuracy. If a title is borderline, I don't use the "non-genre" flag, but I add a note and the source of our information. For example: "According to reviews, this contemporary coming of age novel set in South Africa contains some elements that are at least borderline speculative" or "According to reviews, this thriller contains borderline science fiction/alternative history elements." If it's clearly non-genre or the genre element is so minor as to be beyond "borderline", I use the "non-genre" flag and add a note, e.g. "A contemporary thriller. The only borderline speculative element is a snowstorm of unheard of proportions." That way if another editor disagrees and flips the "non-genre" flag at a later point, we haven't lost any data.
As far as the issue of inclusion goes, I enter the non-genre stories if the vast majority of the titles in the publication are speculative. Otherwise I omit them and add a note. Unless it's a genre magazine, of course. Ahasuerus 17:56, 31 May 2017 (UTC)
Having an incomplete TOC (even with a note that it is so) always sounded to me as a bit of a misnomer. I'd probably add the non-added stories in the notes (so it is clear what is missed)... But that is a different story.
Couple of notes on the above - a story does not need to be fantastic to be included (we are speculative, not fantastic DB) so I would think that if macabre is the word for something, it should be included as genre. If it has no speculative elements, it cannot be macabre in my book... About the one for the ants is not intended as speculative probably because there were no speculative stories at that point :) I still consider it genre. Author's intent is not what is important for the classification - a lot of modern authors try to claim that they do not write speculative fiction. But if the shoe fits.... No opinion on the rest - never read them (or don't remember them anyway). Annie 18:53, 31 May 2017 (UTC)
We all have different versions/attitudes about what constitutes Sf/Fantasy. They're all molded by today's standards. Stories/novels from as much as 150 years ago simply CANNOT be judged by the same criteria. By today's realities, many stories simply cannot be considered fiction because they are now fact [or were many decades ago]. So what. If the conjectures were simply that when the story was written, we have no choice but to include it. --~ Bill, Bluesman 00:52, 3 June 2017 (UTC)
That's right. Works "set in a future that is now in the past" and/or "deal with technological advances that were futuristic at the time they were published" are explicitly defined as "science fiction" and within the scope of the project by ISFDB:Policy. Ahasuerus 01:08, 3 June 2017 (UTC)
Such is the nature of the job we all choose to do. If you can't step outside your prejudices, then don't edit here. --~ Bill, Bluesman 00:52, 3 June 2017 (UTC)
Well, to many of these anthologists, macabre just means gruesome and morbid. So they include stories focusing on gory or strange deaths even if not speculative. --Vasha 01:41, 1 June 2017 (UTC)
You were the one that called it macabre above - had not read the story so was going based on your details for that one :) Annie 01:49, 1 June 2017 (UTC)
I haven't read it either, I'm just going by the detailed Wikipedia summary. It was intended to be the sensational opening to an abandoned novel. The novel was going to concern itself with life in industrial towns and this story sort of sets the thematic tone by using the best technology of the day as a means of murder. So... related to science fiction, kinda-sorta, in that life among technology is put under scrutiny... but not future technology. (By the way, Edward Mendelson's introduction to the Penguin edition of Tono-Bungay makes a pretty convincing case that this novel is science fiction.) --Vasha 02:36, 1 June 2017 (UTC)
I'm a fan of the non-genre flag, and am all for being liberal about inclusion in the db. Many of these stories have a history of inclusion in the genre. A rigorous use of n-g marking is fine, and I think this is a worthy effort. One sub-category of stories I would use it with are ones (we can all think of many examples) in which the plot develops in such a way to lead the reader to think something supernatural is afoot, but in the end a scientific explanation is revealed. In those cases we can see that there is nothing unnatural at work, but for much of the story the mood is supernatural. Just an example. Ldb001 22:28, 1 June 2017 (UTC)
This issue is a tough call. Gothic horror does not require nonrealism to be part of the genre. Almost all of Ann Radcliffe's novels end with elaborate explanations of how the apparent nonrealism happened realistically (I call that the "Scooby Doo Effect"), but her books are undeniably Gothic and set the standard in the 1790s. All of the stories mentioned are highly imaginative and stir the unconscious in a way realism tends not to. Their striking "effect" (which Poe said was the central device of his short fiction) has a cumulative quality as one reads. I would never remove "The Tell-Tale Heart" from a list of Gothic horror stories. Several Poe stories can easily be read as the ravings of a lunatic murderer (e.g., "Ligeia"). Vonnegut's Slaughterhouse-Five can be read as a psychological fantasy of a failed man dreaming of being trapped in an alien zoo with a Playboy model. This initiative opens up a can of worms, and I do not see an easy solution to it. --7:03 1 June 2017 User:Hifrommike65
Yes... I am also in favor of being rather inclusionist about psychological horror. It really is difficult to draw a line through the middle of the horror genre. What about crime and adventure stories, though -- they're "suspense" rather than horror and therefore farther away from that fantastic effect.
As several people have said in this discussion, the important thing is, when you make a decision -- any decision -- you document it. (I'm currently going through Daphne Du Maurier and doing just that-- adding summaries and notes to every single story that I hope will sufficiently explain the speculative content, or lack of, or ambiguity of.)
I only started this discussion because I saw some stories that had a crying need for an explanation as to why they are either in or out, and they were stories that appeared in publications verified by dozens of people, so some sort of broad consensus seems necessary. --Vasha 00:17, 2 June 2017 (UTC)
This discussion is worth having, because it clarifies what constitutes "genre" publication. Jacques Derrida has an essay titled (in English) "The Law of Genre" (first published in English in 1980, and now collected in the book Parages) in which he says genres inevitably mix. An example is Charles Dickens' Bleak House, an otherwise realistic novel in which an evil character explodes by spontaneous combustion. He made the point that some stories "participate in without belonging to" a certain genre. He's getting at the slipperiness of genre as a concept. Please look at the cover of the book Alfred Hitchcock's Spellbinders in Suspense, which illustrates Connell's "The Most Dangerous Game". The cover is Gothic as all get-out and does not distort the tone or feel of the story. That said, I suggest each item included in ISFDB be considered on a case-by-case basis, as Vasha is doing.--User:Hifrommike65 12:30 a.m., 2 June 2017.
Wierdness can show up anywhere. I'm thinking of the brilliantly bizarre 1945 mystery novel The Red Right Hand by Joel Townsley Rogers (who has short fiction in this DB). In the end the mystifying events of the novel have an explanation, and yet Rogers has created such a powerful uncanny atmosphere that it is not explained away-- the rational solution is not sufficient to dissipate the weirdness of the whole. I don't suggest that The Red Right Hand belongs in this database, I'm just saying that weirdness is not a genre really. It can be a reason to include a seemingly-mundane story in a collection of supernatural stories, though. I'm a fan of the kind of stories of unease that the magazine Supernatural Tales publishes; some of them have nothing overtly supernatural in them but, in that context, their creeping strangeness includes them in the genre of weird fiction. So I'm arguing that in that case, genre equals, not plot or content, but effect plus context. Without the context, the effect wouldn't be enough to make me say that it is genre. --Vasha 14:15, 2 June 2017 (UTC)
Not exactly ignoring what's above, but isn't the very idea of 'non-genre' horror a contradiction in terms? Not one person who habituates this site wouldn't laugh at the idea of 'non-genre' fantasy - there simply can't be such a thing. There's really nothing to discuss. Horror is horror, psychological or otherwise. If we're going to draw a line that fine then there's a whole lot more on this DB that has to go. And what would be the point??¿¿?? I agree that we should exclude; I agree that we should not be inclusionists, but not to the extent that anything by any author should be included [take a look at the dreck that's on Robert E. Howard's page for a perfect example; can you imagine if this hack had written romances ........] --~ Bill, Bluesman 00:34, 3 June 2017 (UTC)
At this time the Policy page defines "speculative fiction" to include "supernatural horror" and to exclude "purely psychological horror works that feature no supernatural or fantastical elements". There have been a few Rules and Standards proposals to change the scope of the project to include psychological horror works, but they were unsuccessful. Ahasuerus 01:16, 3 June 2017 (UTC)

Vasha has intriguing ideas about genre. I will be thinking about the ideas suggested. Meanwhile, I'd like to kick in another theorist who may help in this area of inquiry. Tzvetan Todorov's book The Fantastic: A Structural Approach to a Literary Genre (translated in 1973) says the fantastic occurs when the reader, identifying with a story's protagonist, hesitates in deciding whether events in the story depart from reality or not. If they depart from reality, we have "the uncanny" (the term Freud used); if they do not depart from reality, we have "the marvellous" (e.g., Radcliffean or Scooby Doo Gothic). This position is perfect for Henry James, who built ambiguity into most of his weird fiction. It does not work for every writer on the border of weird. --User:Hifrommike65 4:50 p.m., 2 June 2017

Sure-- not all critics of fantasy are fond of Todorov's definitions, and I think they're overly narrow myself, but they've certainly been hugely influential, and I think they've helped shape what gets included in canon lists and anthologies, and thus they've helped shape genre. How many anthologies of Latin American cuentos fantásticos have I seen which start out with an introduction defining fantástico and invoking Todorov? His definition provides a reason to include Juan Rulfo's story "Luvina." It's told through the narration of a former schoolteacher who was once assigned to the village Luvina which is completely arid and only inhabited by old people whose children work elsewhere to send them food-- the teacher describes the terror of life there and keeps talking about the wind as if it's a living, malignant thing, but he always says it in the form of similes, "like" a monster. And Luvina clearly had a dreadful psychological effect on him, but, satisfying Todorov, you would hesitate to say there's anything unnatural about it. "Luvina" is also a weird story, haunted by a sense of the inexplicable, a sense of being on insecure footing. But as for something magical you could point out, there is none. So it is fantastic without being fantasy. And I should add that when I was entering the above-mentioned Latin American anthologies into the database, somebody (I think it may have been Hervé Hauck) said that in their opinion, fantastic-not-fantasy doesn't really belong in the database. --Vasha 02:05, 3 June 2017 (UTC)
It is the narrowness of Todorov's definition of the fantastic that makes it useful. His methodology works because he defines so tightly. It doesn't account for a wide range of material, but is excellent for a certain type of narrative, such as Theodora Goss's "Pip and the Fairies," my favorite genre story published in the 21st century. I believe it was Eric Rabkin in The Fantastic in Literature who said the fantastic moves beyond the unexpected to the anti-expected in narrative. The problem is that the anti-expected quickly becomes a cliche. Even Lewis Carroll had his heroine wake up at the end of her story. But virtually all nonrealistic narrative has its roots in the unconscious and dreams, if psychologists are to be believed. To move on to a genre story without nonrealism, we have only to look at Shirley Jackson's "One Ordinary Day, with Peanuts." Published in F&SF, the story was reprinted in both Judith Merril's sf annual and The Best American Short Stories. I read it originally in Damon Knight's A Science Fiction Argosy. There is no overt fantasy in the story, but it is undeniably weird fiction, and the anti-expected works well to place it. --User:Hifrommike65 2:59 p.m., 3 June 2017
Right, if you take Todorov's "fantastic" as the tight definition of a certain kind of fantastic effect, it is useful, and it relieves you of having to decide whether the fairies in "Pip and the Fairies" are real in order to decide what sort of story it is. Likewise, "anti-expected" certainly doesn't apply to some works, like mythic fiction that relies on moving down the familiar path of myth.
We are once again brought back to the contextual identification of genre. If there is no one definition that covers every story that might appear in genre publications and genre conversations, then we know them by the company they keep. (Definitions are useful, though, for recognizing stories in non-genre publications.) The reason that I have a problem with some of the stories listed above is that they appear in contexts that overlap with this database's genre but aren't the same. Suspense and terror is a different genre than science fiction and fantasy. The fact that some stories in the suspense genre are also SFF doesn't change that. It is perfectly possible for a story to belong to multiple genres. So when you have an anthology which, overall, is centered on the suspense genre, it can contain SFF stories without shifting its genre. Therefore, and here is the crux of my argument, the fact that non-speculative suspense stories share an anthology with speculative ones doesn't put them into a SFF genre context. People adding anthologies to this database haven't paid attention to that; they're just like, okay, this one has supernatural stories in it, let's put all the contents in. I propose to reexamine each story in a more careful case-by-case way, and to go through suspense anthologies and mark as non-genre the stories that aren't also SFF or fantastic.
Also, saying that horror is a genre that overlaps with SFF but is separate resolves the dilemma of how the policy page can divide speculative from non-speculative horror. Just say that horror is not in the scope of this database, but a story can be included when it is horror and also SFF/fantastic. --Vasha 15:38, 4 June 2017 (UTC)


Fascinating discussion. Glad you called myth a "familiar path," because I once called myth a genre and an unidentified reader of my manuscript recommended against publishing it. Not every literary work has genre status. Also, after decades of critical wrangling over what the "Gothic" is, Diane Long Hoeveler came along and rejected genre, mode, and style, calling it an ideology (in her book The Gothic Ideology: Religious Hysteria and Anti-Catholicism in British Popular Fiction, 1780-1880 [Wales, 2014]). (Anything to avoid the "return of the repressed" Freudian view.) Ellen Datlow looks for two types of stories for her horror anthologies: tales of supernatural horror or psychopathology. The latter mostly does not have nonrealism, and might be better called a grotesque fiction or a conte cruel. But "conte cruel" does have an entry in The Encyclopedia of Fantasy. It's possible what we're discussing here is not what constitutes horrific but what constitutes speculative. --Hifrommike65 5:02 p.m., 4 June 2017

You'd think it would be possible to define speculative, wouldn't you? And yet... ever since Poe, stories of psychopathology have been absorbed into fantastic literature, especially when they're told from the madman's point of view, causing a disorienting shift in the reader's perceptions too. Mariane Bury and Anne Richter, editors of extensive collections of Guy de Maupassant's fantastic stories whose contents don't entirely overlap, both chose to include "Un cas de divorce," about a man whose madness takes the form of displacing eroticism from his wife to flowers; it's odd but not the least bit supernatural. --Vasha 01:17, 5 June 2017 (UTC)
The "New Wave" helped muddy the waters. Damon Knight started publishing stories in Orbit that he liked from the Milford Writers' Conference that had no overt nonrealism. He said the stories looked like a duck and quacked like a duck, so who cared whether they were ducks or not. Not long after, Norman Spinrad defined science fiction (in the 1974 Anchor anthology Modern Science Fiction) as "anything published as science fiction." That makes something science fiction strictly by publishing category. Trouble is the publishing category can change. Reminds me of Kingsley Amis's great short verse:
“SF's no good!"
they bellow 'til we're deaf.
"But this is good."
"Well, then, it's not SF!” --Hifrommike65 12:24 a.m. 05 June 2017

To the list above, I've appended the notes and synopses I'd like to add to all the stories, and whether I think they should be marked non-genre or not. Anyone with a different opinion, please speak up. I intend to make these changes in 48 hours if no one has anything to say. --Vasha 05:12, 5 June 2017 (UTC)

On your annotations, my only quibble is to note that "A Rose for Emily" is told from the "we" point of view. Your description implies that it is from a single point of view. The speaker represents the community as a whole, which is the actual monster (as I interpret the story in my dissertation). --Hifrommike65 12:31 a.m. 05 June 2017
Thanks, corrected. --Vasha 14:11, 5 June 2017 (UTC)

Here are the remaining NG stories from Great Tales of Terror and the Supernatural:

  • La Grande Bretèche (Honoré de Balzac). Genre? No. SYNOPSIS: A traveler hears the story of the titular manor house, which has been deserted since the death of its owner Madame de Merret. She once had a lover, and was entertaining him in her bedroom when her husband arrived unexpectedly. She hid her lover in a closet, and when her husband became suspicious, swore on the crucifix that no one was in there but forbade her husband to open it. Thereupon he ordered the closet walled up. Madame de Merret could not say anything, and she and her husband silently listened for days to noises from the walled-off space. NOTE: A horrific non-speculative tale that belongs among sensational literature but not fantastic literature.
  • The Interruption (W. W. Jacobs). Genre? No. SYNOPSIS: Goddard murders his wife and is blackmailed by his cook, Hannah. Hannah is very clever and frustrates Goddard's attempts to be rid of her. NOTE: A non-speculative story of psychological suspense. There is a slight hint at the end that Fate may be taking a hand in Goddard's downfall, but that's not enough to bring the story into genre territory.
  • Suspicion (Dorothy W. Sayers). Genre? No. SYNOPSIS: Mr. Mummery is feeling sick as he reads in the paper about a cook, Mrs. Andrews, who killed a family with arsenic. He begins to wonder about the cook he and his wife just hired as illnesses and oddities pile up. NOTE: A non-genre mystery-suspense story.
  • Back for Christmas (John Collier) Genre? No. SYNOPSIS: Dr. Carpenter has been invited to lecture in America; his wife is making arrangements for the trip, the way she arranges everything in their lives. Before his departure Dr. Carpenter carries out a plan to murder his wife and hide her in the hole in the cellar that he'd told her was a new wine bin. Will he get away with it? NOTE: A non-genre murder story.
Thanks for an outstanding effort on teasing out aspects of genre. Do you want to discuss what constitutes speculative? --Hifrommike65 04:49 p.m. 06 June 2017
Actually, what I really want is to get back to my usual job of keeping 2017 short fiction up to date -- that takes most of my time. I do love these sorts of discussions though!
Out of curiosity, what name have you published papers under? Never mind if you'd rather not say. --Vasha 00:00, 7 June 2017 (UTC)
I'd rather not post the info publicly here. Is there a way to private message you? --Hifrommike65 1925, 06 June 2017
Eh, never mind. It's been really interesting hearing from you. --Vasha 04:51, 7 June 2017 (UTC)
Of genre interest, I have published an article on Robert Silverberg's invasion theme, an article on Samuel R. Delany in a Greenwood Press biocritical reference book, many capsule reviews of new academic books on science fiction, fantasy, horror, and equivalent films for Choice magazine, and a dissertation on Gothic narrative, including a chapter on Stanley Kubrick. My best work of genre interest from conference presentations is on Arch Oboler's 1951 feature Five, the first anti-nuke post-apocalyptic film.--Hifrommike65 16:50, 07 June 2017 (UTC)

This batch of stories has all been updated. But I'd like to strongly urge other people to do some work of the same sort. Just to throw out one starting suggestion: here's a list of the stories by Wilkie Collins in the database that aren't included in his three-volume Collected Supernatural and Weird Fiction. Anyone want to investigate them? Armadale, A Passage in the Life of Mr. Perugino Potts, Mr Wray's Cash Box, The Ostler, The Diary of Anne Rodway, Fauntleroy, Mrs. Bullwinkle, Picking Up Waifs at Sea, John Jago's Ghost; or, The Dead Alive, After Dark, Fatal Fortune, Miss or Mrs.?, Miss Bertha and the Yankee, Mr. Policeman and the Cook, Fie! Fie! or, the Fair Physician, Mr. Lepel and the Housekeeper, Our Last Walk, The Yellow Tiger --Vasha 00:00, 7 June 2017 (UTC)

OCLC access

As of a few minutes ago, OCLC seems to have launched a new version of itself. So far it's impenetrable .... they may simply have ignored older operating systems, but repeated attempts to log on have been futile. Can anyone get on there and tell them their new system just doesn't work????? --~ Bill, Bluesman 00:19, 3 June 2017 (UTC)

Which OCLC page are you trying to access? When I go to their home page, it looks unchanged. The record that I happen to have displayed at the moment also looks OK. Ahasuerus 00:59, 3 June 2017 (UTC)
It may have been a brief test. Looks normal today, though the sign-in page is different. Yesterday it took forever [several minutes] for a single page to partially load, and then it would just quit. The entire look of the 'new' version was utterly different. And I'm sure I wasn't hallucinating ... been off those meds for awhile ........ ;-) --~ Bill, Bluesman 16:08, 3 June 2017 (UTC)
Oh, I see. A partially loaded Web page can be very different from its fully loaded version. Basically, your browser receives partial instructions re: how the page should be displayed, so the end result can look odd.
It's less of a problem with simple Web pages, which simply look incomplete. On the other hand, a complex page can be totally messed up. It's like trying to build a car with half the instructions :-) Ahasuerus 16:20, 3 June 2017 (UTC)

Public backups: hosting changes

The May bill for making our public backups available online was 6 times the average for the previous year. In a way it's good news because it means that there are people out there who are interested in our data. And, of course, the more people have the latest copy of our backups, the easier it will be to recreate the site if something bad happens to the administrators.

On the other hand, there is no way of telling how much more expensive it will get going forward. In addition, the current hosting solution depends on me paying the bill every month. If I become unavailable and the bill is not paid, the account will be suspended.

For these reasons I am going to move the backup files to Google Drive, which is free as long as the total amount of data is under 17Gb. I expect the migration process to be completed some time tomorrow. Older backup files will be unavailable until we can find a good home for them. I'll update this post once everything has been migrated. Ahasuerus 16:55, 3 June 2017 (UTC)

Do you need someone to help you foot the bill? Hauck 17:24, 3 June 2017 (UTC)
Thanks for asking, but I am fine for now. The problem is that there is no way of telling what may happen in the future. In April it was $N, in May it was $NN, but what if two months from now it's $N,NNN? For example, a couple of days ago I received an e-mail request to help this blog extract certain data from the database. Apparently it's a very popular blog and its readers are interested in data and databases. Once the owner massages our data and publishes her results, we may get thousands of new visitors, some of whom may decide to download our backups. It's best to be prepared :) Ahasuerus 17:49, 3 June 2017 (UTC)
How big are the backup files, and how often are they created? I might be able to work out something to help that would be more flexible than Google Drive. ···日本穣 · 投稿 · Talk to Nihonjoe 22:52, 3 June 2017 (UTC)
There are 2 types of publicly available backup files: "database backups" and "image backups".
"Database backups" contain non-graphical ISFDB data: titles, pubs, authors, etc. They do not contain submissions or detailed user data. Each file is available in 2 different formats, one per supported MySQL version. Each file is 145MB compressed. A new version is created every Saturday. At the moment only the latest version (2017-06-03) is publicly available, but we have an archive going back to 2006.
"Image backups" contain all of the images hosted by the ISFDB. The total size is almost 10GB. Only one version is available at any point in time, although I have earlier versions on DVDs. New versions are upload every 4-6 weeks.
The files are linked from the ISFDB Downloads page. Ahasuerus 01:17, 4 June 2017 (UTC)

(unindent) The image files have been moved to Google Drive and are once again downloadable. Ahasuerus 14:36, 4 June 2017 (UTC)

Audie Awards

Can we add the Audie Awards? They are not always given to specfic titles, but those often win awards. The lists of winners are here:

Just to give a couple sources. ···日本穣 · 投稿 · Talk to Nihonjoe 15:53, 4 June 2017 (UTC)

I am not familiar with these awards, but many of them seem to be given to the casts of dramatizations. For example:
  • DISTINGUISHED ACHIEVEMENT IN PRODUCTION
  • The Graveyard Book: Full-Cast Production
  • by Neil Gaiman
  • Narrated by a full cast including Neil Gaiman, Derek Jacobi, Robert Madge, Clare Corbett, Miriam Margolyes, Andrew Scott, and Julian Rhind-Tutt (HarperAudio)
The first link above claims that they are "an equivalent to the Oscar for the spoken-word industry". If so, then wouldn't they be "out of scope" for a fiction database just like the Oscar awards are ineligible? Ahasuerus 15:46, 5 June 2017 (UTC)
Regular, narrated audiobooks are given the awards, too. It might be limited to just a few categories, though, as there are (as you said) a number of awards which would be outside our scope (the dramatizations). ···日本穣 · 投稿 · Talk to Nihonjoe 15:52, 5 June 2017 (UTC)
I see. Are the awards given to narrated audiobooks given to their authors or to the narrators? Ahasuerus 16:37, 5 June 2017 (UTC)
They are given to the work. ···日本穣 · 投稿 · Talk to Nihonjoe 03:43, 6 June 2017 (UTC)
Do they have a Web page describing the selection/nomination process, who gives the award, etc? Ahasuerus 23:24, 6 June 2017 (UTC)
Not that I can see. ···日本穣 · 投稿 · Talk to Nihonjoe 23:35, 6 June 2017 (UTC)
Hm... It's difficult to decide what we want to do about this award with so little information. I wonder if there may be a place where we could ask. Perhaps some Wikipedia editors specializing in media research may know? Ahasuerus
It is an APA award. The site has only a "Publishers and rights holders enter titles in various categories for recognition of achievement. Finalists are selected, and from that group of finalists one winner is awarded." note. This PDF is the call for entries for this year containing also the eligibility and contest rules. However - I disagree that this is an award for a title - it is an award for a publication, not for the title itself. :) Annie 18:04, 7 June 2017 (UTC)
Which is why I used the word "work". :) ···日本穣 · 投稿 · Talk to Nihonjoe 23:23, 7 June 2017 (UTC)
In my mind "work" means the title here - not the publications. :) Annie 23:27, 7 June 2017 (UTC)
Poe-tay-toh, poe-tah-toe, and all that. :) ···日本穣 · 投稿 · Talk to Nihonjoe 16:19, 8 June 2017 (UTC)

(unindent) The Wikipedia page for the award says that the award is given to "outstanding audiobooks and spoken-word entertainment", and seems to emphasize the quality of the reader. Nevertheless, the awards themselves seem to be given to specific books and authors, and the reader is not generally named in the award. That makes it look to me like the emphasis is on the book itself, hence the award would fall within our scope. Chavey 00:31, 8 June 2017 (UTC)

Exactly, other than those awards that specifically mention something like "best dramatization" or similar, where it's obvious the award is for a cast of characters or narrators. ···日本穣 · 投稿 · Talk to Nihonjoe 16:19, 8 June 2017 (UTC)
Checking their category descriptions, I see that the genre-specific ones say that they are "for excellence in narration, production, and content of a [SF/mystery/romance/etc] audiobook". "Audiobook of the Year" says:
  • This award recognizes a title with high-quality content and production values that stands as a benchmark of excellence for the industry. Additional items weighed by the jury include success of marketing and publicity, visibility, impact, commercial success, and recruitment of new listeners to the format. The Audiobook of the Year should ser ve as a worthy ambassador to new and current listeners, and be a paragon of the audiobook art."
Although the category-specific ones mention "content" as one of the selection criteria, it would appear that for the most part these awards are given for the narration/production component and therefore probably ineligible. Ahasuerus 04:24, 9 June 2017 (UTC)
Well, they are awards for an entirely audio industry, so that make sense. :)
Still, I think if the work in question is simply narrated/read rather than dramatized, it should be eligible here. That seems to be the line I've seen in the past. Basically, if there is one (or two at the most) narrator(s), it should be fine. When it starts getting a cast, that's where we should draw the line. ···日本穣 · 投稿 · Talk to Nihonjoe 05:37, 9 June 2017 (UTC)
Narrated stories/novels are eligible for inclusion as publication records. However, I don't think that an award which considers narration/production, not to mention "success of marketing and publicity, visibility, impact, commercial success, and recruitment of new listeners to the format", can be said to be given to titles. Ahasuerus 15:35, 9 June 2017 (UTC)
Which was my point above - it is given to publications, not titles. So unless if we want to start having new variants for each narrator (as we do for translators) or even for each separate production, I am not sure that we really can include the award here... And I really do not want to see the DB go that path - translators change the text after all; narrators just read it... Annie 15:56, 9 June 2017 (UTC)
Okay, whatever. ···日本穣 · 投稿 · Talk to Nihonjoe 16:08, 9 June 2017 (UTC)
Maybe start adding notes in the publications that win? This way we will have the record and if one day there are pub-level awards here, it can be converted. We may need a new generic awards template (something like {{Award|Name_of_award|Category/Year}}... This will be very useful for other cases as well anyway (international awards that are not eligible to be added as awards here but are still important in the context of the local fandom and speculative fiction. Annie 18:03, 9 June 2017 (UTC)
That could be useful. Can we get that created to be used for awards not currently in the system? ···日本穣 · 投稿 · Talk to Nihonjoe 21:08, 9 June 2017 (UTC)
I am afraid that ISFDB templates -- as opposed to Wiki templates -- can take only one parameter at this time :-( Ahasuerus 01:58, 13 June 2017 (UTC)

Changes to "View Submission" pages

All "View Submission" pages have been modified to display a link to the newly created record. Assuming that one was created, of course :) Ahasuerus 01:56, 13 June 2017 (UTC)

New cleanup reports for external identifiers

The following "external identifiers in notes" cleanup reports have been added:

  • SFBG (26 pubs)
  • Deutsche Nationalbibliothek (5,000+ pubs, limited to the first 500 pubs for performance reasons)
  • FantLab (2 pubs)
  • Direct Amazon links (700+ pubs)
  • BNF (1,100+ pubs)
  • Library of Congress (11,000+ pubs, limited to the first 500 pubs for performance reasons)
  • OCLC/WorldCat (74,000+ pubs, limited to the first 500 pubs for performance reasons)

The data will become available tomorrow morning. Ahasuerus 20:57, 13 June 2017 (UTC)

Any way to automate the last two at least partially (and DNB as well)? Otherwise that will be one very long project... Annie 21:34, 13 June 2017 (UTC)
Well... The reason why I was able to create a script to auto-move 90%+ of our ASINs to the new field was that they had been originally entered by Fixer. Now, Fixer is a well-behaved robot, even if I say so myself. He has always used the same Note format, which made it easy to extract ASINs from Notes and move them to the new field.
On the other hand, our OCLC/LoC/DNB/BnF/BL/etc links were originally entered by humans. Admittedly, humans have their uses, but consistency isn't one of their strengths. A few random examples:
I have run a bunch of tests and it looks like it may be possible to auto-move certain LCCNs and OCLC numbers safely, but I'll need to spend more time on the logic to make sure that I don't accidentally zap something. In the meantime I have decided to make the data available for cleanup in case anyone wants to tackle it. Ahasuerus 22:50, 13 June 2017 (UTC)
How about semi-automatic (as we did with the title languages)? Identify a syntax, get them on a report for clearing (so all that do not match whatever we are looking for exactly but matches because of humans being humans, get manually fixed); the remainder get direct replacements. That should reduce the numbers (leaving the stragglers and non-patterned messes to deal with last) and reduce the moderators work on the ones that standard (when an editor and moderator are not the same person)... Annie 22:56, 13 June 2017 (UTC)
Hm, it may just work! I'll take a closer look once I finish the cleanup patch that I am currently working on. Ahasuerus 23:13, 13 June 2017 (UTC)
Having just moved several hundred of the OCLC links, I think it's safe to say that most of them fit under a very specific format, with a single OCLC link on a single line, with no additional text -- slight variation in whether it's "OCLC" or "OCLC:", whether the line ends in a period or not, and the format of the URL. Those should be able to be moved semi-automatically. Other links include additional text on the line ("OCLC xxx, which gives a page count of 289") that necessarily have to be moved by a human so as to keep the additional information. Some links don't match the "usual" format, such as the editor who likes to have the linking text be the word "WorldCat" instead of the actual OCLC number. But if you could pull out those standard formatted OCLC (and LCCN) numbers, it would put the remainder within much easier reach. Chavey 07:13, 15 June 2017 (UTC)
(Added) I would focus on notes in which there is a single line of text containing the OCLC reference, and the OCLC reference fits one of the most standard patterns. I think you could even write a regular expression that matches such lines. Chavey 07:42, 15 June 2017 (UTC)
Yup, those are the patterns that I have been looking into :) Ahasuerus 14:47, 15 June 2017 (UTC)
I did about 600 of the OCLC records yesterday, using a semi-automated script that required me to look at everything before submission. Partly the idea was to see if there were other issues that would interrupt the usefulness of the script you're working on. I identified fairly few additional issues, but I'll mention them. It was very uncommon to see any of the "easily" converted OCLC listings on anything other than a single line in the Notes. (We have one editor who likes to enter notes as a continuous flow of text with no manual line-breaks, but that's quite rare.) The beginning of an appropriate line can start in various ways, almost exclusively (deleting angle brackets) li, br, br•, or one of these with an extra space. Appropriate OCLC lines end with a manual return, a period, /li, or br (again, ignoring brackets and spaces). Almost any other ending probably requires a human to look at it. If there is more than one potential OCLC # on a line, the record should be left for humans. These oddities are so infrequent that, IMHO, it isn't worth the effort to try to find/correct them in the script; the lines meeting the criteria above will do 95% of the OCLC external IDs. Chavey 16:52, 16 June 2017 (UTC)
Many thanks! Ahasuerus 18:13, 16 June 2017 (UTC)
One other little point: In a substantial number of cases, removing the line with the OCLC number (or that and the LCCN #) leaves a single line of notes, but still inside the ul-li-/ul paradigm. While that's not an actual problem, the majority of our editors don't use the ul-li format or the br• format when there's only one note, leaving that single note to be on the same line as the Notes: header. Doing things manually, I converted all such Notes to the single line format, which would not be natural when doing it via a script. My guess though is that this detail is not worth worrying about, and the advantages of getting most of these numbers moved via scripting well outweighs the advantages of one format style over the other for such single line "Notes". Chavey 16:52, 16 June 2017 (UTC)
An empty "ul" list is invalid in HTML, but it too can be removed programmatically. Ahasuerus 18:13, 16 June 2017 (UTC)
There were only a couple of hundred records where the OCLC line ended with (/li) (in brackets), so I just converted them all. There are about 720 records where the OCLC line ends in (br), which is doable by hand. So if you wanted to simplify your developing script, you could assume that the lines we're looking for all end in (/a) with the possible addition of a single period and/or additional white space. Of course there are a ton of records that have the OCLC line without a URL at all, so the pattern looks like (the various starts of lines)OCLC(optional :)(optional space) number (optional period)(optional space). Chavey 04:22, 18 June 2017 (UTC)

(unindent) What are you looking for exactly for the FantLab report? As of now, it is empt but I keep finding links in the Russian works. I can find them out and move them easily enough but it seems like the report does not look for the expected format... for example - it has a standard link to https://fantlab.ru/edition52157. Annie 19:33, 20 June 2017 (EDT)

The original logic was limited in scope in order to avoid false positives. As it turns out, it missed a whole class of FantLab links. I have tweaked the logic and added the ability to "ignore" publication records. The new data will become available tomorrow morning. Thanks for raising the red flag! Ahasuerus 21:50, 20 June 2017 (EDT)
I knew the number was too low but figured you moved them with a script or something - the FantLab links are pretty standardized for the most part. Then started finding the ones today and figured I will ask. Thanks! Annie 21:56, 20 June 2017 (EDT)

(unindent) A question about the Publications with direct Amazon links in Notes: most of the still standing links are because a direct link to an Amazon page (using ISBN or ASIN) or LookInside (based on ISBN or ASIN). Do we want to ignore these? Or do we want to clean the Amazon links? All valid ISBNs and ASINs are usually already extracted; it is just that we have an additional link needed. The direct links (when to amazon.com can easily be replaced with the ASIN template (ASIN is the 10 character ISBN when it exists) but we have links to amazon.de, amazon.jp, co.uk and all those LookInsides. Thoughts? Annie 23:37, 25 June 2017 (EDT)

If an explicit link can be replaced with a template, then I would suggest using that template. If it can't be replaced, e.g. if the note links to Wayback Machine snapshots of Amazon pages, then I don't think we have any choice but to "ignore" the affected record. Ahasuerus 00:40, 26 June 2017 (EDT)
That's what I started doing and then stopped to post - because that left a lot of direct links over there. What would reduce them is a LookInside template (it is always https://www.amazon.com/gp/reader/<ASIN> unless if it is to another Amazon but we cannot help that). Does it make sense to add it? Or even a series of LookInside to the big amazons? We have ~350 of those links (searching for "gp/reader/" in Notes). Annie 00:51, 26 June 2017 (EDT)

Post-submission bug fixed

The post-submission bug introduced about an hour ago has been squashed. Sorry about that! Ahasuerus 01:30, 14 June 2017 (UTC)

British Library ISBN links

British Library ISBN links (accessible under "Other Sites") should be working again. Ahasuerus 22:58, 14 June 2017 (UTC)

Andrew M. Greeley: Nongenre?

I am pretty sure most of what's on Andrew M. Greeley's page is non-genre-- can anyone comment? To be more specific, my rather cursory search of Goodreads suggests that the Family Saga, Father Blackie Ryan, Passover, Time Between the Stars, and World of Maggie Ward series don't belong, and none of the nonfiction does. --Vasha 00:07, 15 June 2017 (UTC)

It is kinda funny - I was looking for something else earlier today and stumbled on one of his non-fictions - and submitted a "non-genre" update because it is clearly not genre. Looking at the full list now, none of this non-fiction sound even remotely spec-fic... And reading the note on SFE3, clearly most of his fiction is not genre either... Annie 00:33, 15 June 2017 (UTC)
Ready to exterminate... (in the afternoon).Hauck 06:14, 15 June 2017 (UTC)
If memory serves, an editor looked into this author's output a while back and discovered that a number of seemingly non-genre works contained genre elements (perhaps psychic abilities?) We may want to do additional digging before zapping his series. Ahasuerus 14:20, 15 June 2017 (UTC)
So far, I've only found Angels of September to be vaguely supernatural, the rest being mysteries. Note that at the present time there was only one publication that was verified (out of nearly a hundred) which is quite telling. Hauck 15:04, 15 June 2017 (UTC)

<sarcasm> Adventures in Gaming </sarcasm>

My brain has shut down for the night and so maybe someone else can figure out how best to do this.

https://www.amazon.com/dp/1530312051#reader_1530312051 contains a story titled

<sarcasm> Adventures in Gaming </sarcasm>

You can't see the <sarcasm> in the title at http://www.isfdb.org/cgi-bin/title.cgi?2145123 but it's visible if you click "edit title".

The easy fix is to change the title to

&lt;sarcasm&gt; Adventures in Gaming &lt;/sarcasm&gt;

The problem is - someone searching for that title is more likely to use <sarcasm> than &lt;sarcasm&gt;

Is there a way to do this that would create the least astonishment? --Marc Kupper|talk 08:41, 15 June 2017 (UTC)

There is a patch in the works to auto-convert reserved HTML characters to their HTML-encoded representation at data entry time. The only exception will be our Note fields which will allow a limited subset of HTML. The patch will also modify the search logic to change entered reserved HTML characters to their HTML-encoded form. Ahasuerus 14:26, 15 June 2017 (UTC)
Thank you. Are you planning on using the decimal &#nnn; form or will you use the hex and/or named entities? I might as well fix the <sarcasm/> title but also want to normalize it to the form you'll use. --Marc Kupper|talk 18:25, 15 June 2017 (UTC)
The ISFDB software has been using entities like "& l t ;" all along, so that's what I plan to use in the patch that I have been working on (with Marty's help.) Ahasuerus 23:19, 15 June 2017 (UTC)

Lots of publication updates

During the evening I moved several hundred OCLC numbers from Notes into External IDs. Barely made a dent in the backlog, but for some editors this may have dumped a large number of "Changed Primary Verifications" into your report by that name, for fairly trivial changes. My apologies for that, but it seems inevitable as we try to move these IDs. Chavey 10:27, 15 June 2017 (UTC)

I am doing the same on the ASINs list and someone is working steadily on the British Library list. I dropped notes to the PVs that I see often (just so they do not freak out when they login) but your note is a good idea. Annie 17:37, 15 June 2017 (UTC)

Title merge bug fix

The following bug has been fixed:

  • Merging a SHORTFICTION title with a "length" value with a non-SHORTFICTION title can cause the resulting title to have a "length" value even if the title type is not SHORTFICTION.

Ahasuerus 18:13, 15 June 2017 (UTC)

"Editing Tools" change

The "Editing Tools" section of the navigation bar is no longer displayed if there are no editing options available. Moderators will not be affected by this change because the "Editing Tools" sections contains the main "Moderator" link. Ahasuerus 21:22, 15 June 2017 (UTC)

979 ISBNs

"979" ISBN-13s no longer display a bogus derived ISBN-10 on Publication pages. Ahasuerus 19:50, 16 June 2017 (UTC)

"Publication Format" --> "Binding"

All data entry forms have been changed to use "Binding" instead of "Pub. Format". This should synchronize the terminology used on bibliographic and data entry pages. Ahasuerus 20:26, 16 June 2017 (UTC)

Karel Čapek

We currently have the canonical name of Karel Capek without the hachek over the C in his name. We have his name in what I believe is the proper and more frequently appearing form as a pseudonym, Karel Čapek. I believe that most of the titles that currently have the author as "Karel Capek" are actually incorrectly entered. These were undoubtedly entered before we fully or easily supported other alphabets. I'd like to clean this up, but wanted to ensure there was no objection first since this will involve several verified publications. Thus, I'm doing a community notification because of the number of publications involve. My intent is to convert all titles to "Karel Čapek" and then make new variants for "Karel Capek" only for those few cases where it is truly appropriate. If there are any objections, please post them here. Thanks. --Ron ~ RtraceTalk 23:56, 16 June 2017 (UTC)

I agree that "Čapek" should be the canonical name just like "Stanisław Lem" is the canonical name.
As far as the process goes, I believe it would be safe to change the author of all of our Czech titles/publications to "Čapek". On the other hand, other languages may require additional digging. Some clearly use the original spelling, e.g. this German pub or this English pub. Others don't. For this reason I think it would be safer to convert his titles and pubs individually. Keeping title/pub author pairs in sync can be a pain when doing mass changes.
Another thing about our Čapek's bibliography page is that we list many stories which, if memory serves, are not SF. I read a number of his books some decades ago and most of his fiction wasn't SF. Ahasuerus 01:20, 17 June 2017 (UTC)

New Moderator nomination

It's been a while since there has been a new Mod. I think it's time. I would like to nominate the one and only [Annie]. She's indefatigable, conscientious, actually listens, and seems to understand the database extremely well. Participates [more than I do] avidly and intelligently. A great long-term addition. I'll leave the usual set-up process to Ahasuerus. Cheers all! --~ Bill, Bluesman 01:25, 18 June 2017 (UTC)

(Bureaucrat notes: See Moderator Qualifications. The nominee has accepted. Ahasuerus 14:55, 18 June 2017 (UTC))

Support

  1. Support, as nominator. ~ Bill, Bluesman 01:25, 18 June 2017 (UTC)
  2. Support. I think Annie would make an excellent addition to the moderators. She has been working on lots of parts of the system, which I think is crucial for a moderator, and is well-skilled in many parts of it. Chavey 04:15, 18 June 2017 (UTC)
  3. Support. C'est parti ! Hauck 05:52, 18 June 2017 (UTC)
  4. Support. Absolutely! PeteYoung 06:21, 18 June 2017 (UTC)
  5. Support. Agree! Should we nominate Nihonjoe as well? --Willem 08:14, 18 June 2017 (UTC)
  6. Support. I've been working with a bunch of Brits lately, and "brilliant!" immediately leaps to mind. Willem, I like your suggestion, too. --MartyD 11:04, 18 June 2017 (UTC)
  7. Support. Highly agree. -- JLaTondre (talk) 12:25, 18 June 2017 (UTC)
  8. Support. Agree, Annie's climb up the learning curve is the fastest that I have ever seen. —The preceding unsigned comment was added by Rkihara (talkcontribs) .
  9. Support. With over 20,000 submissions to her name, Annie is certainly familiar with the process. I believe she meets the rest of the requirements as well. Just don't hesitate to ask if you come across anything remotely unclear! :-) Ahasuerus 15:16, 18 June 2017 (UTC)
  10. Support. A natural! Stonecreek 15:36, 18 June 2017 (UTC)
  11. Support. I think she will do a fine job. ···日本穣 · 投稿 · Talk to Nihonjoe 05:23, 19 June 2017 (UTC)
  12. Support though I'd been wondering who Anniemod in the new-submissions queue at times is and assumed she was already a moderator. --Marc Kupper|talk 07:44, 20 June 2017 (UTC)

Oppose

Comments/Neutral

Outcome

The nomination was a success; the moderator flag has been set on the account. Congratulations! Ahasuerus 01:18, 23 June 2017 (EDT)

Submission view changes

The web pages which display submission data have been modified. They now show the date/time when the submission was create/approved/rejected as well as the name of the reviewing moderator. Ahasuerus 16:31, 18 June 2017 (UTC)

Along similar lines, all pages that show lists of "Recent Edits", "My Recent Edits", "Recent Rejections", etc have been modified to display more data about each submission. In addition, each page displays the current ISFDB server time to provide a frame of reference for the displayed information. Ahasuerus 20:44, 18 June 2017 (UTC)
"New Submissions" has been updated as well. Ahasuerus 19:26, 19 June 2017 (UTC)
While you are playing with those pages - can we have a "Are you sure you want to cancel?" popup on the "Pending" list's Cancel links? I won't even try to count how often I will open the page to see where I left it off and press cancel on one of the items (especially when working from one of the touchscreens or with the laptop pad... Usually it is something easy to redo but one of those days, I will wipe out something that is not that easy to reproduce... :)Annie 21:12, 19 June 2017 (UTC)
We can certainly create a Feature Request, but I am not sure it would help much. People tend to become accustomed to pop-ups and dismiss them without reading. Still, it would be better than nothing. Ahasuerus 22:33, 19 June 2017 (UTC)
And then it is their own fault that they canceled it. :) But just because I was in the vicinity of the request with a mouse and was distracted, I had to redo some requests. May be just me being clumsy - but the thing is just too annoying when it happens - and it is not an operation everyone needs to do too often - so it won't have much of a delaying factor for most editors. So thought I should mention it :) Annie 23:03, 19 June 2017 (UTC)
Personally I would find such a pop-up really annoying. And on the rare occasion that I cancel something I didn't intend to, I can just go to the Rejected Submissions, open the record of the one I just cancelled, and copy it to a new submission. Vasha 23:06, 19 June 2017 (UTC)

Translations - display changes

As per FR 1059, the way translations are labeled on Summary Bibliography and Series pages has been changed. The label, which used to say "Variant Title", now says "Variant" or "Translation" as appropriate. Here is what Heinlein's Future History" looks like now.

I believe that the new labeling system is more clear, but there is room for improvement. For example, we currently display variants and translations together, e.g.:

  • Variant: Life-Line (1939) [as by Robert Heinlein]
  • Translation: Ligne de vie [French] (1967)
  • Translation: Lebenslinie [German] (1972)
  • Variant: Lifeline (1977) [as by Robert Heinlein]
  • Translation: Levenslijn [Dutch] (1979)

My thinking is that we may want to display variants first and translations second:

  • Variant: Life-Line (1939) [as by Robert Heinlein]
  • Variant: Lifeline (1977) [as by Robert Heinlein]
  • Translation: Ligne de vie [French] (1967)
  • Translation: Lebenslinie [German] (1972)
  • Translation: Levenslijn [Dutch] (1979)
  • etc

Once we do that, we can consider other improvements. Ahasuerus 23:09, 19 June 2017 (UTC)

I like the idea of having them grouped together with variants before translations. I would make a case that we should also group the translations into a certain language (when more than one is done) together as well. That would make it easier to spot that you have a few translation in a language and to be extra careful which one you are importing. Annie 23:14, 19 June 2017 (UTC)
I think there are advantages and disadvantages to grouping either chronologically or by language. We'll also need to consider how this will affect the display of variants and translations on the Title page. Ahasuerus 01:27, 20 June 2017 (UTC)
I like Annie's suggestion. Chavey 02:12, 20 June 2017 (UTC)
What should we do with our Title pages then? Would you be in favor of ordering their "Translations" sections by language and then by year? Ahasuerus 03:08, 20 June 2017 (UTC)
Either that or by year of first translation into the language and then add the rest of the ones into the same language. So if the first ever translation is into French, French goes first but instead of followed by First English, you get all French first. Annie 03:11, 20 June 2017 (UTC)
I prefer the first solution with timeline (chronologically)--Wolfram.winkler 02:55, 22 June 2017 (EDT)

Variants and disambiguated authors

I am looking at this and I am confused on what is the idea here. Shouldn't we just change the author name inside of the two publications to the (I) variant of the name and be done with it? Yes, they did not have the (I) in the actual publication but that is the one case where we do not record as written in the book because of operational reasons. Am I missing something here or does it need fixing (change author inside of the pubs and delete this unneeded variant)? Annie 17:48, 20 June 2017 (UTC)

You are correct. Fixed. -- JLaTondre (talk) 14:41, 20 June 2017 (EDT)
Thanks! :) Annie 14:45, 20 June 2017 (EDT)

ISBN template

While moving FantLab links, I am also adding the additional ISBNs where applicable. Can we have a template so they are easier to spot and run a report on when we eventually have the ability to add multiple ISBNs? Annie 22:00, 20 June 2017 (EDT)

Good point, will do! Ahasuerus 22:46, 20 June 2017 (EDT)
Thanks! And talking about multiple ISBNs and Publishers - in the Russian case they are (almost?) always paired (an ISBN per publisher). I am yet to find one that does not match that rule - but it is not impossible. I know we cannot do a 2 parameter template so I will use the notes to pair properly but I hope that when we have the feature one day, we will be able to pair them as well when applicable. Or I am overthinking again... Annie 23:13, 20 June 2017 (EDT)
OK, {{ISBN}} has been added.
Re: publisher-specific ISBNs, I am not sure even Russian publishers are consistent. For example, take this Russian translation of The Lord of the Rings. The FantLab page lists two publishers, one additional imprint, four ISBNs and two printings. Ahasuerus 12:40, 21 June 2017 (EDT)
Thanks! For the publisher-specific ISBNs: АСТ Москва is a separate publisher even if it is part of the АСТ group and adds its own ISBN when published. So we have 3 publishers here:). I am seeing the АСТ Москва (5-9713-2149-8), the АСТ (5-17-018845-5) and the Terra Fantastica (5-7921-0618-5) ISBNs. The 4th one is a Belorussian one - I need to do some more digging but I think I had seen that before - Belorussian distribution only ISBN from АСТ - will be back with more details as soon as I find them again (or find my notes wherever they may be). So yeah, they are consistent (mostly :) ) and even if we cannot pair all of them, some pairing is possible. Annie 15:49, 21 June 2017 (EDT)
Plus it looks like that last ISBN could have been only on the additional printing (being ISBN-13) and even then 2006 is stretching it so I wonder if it is even valid for that book or if it was a distribution only one added later and one that was on whatever copy was looked at for the entry. In which case it does not belong on the record really... so the consistency holds. Annie 16:14, 21 June 2017 (EDT)
Well, ultimately we will want to allow multiple publishers per publication. At that point we will have to decide whether we want to associate ISBNs with individual publishers. In certain cases it may be feasible (see above), but in many other cases it may not be, e.g. when a jointly published book has only one ISBN.
More importantly, we will first have to decide how we want to reorganize our publisher records, including:
  • imprints vs. publishers
  • publishers becoming imprints after an acquisition
  • imprints getting spun off as separate publishers
  • "stated published" vs. "regularized publisher"
  • multiple forms of the publisher name used in the same publication
etc. Ahasuerus 17:49, 21 June 2017 (EDT)
Understood :) I was just adding that to the mental notes of the thinking - I don't like having derivable information just sitting because we did not structure properly for it. That's all. Being elbows deep in the Russian mess of ISBNs make me think of all kind of weird things about easier way to figure out what is what. In the long run, it won't really matter that much - it was just my curiosity a few years ago to untangle the mess that made me aware of how the Russian ISBNs work at all. And I have a suspicion that each country may have an oddity or two hiding under the hood. Annie 18:01, 21 June 2017 (EDT)

Apex publisher name?

Apex Book Company is the same company as Apex Publications and as far as I can see the two names are used interchangeably. Does anyone know if there's actually some reason to use either one or the other? --Vasha 11:20, 21 June 2017 (EDT)

I cannot see a reason to have them separate unless if we want to keep the magazine publisher from the book publisher (and then a lot of books need moving). Annie 16:18, 21 June 2017 (EDT)
I think Apex Publications is more general (FWIW I have two Apex books and they both say Publications throughout); but do we ever do alternate/pseudonym publisher names? ABC redirected to AP? --Vasha 18:24, 21 June 2017 (EDT)
In the system now? Only soft-linked via notes. Or in some cases the notes contain the alternative names. We chose one for "the name" and that is it in this system. IMHO opening the door for pseudonyms will end up with abolishing the regularization and will make a mess with mergers and splits and what's not... Annie 18:33, 21 June 2017 (EDT)
I think we will need a single comprehensive solution to the "publisher problem". It will have to address the issues of multiple publishers, imprints and name regularization at the same time. We'll need to come up with a solid design first. Ahasuerus 18:36, 21 June 2017 (EDT)
I tried that Verified Publishing Names thing for a while but it never gained traction. I've drifted towards recording less detail and not being as picky about inconsistencies in the details. If a bunch of people PV a publication as a Bantam/Spectra but I only see "Bantam" for example I'm likely to let it slide. Maybe I'll add a pub note but that means notifying people and creating work to fetch and recheck a pub for an area that the consensus does not pay attention to.
I also discovered that as we record more details that it increases the chance of error, inconsistencies, and subjective calls. I assume it also increases the verification load on later PVers as they all presumably need to check and compare a detail that's not part of what we normally cover. The only exception to this is that my personal PV process for collections and anthologies is to double check the spelling of the titles, author names in the Contents, copyright acknowledgments, and the start of each work. I'll document inconsistencies on the theory that it would explain why different sources have slightly different data about a topic.
I believe this was an area where the publication wiki pages were useful as we could record extra details that were not part of the primary verification record. I did not care that the wiki part was not moderated as the edit history is available and it was never something that was formally part of the PVed information. Someone doing a later PV could chose to ignore the link to the wiki page. As long as the main record matched we had the data that most of us care about.
I don't have any Apex publications and so can't answer the original thread topic. --Marc Kupper|talk 20:06, 21 June 2017 (EDT)
That Verified Publishing Names thing looks like a lot of work-- I can understand why no one devoted themself to it for long enough to get it to a usable state. --Vasha 21:11, 21 June 2017 (EDT)
I agree, it was a lot of work. Something that could have simplified the workload would be to include scans of the title page, copyright page, and other parts of the publications that seemed relevant. That also would have allowed for future projects such as confirming the copyright date, or other things.
On the Apex issue - There are Amazon Look Insides available for the 2015 and 2016 books. All of the Apex Book Company publications are by "Apex Publications, LLC." per the copyright pages meaning the proposed merge is safe. The title pages have "An Apex Publications Book" over "Lexington, Kentucky". We are missing 978-1-937009-31-1 and 978-1-937009-35-9 on ISFDB.
2015
978-1-937009-31-1 For Exposure hardcover - look inside is of tp
978-1-937009-35-9 The Apex Book of World SF: Volume 2 - Apex Publications
For Exposure: 978-1-937009-30-4 - Apex Publications
King of the Bastards 978-1-937009-32-8 - title page and copyright page not available. Back cover has Apex Publications
The Apex Book of World SF 978-1-937009-36-6 - Apex Publications
The Apex Book of World SF: 4 978-1-937009-33-5 - Apex Publications
The Apex Book of World SF 3 978-1-937009-34-2 - Apex Publications
2016
Best of Apex Magazine: Volume 1 978-1-937009-37-3 - Apex Publications
978-1-937009-38-2 - 404 not found on Amazon
978-1-937009-39-1 - 404 not found on Amazon
978-1-937009-40-7 - 404 not found on Amazon
978-1-937009-41-6 - 404 not found on Amazon
978-1-937009-42-5 - 404 not found on Amazon
978-1-937009-43-4 - 404 not found on Amazon
978-1-937009-44-3 - 404 not found on Amazon
978-1-937009-45-2 - 404 not found on Amazon
978-1-937009-46-1 - 404 not found on Amazon
978-1-937009-47-0 - 404 not found on Amazon
978-1-937009-48-9 - 404 not found on Amazon
978-1-937009-49-8 - 404 not found on Amazon
2017
978-1-937009-50-4 - 404 not found on Amazon - Ugly As Sin on ISFDB
978-1-937009-51-3 - 404 not found on Amazon - The Wicked on ISFDB
978-1-937009-52-2 - 404 not found on Amazon
978-1-937009-53-1 - 404 not found on Amazon - Beautiful Sorrows on ISFDB
978-1-937009-54-0 - 404 not found on Amazon - Greener Pastures on ISFDB
978-1-937009-55-9 - 404 not found on Amazon
978-1-937009-56-8 - 404 not found on Amazon
978-1-937009-57-7 - 404 not found on Amazon
978-1-937009-58-6 - 404 not found on Amazon
978-1-937009-59-5 - 404 not found on Amazon
--Marc Kupper|talk 06:00, 22 June 2017 (EDT)
Wow, thanks for checking all that. I've put a request for the merge on the moderator page.
You have mistakes in those ISBNs you couldn't find. Ugly as Sin is 978-1-937009-50-2; The Wicked is 978-1-937009-51-9; Beautiful Sorrows is 978-1-937009-53-3; etc. (wrong final digits) --Vasha 07:31, 22 June 2017 (EDT)
Thanks for catching that. It turns out I had bugs in the code for a web page that lets me enter an ISBN as 978-1-937009-5?-?. It's supposed to generate a list of the ISBNs with each linking to amazon.com. It makes it easy to scan a range of ISBNs. It's been working well for ISBN10s but had bugs for ISBN13s. Here's an updated list formatted as a table. Non-linked titles are pubs we don't have. All of them seem to be specfict and we already have the authors.
ISBN13AmazonTitle/publicationAuthor
978-1-937009-00-7 1937009009 The Tower of the Forgotten Sara M. Harvey
978-1-937009-01-4 1937009017 Escape from Zombie City: A One Way Out Novel Ray Wallace
978-1-937009-02-1 1937009025 The Fields Ty Schwamberger, Jonathan Maberry
978-1-937009-03-8 1937009033 Disintegration Visions J.M. McDermott
978-1-937009-04-5 1937009041 On ISFDB but not Amazon
978-1-937009-05-2 193700905X The Apex Book of World SF 2 Lavie Tidhar
978-1-937009-06-9 1937009068 Appalachian Undead Eugene Johnson
978-1-937009-07-6 1937009076 Dark Faith: Invocations Maurice Broaddus, Jerry Gordon
978-1-937009-08-3 1937009084 Seasons of Insanity Gill Ainsworth, Frank W. Haubold
978-1-937009-09-0 1937009092 Like Death Tim Waggoner
978-1-937009-10-6 1937009106 The Lost Level Brian Keene
978-1-937009-11-3 1937009114 The Book of Apex Catherynne M. Valente
978-1-937009-12-0 1937009122 What Makes You Die Tom Piccirilli
978-1-937009-13-7 1937009130 Machine Jennifer Pelland
978-1-937009-14-4 1937009149 Desper Hollow Elizabeth Massie
978-1-937009-15-1 1937009157 Plow the Bones (Apex Voices) (Volume 1) Douglas F. Warrick
978-1-937009-16-8 1937009165
978-1-937009-17-5 1937009173 I Can Transform You (Apex Voices) (Volume 2) Maurice Broaddus, Matt Forbeck
978-1-937009-18-2 1937009181 Appalachian Undead Jonathan Maberry, John Skipp, Gary A. Braunbeck, Jason Sizemore, Eugene Johnson
978-1-937009-19-9 193700919X Glitter & Mayhem Lynne M. Thomas, John Klima, Michael Damian Thomas, Amber Benson
978-1-937009-20-5 1937009203 The Book of Apex Lynne M. Thomas
978-1-937009-21-2 1937009211 Maze J.M. McDermott
978-1-937009-22-9 193700922X Mountain Dead Geoffrey Girard, Sara M Harvey, Lesley Conner, K. Allen Wood, Jason Sizemore, Eugene Johnson
978-1-937009-23-6 1937009238 Midnight Mari Adkins
978-1-937009-24-3 1937009246 The Apex Book of World SF 3 Benjanun Sriduangkaew, Lavie Tidhar
978-1-937009-25-0 1937009254
978-1-937009-26-7 1937009262 War Stories: New Military Science Fiction Karin Lowachee, Andrew Liptak, Jaym Gates
978-1-937009-27-4 1937009270 Severance Chris Bucholz
978-1-937009-28-1 1937009289 Sing Me Your Scars (Apex Voices) (Volume 3) Damien Angelica Walters
978-1-937009-29-8 1937009297 Rosewater Tade Thompson
978-1-937009-30-4 1937009300 For Exposure: The Life and Times of a Small Press Publisher Jason Sizemore
978-1-937009-31-1 1937009319 For Exposure: The Life and Times of a Small Press Publisher Jason B Sizemore
978-1-937009-32-8 1937009327 King of the Bastards Brian Keene, Steven L. Shrewsbury
978-1-937009-33-5 1937009335 The Apex Book of World SF: Volume 4 (Apex World of Speculative Fiction) Usman T. Malik, Mahvesh Murad, Lavie Tidhar
978-1-937009-34-2 1937009343 The Apex Book of World SF: Volume 3 (Apex World of Speculative Fiction) Karin Tidbeck, Lavie Tidhar
978-1-937009-35-9 1937009351 The Apex Book of World SF: Volume 2 (Apex World of Speculative Fiction) Lauren Beukes, Lavie Tidhar
978-1-937009-36-6 193700936X The Apex Book of World SF: Volume 1 (Apex World of Speculative Fiction) S.P. Somtow, Lavie Tidhar
978-1-937009-37-3 1937009378 Best of Apex Magazine: Volume 1 Ursula Vernon, Lesley Conner, Jason Sizemore
978-1-937009-38-0 1937009386 Freeze/Thaw Chris Bucholz
978-1-937009-39-7 1937009394 Apex Magazine 2015 Sampler Jason Sizemore
978-1-937009-40-3 1937009408 The Kraken Sea E. Catherine Tobler
978-1-937009-41-0 1937009416 first communions Geoffrey Girard
978-1-937009-42-7 1937009424 Glitch Rain Alex Livingston
978-1-937009-43-4 1937009432 Stay Crazy Erica L. Satifka
978-1-937009-44-1 1937009440 Upside Down: Inverted Tropes in Storytelling Monica Valentinelli, Gates Jaym
978-1-937009-45-8 1937009459 Kentucky Kaiju Shawn Pryor, Justin Stewart, Tressina Bowling
978-1-937009-46-5 1937009467 Upside Down: Inverted Tropes in Storytelling Valentinelli Monica, Gates Jaym
978-1-937009-47-2 1937009475 Yours to Tell: Dialogues on the Art & Practice of Writing Steve Rasnic Tem, Melanie Tem
978-1-937009-48-9 1937009483 Apex Magazine SFFH: Issue 0, Winter 2017 Lavie Tidhar, Douglas F. Warrick, A. Merc Rustad, K. T. Bryski, Jason Sizemore
978-1-937009-49-6 1937009491
978-1-937009-50-2 1937009505 Ugly As Sin James Newman
978-1-937009-51-9 1937009513 The Wicked James Newman
978-1-937009-52-6 1937009521 Mars Girls Mary Turzillo
978-1-937009-53-3 193700953X Beautiful Sorrows Mercedes M. Yardley seems non-genre though
978-1-937009-54-0 1937009548 Greener Pastures Michael Wehunt, Michael Bukowski
978-1-937009-55-7 1937009556 Shine Your Light On Me Lee Thompson
978-1-937009-56-4 1937009564
978-1-937009-57-1 1937009572
978-1-937009-58-8 1937009580
978-1-937009-59-5 1937009599
978-1-937009-60-1 1937009602
978-1-937009-61-8 1937009610
978-1-937009-62-5 1937009629
978-1-937009-63-2 1937009637
978-1-937009-64-9 1937009645
978-1-937009-65-6 1937009653
978-1-937009-66-3 1937009661
978-1-937009-67-0 193700967X
978-1-937009-68-7 1937009688
978-1-937009-69-4 1937009696
978-1-937009-70-0 193700970X
978-1-937009-71-7 1937009718
978-1-937009-72-4 1937009726
978-1-937009-73-1 1937009734
978-1-937009-74-8 1937009742
978-1-937009-75-5 1937009750
978-1-937009-76-2 1937009769
978-1-937009-77-9 1937009777
978-1-937009-78-6 1937009785
978-1-937009-79-3 1937009793
978-1-937009-80-9 1937009807
978-1-937009-81-6 1937009815
978-1-937009-82-3 1937009823
978-1-937009-83-0 1937009831
978-1-937009-84-7 193700984X
978-1-937009-85-4 1937009858
978-1-937009-86-1 1937009866
978-1-937009-87-8 1937009874
978-1-937009-88-5 1937009882
978-1-937009-89-2 1937009890
978-1-937009-90-8 1937009904
978-1-937009-91-5 1937009912
978-1-937009-92-2 1937009920
978-1-937009-93-9 1937009939
978-1-937009-94-6 1937009947
978-1-937009-95-3 1937009955
978-1-937009-96-0 1937009963
978-1-937009-97-7 1937009971
978-1-937009-98-4 193700998X
978-1-937009-99-1 1937009998 Let's Play White Chesya Burke
Beautiful Sorrows seems non-genre. --Marc Kupper|talk 16:32, 22 June 2017 (EDT)
Kentucky Kaiju is a graphic novel; I added the other missing ones. --Vasha 18:41, 22 June 2017 (EDT)

Vote changes

As per FR 1049, the way votes are displayed on Title pages has been changed. Previously, a title had to have at least 5 votes before the average was displayed. With the new system the average rating is displayed regardless of how many votes have been cast. The number of votes is displayed in parentheses.

For example, the title page for Nine Princes in Amber now says:

  • User Rating: 9.22 (36 votes)

The part of the FR which requests to "calculate an additional "Combined User Rating" across the main parent title and its variant titles" hasn't been implemented yet. Ahasuerus 18:58, 21 June 2017 (EDT)

I am not seeing votes for titles I voted on (although I do see it for Nine Princes in Amber). For example Hunted says "This title has no votes" even though my vote is there. Do you see Hunted differently? --Vasha 21:20, 21 June 2017 (EDT)
Oops! It's a display problem which affects titles that have fewer than 5 votes recorded prior to the change. It's just a display issue, so no data has been lost. Sorry about that! I'll fix it tomorrow morning. Ahasuerus 21:55, 21 June 2017 (EDT)
Fixed. Ahasuerus 14:45, 22 June 2017 (EDT)
The Title page has been modified to display the currently logged-in user's vote (if present.) Ahasuerus 16:48, 22 June 2017 (EDT)
The part of the FR which asked to "calculate an additional "Combined User Rating" across the main parent title and its variant titles" has been implemented. For example, the Title page for The Hobbit, or There and Back Again now says:
  • User Rating: 7.52 (21 votes) Including variants and translations: 7.60 (42 votes)
Ahasuerus 18:30, 22 June 2017 (EDT)

Cloning and External IDs

A new chechbox has been added to the intermediate cloning page. It asks the editor whether to re-use external IDs. By default the box is not checked. Ahasuerus 22:45, 22 June 2017 (EDT)

"Other [records] with the same name"

A while back we enhanced the Summary Bibliography page to display a note if there were other authors with the same name, e.g. Andrew Smith (V). The software has been further modified to display similar warnings when viewing disambiguated series, publication series and publisher pages. Ahasuerus 17:26, 23 June 2017 (EDT)

Is there a reason placement of the series one is not consistent with the other three? -- JLaTondre (talk) 17:34, 23 June 2017 (EDT)
Yes, indeed! It's known as "human error" :-) , now fixed. Ahasuerus 17:59, 23 June 2017 (EDT)
They look at the same place for all 3 elements for me? Annie 17:57, 23 June 2017 (EDT)
That's because I fixed the Series page 3 minutes ago :) Ahasuerus 17:59, 23 June 2017 (EDT)
Ha, testing done before you even managed to announce it. Annie 18:22, 23 June 2017 (EDT)

New cleanup report: Titles with mismatched parentheses

A new cleanup report, "Titles with mismatched parentheses", has been deployed. The data will become available tomorrow morning. Of the 232 suspect titles that I expect it to find about 20% will be false positives, so the report lets moderators "ignore" records. Ahasuerus 19:50, 23 June 2017 (EDT)

Down to 2 - which may need an ignore but I am still to talk to the verifiers. Annie 19:23, 25 June 2017 (EDT)
Thanks! Ahasuerus 11:52, 26 June 2017 (EDT)
The verifier fixed those 2 remaining ones so we are all done with Titles with mismatched parentheses legacy issues. Anything showing from now on will be newly entered data. Annie 15:29, 4 July 2017 (EDT)

Award pages and parent titles

As per FR 1058, the following pages have been enhanced to display parent titles:

  • The Award Bibliography page, e.g. see this page
  • The Award Year page, e.g. see this page -- note how "The Dark Forest", "Folding Beijing" and "Hollowgirl" are displayed

Ahasuerus 18:21, 24 June 2017 (EDT)

I like it! ···日本穣 · 投稿 · Talk to Nihonjoe 20:06, 24 June 2017 (EDT)
Very nice! Annie 22:16, 24 June 2017 (EDT)

ASIN IDs - Display changes

The way ASIN IDs are displayed has been changed. In addition to the 6 Amazon stores that were already supported (Canada, France, Germany, Japan, UK, US), we now link to the Amazon stores in Australia, Brazil, China, Spain, India, Italy, Mexico and the Netherlands. The Chinese store seems to be missing a fair number of ASINs, but the ones that are available seem to use the same ASINs as the rest, e.g. see this publication. Please note that the "Other Sites" section of the navigation bar on the left hasn't been changed (yet.) Ahasuerus 15:38, 25 June 2017 (EDT)

I still wonder if all the external IDs should be moved out from the top panel - and it their own one or in the Secondary verifications one. This way the top will remain for what the book is and it won't look too crowded when we have no notes (or single lines). Annie 17:25, 25 June 2017 (EDT)
There are certain advantages to displaying External IDs immediately below the Note field. For example, if the Note field says "Data from BNF", then having the BNF number right below it makes it easier to figure out what it's talking about (and follow the link if needed.) Ahasuerus 19:15, 25 June 2017 (EDT)
Oh, I know - but when they take over visually, the notes are becoming invisible. Annie 19:17, 25 June 2017 (EDT)
Well, how about we move them below the "Upload new cover scan" line? It would hopefully separate the Note field and the ID area. Ahasuerus 20:49, 25 June 2017 (EDT)
That may work - it will push these under the image (or low enough so it does not crowd the Notes. Annie 20:52, 25 June 2017 (EDT)
Let's see what other editors think. In the meantime Amazon has made certain changes that affect Fixer (effective almost immediately), so I am mega busy trying to mitigate the fallout. Ahasuerus 13:19, 26 June 2017 (EDT)
Not urgent at all -- good luck dealing with the Amazon changes! I will stop bugging you and go fix some ASINs :) Annie 13:23, 26 June 2017 (EDT)

A new cleanup report: ISBN-less e-pubs without ASINs

A new cleanup report, "ISBN-less e-pubs without an ASIN", has been deployed. I expect it to find around 3,200 publication records when it runs overnight. Moderators will be able to "ignore" records.

Once the data has been cleaned up, Fixer will be able to submit ISBN-less publications without having to worry about creating duplicates. Ahasuerus 19:09, 25 June 2017 (EDT)

POD numbers

Do we want to add in a field for tracking POD numbers (print on demand numbers, they are under the barcodes on the last page inside POD books)? Since POD is becoming a bigger and bigger thing, it might become a useful tool. Usually, they have the POD number, a print date, and a print location. ···日本穣 · 投稿 · Talk to Nihonjoe 02:17, 28 June 2017 (EDT)

Oh, and the POD numbers sometimes have letters in them, so they would have to be alphanumeric instead of just numeric. ···日本穣 · 投稿 · Talk to Nihonjoe 02:18, 28 June 2017 (EDT)
Is there a Web page explaining how these numbers are assigned, what the standard is, etc? Ahasuerus 15:57, 29 June 2017 (EDT)
Not that I know of. All the POD books I own have the same basic info (what I described above) on the last page inside the book. It is very consistent. I can take pictures of several if you want. Maybe some others here know of something. ···日本穣 · 投稿 · Talk to Nihonjoe 17:25, 29 June 2017 (EDT)
Most of the printers print there an identification for the data file being used. I am not sure if Batch number is not incorporated as well. Annie 19:27, 29 June 2017 (EDT)
Yup. I thought it might be another useful way of tracking them. We might be able to see patters after a certain number have been entered (not sure what that certain number would be). ···日本穣 · 投稿 · Talk to Nihonjoe 20:18, 29 June 2017 (EDT)
We will only see a pattern if we have multiple copies from the same book and know which ones are from the same batch (based on the date maybe). I don't think that it will any useful information to the db but I may be wrong. I certainly would not like to copy a long alphanumerical string anytime I am entering a book. :( Annie 20:35, 29 June 2017 (EDT)
It would only be with POD books, not every time. And we already copy a number of different long strings: LCCN, ISBN, and so on. ···日本穣 · 投稿 · Talk to Nihonjoe 02:09, 30 June 2017 (EDT)
I agree with Annie. While I don't have that many POD books I'm not inclined at all to be hand-entering and double checking "LaVergne, TN USA" over "04 December 2009" over "165914LV00005B/13/P" from Youthanasia. If the information was available via barcode or data matrix then I would consider scanning it. I am entering the POD date in ISFDB's date field in the hopes that others also check the POD date to start gathering data on if there are many publications with the same date of if a few, or even one, publication is printed each time someone orders a copy.
@Nihonjoe, an LCCN is much shorter and can also be verified on the spot by clicking on the resulting link. If it links to the correct record on the LOC site then you know you have the right value. An ISBN has a checksum meaning the editor gets near immediate feedback if there's an error and can also instantly click to verify that it leads to the expected record on many web sites. The POD value is 19 characters in the book I pulled off the shelf that can't be verified other than by manual inspection. It's also has long string with no delimiters, for example with "165914LV00005B/13/P" I had to be extra careful with that long run of zeroes.
Earlier this week I ran across an anthology I want to buy. While checking to see who had copies for sale I noticed there were eight AbeBooks sellers with all of them stating they were selling new copies printed on demand. It made me wonder if all eight had their own POD presses or how that worked. --Marc Kupper 02:51, 30 June 2017 (EDT)
Well, all of the POD books have the POD number with an attached barcode for it on the lower left of the last page, and the date and location of printing on the lower right. All of them are that way. Either way, it seems people don't care for this idea, so we'll leave it at that. ···日本穣 · 投稿 · Talk to Nihonjoe 14:40, 30 June 2017 (EDT)
But you do care. And that is important. So one option may be to use the wiki (under your account or in general on the wiki) - create a POD/PUB page and create a table with publinks and the POD numbers? Maybe something will emerge from that. This way if one day it becomes trackable/important, moving the data will be possible. Annie 14:51, 30 June 2017 (EDT)
How about this? Any suggestions on columns? ···日本穣 · 投稿 · Talk to Nihonjoe 18:36, 30 June 2017 (EDT)
None of my POD books looked like yours - I have some where the code is on the left and the ISBN is on the right, I have some where the code is on the left and the date/place is in he middle and I have some where the whole thing is in the middle lower part of the page. So really starts sounding like each printer having its own ideas? But I am getting my library in order these days so I will see if I can find more of the books and see if I can spot some pattern... Annie 14:51, 30 June 2017 (EDT)
One of the lessons that we have learned over the years is that it's important to have a solid understanding of real life practices and usage before we try to support them in the software. After all, our database is an attempt to model the world of books and needs to follow it closely. On the other hand, if we wait too long before we add software support, we end up with a lot of "stuff" in notes which takes forever to convert once the software has been updated. The balance can be tricky. Ahasuerus 11:01, 1 July 2017 (EDT)
I have a lot of POD books (not just genre - I have a soft spot for small and micro-publishers and most of them had switched to POD in the last years). And it seems to be different - sometimes I will get a book printed months ago, sometimes Amazon will deliver me a book that had been printed on the day I ordered. It is great for remembering when I bought the book but it will be meaningless for a bibliography. It seems like there are printers out there where you can just order a copy and get it in a few days (depending on the ordering agent) as long as they already have the file. Doesn't Lulu also operate in a similar way - they print and mail the book when you order it? Annie 02:59, 30 June 2017 (EDT)

Bad HTML in publication notes

A while back we added a cleanup report to identify and correct notes with empty HREFs. This report has been now expanded to look for HREF values without quotes. Although some versions of HTML allow "naked" HREF values under certain circumstances, they will become invalid in all notes Real Soon Now (tm) as we continue to tighten security.

The report has been renamed and moved to the Notes/Synopsis area of the cleanup menu. The data, a few hundred publication records, will become available tomorrow morning. Ahasuerus 15:41, 29 June 2017 (EDT)

P.S. A new cleanup report, "Mismatched HTML tags in Publication Notes", has been deployed. It looks for HTML tags without a closing tag and vice versa: "b" and "/b", "i" and "/i", etc. Approximately 830 pubs are currently affected. Ahasuerus 23:09, 29 June 2017 (EDT)

I just did 50 of those "Mismatched HTML tags". The vast majority of the problems, 44 of them (88%), were mismatched ul - /ul tags. Surprisingly, the second most frequent error (3 of the remaining 6) were editors who preferred to add the (unnecessary) /li tag with their li tags, but then mistyped the "/li", e.g. as "/l" or as "/i". Chavey 00:21, 1 July 2017 (EDT)
I am also seeing mismatched pairs of "i" or "/i" and a quote on the other side. But yeah - the missing "ul" or one too many "/ul" are the most common.
On the other hand most of the HREFs with no quotes in the other report are for OCLC or LCCN links, a smattering a links to a Portuguese library (same editor I would presume - most of them not verified) and a few links to artist's sites and Amazon.Annie 01:07, 1 July 2017 (EDT)
HTML errors are hard to notice when moderating submissions. Things will get a bit better once we restrict human-entered HTML to a subset of valid tags. We'll also want to add moderator warnings about mismatched tags and other HTML weirdness. Ahasuerus 09:51, 1 July 2017 (EDT)
I was working on the mismatched tags starting with titles that began with "The ..." (so that Annie and I weren't fighting over the same sets of books :-). I made fairly good progress, and all of the remaining books alphabetically from "The Best .. " to "The Golden .." share the same issue: They used tables that included styles or other settings within the td or tr specifiers. (And all in ways that, IMHO, were appropriate.) None of those 6 titles actually has any mismatched html tags. So eventually, you will want to add the ability to recognize a wider variety of td and tr tags, or else add the inevitable "ignore" buttons. I suspect the later. Chavey 12:17, 1 July 2017 (EDT)
I see. The report was looking for things like "<table>"/"</table>" mismatches and found notes with snippets like "<table align=center>".
The bad news is that, as we discussed a few weeks ago, we will discontinue support for all user-entered HTML attributes (except "href") in the near future. They create huge security issues, including potentially having our data destroyed or, worse luck, massively corrupted.
The good news is that the impact of the security changes will be very limited. As of this morning, here is how many notes will be affected:
2 "tr" tags
19 "table" tags
1 "th" tag
19 "td" tags
If a record absolutely has to include HTML attributes, the note can be moved to the Wiki and linked from the database side. Ahasuerus 13:18, 1 July 2017 (EDT)
I was wondering if that was the case. After we correct all the easier cases of these mismatched tags, I'll go back and see if I can find other ways to deal with these tables using "easy" html. Chavey 13:32, 1 July 2017 (EDT)
The non-table ones are all cleared. The remaining 16 are there because of tables :) Annie 20:22, 2 July 2017 (EDT)

(unindent) "Invalid HREFs in Publication Notes" are all taken care of. All hrefs in the system are now decent and clothed - no more naked ones hiding in corners. :) Annie 03:12, 2 July 2017 (EDT)

Thanks! Ahasuerus 08:21, 2 July 2017 (EDT)
Good job! I did a bunch yesterday and today, and was very pleased to see them all finished while I was out dancing :-) Thanks much. Chavey 02:18, 3 July 2017 (EDT)

2017-06-30 downtime at 4pm

The server will be unavailable between 4pm and 4:03pm server time. Ahasuerus 15:50, 30 June 2017 (EDT)

Oops! I meant 4pm. Ahasuerus 15:56, 30 June 2017 (EDT)
Everything should be back up. Approximately 3,300 OCLC IDs have been moved from notes to the External ID field. More to follow. Ahasuerus 16:02, 30 June 2017 (EDT)

New cleanup report: Mismatched OCLC URLs in Publication Notes

While working on moving OCLC links to the External IDs field, I noticed that some links were internally inconsistent, i.e. the displayed OCLC record number and the URL did not match. I ended up creating a new cleanup report which will become available tomorrow morning. Some mismatches will require additional TLC and/or judicial application of the OCLC template. I expect the new report to find fewer than 200 records. Ahasuerus 18:37, 30 June 2017 (EDT)

There are some of those in the other kinds of links as well - a few in the FantLab ones for example while I was clearing them and I've seen a few in ASIN and the LCCN links(and not the 62-21 format vs the 6200021 format which are normal). Annie 18:44, 30 June 2017 (EDT)
Originally I planned to auto-move LCCNs and OCLC IDs at the same time. However, LCCNs have additional quirks, including the hyphens that you mentioned, so they will be handled separately. One monstrosity at a time... Ahasuerus 10:35, 1 July 2017 (EDT)
I constructed a macro that was pretty good at moving the OCLC numbers into external identifiers (but still, one at a time), but it was the hyphens in the LCCN numbers that made me despair of doing that as a macro. Well, that and the "sometimes a hyphen, sometimes not". Chavey 12:12, 1 July 2017 (EDT)
If you are on Windows, AutoHotkey is very useful for semi-automating moving the external ids. You can bind the appropriate key presses (ex. copy selection, select line, delete, tab, select id type, tab, paste) to a single key. Doesn't matter what format the LCCN is, just need to select the 'correct' one. As for LCCN formats, I would prefer we don't try to standardize them, but keep allowing entry of the format shown in the publication. But that is a very minor preference. -- JLaTondre (talk) 08:33, 2 July 2017 (EDT)
I do not think that we are standardizing - I know I am not. What we are talking about is the case where the URL has the hyphen and the string does not and the far more often vice versa - which is a mismatch for automation. In such cases when moving, you need to make sure they are indeed just different formats as opposed to a typo in one of them( in which case you need to find out which is the correct one to leave in place). I go for the hyphen format in such cases or whichever is the correct ifone of them is a typo. But it is up to the moving editor preference when both are presented. Annie 18:21, 2 July 2017 (EDT)

(unindent) Those are done. It was mix of a few situations:

  • Different numbers on purpose from 2009/2010 (if I remember correctly, WorldCat was doing some reindexing back then - I've seen this kind of weird double links before). The one that points to the entry exactly is now left in place
  • Transposed numbers in one of the two numbers (don't people use copy/paste?)
  • Missing or additional number at the start or end of one of the two numbers.
  • Non-number string in the text part or additional parameters in the link part
  • Non-finished editing of a template (12345678 was a very popular one amongst the ones I cleaned
  • Two different numbers that have nothing to do with each other or with any OCLC that belongs to that book.

Now we have a funny python error in the empty report page. Did you forget to check of an array or a result is empty or something? :) Annie 17:12, 6 July 2017 (EDT)

Yup, all fixed now. Thanks for working on these! Ahasuerus 18:42, 6 July 2017 (EDT)

Title missing publications

This title is missing publications. Anyone know where it's pubs went? ···日本穣 · 投稿 · Talk to Nihonjoe 19:49, 30 June 2017 (EDT)

Most likely, it was ejected from the publication it was initially added in (because someone decided they do not need to record this) and as we do not have a report to find the orphan essays, it was not spotted before. It needs a deletion. :) Annie 19:52, 30 June 2017 (EDT)
Deleted. -- JLaTondre (talk) 09:17, 2 July 2017 (EDT)

"Publications with Invalid Prices"

The cleanup report "Publications with Invalid Prices" has been enhanced to include prices without a currency symbol. Ahasuerus 16:26, 1 July 2017 (EDT)

Proposal: modifying capitalization help

See discussion at R&S. --Vasha 11:05, 2 July 2017 (EDT)

OCLC URLs in Notes

As we discussed a few weeks ago, I have been trying to create a script to move OCLC numbers from publications notes to the external ID field. Unfortunately, I have found a number of odd permutations which make it hard to perform an automatic conversion. For example, consider this 1985 publication. The two lines that contain OCLC URLs are followed by additional lines related to the referenced OCLC records. The way my script is currently written, it would move the OCLC numbers to the External ID field and delete them from the Note field, which would create a mess. There are other patterns that make it difficult to automate the process without jeopardizing the integrity of the data.

On the plus side, our indefatigable editors have manually migrated close to 10,000 OCLC numbers to the new field over the last month+. At the rate we are going, the conversion process may be done by early 2018. Ahasuerus 16:53, 2 July 2017 (EDT)

If OCLC, or WorldCat (in all kinds of combinations of small and capital letters) are mentioned anywhere in the note except on the OCLC line itself, then it should not be touched, I agree. But we still have a lot of them where it is only on a line on its own, never referenced outside of the line... I know that text analysis of the full field to check if it is free to lift is time consuming but that will move a lot of OCLC numbers automatically Annie 18:29, 2 July 2017 (EDT)
Well, let's consider the publication that I linked above. I will copy and paste the relevant section of the Note field to make it easier:
How would the parsing logic know not to touch these 2 OCLC numbers even though there are no other references to OCLC or WorldCat next to them? A human would immediately realize that the lines that follow are related, but it's really hard for a computer program to make that type of determination. Ahasuerus 21:40, 2 July 2017 (EDT)
This record have a third OCLC and the word WorldCat under these two - which will mean "nope, human needed". And these 2 messages are faulty anyway - one of them is a bit wrong (they are the same number, different text). I can see the problem though - single OCLC but text on the next line for it.
Here is a new idea - OCLC as a last line on the notes? Annie 22:00, 2 July 2017 (EDT)
That's a good point. There are at least 14,661 notes with an OCLC link as the last line (except for the optional "/ul".) I'll see if I can take advantage of that. Ahasuerus 22:17, 2 July 2017 (EDT)
There are also lots of records where the last 2 lines are: OCLC link; then LCCN link. Those should also be easy to recognize and move. Chavey 02:27, 3 July 2017 (EDT)
It looks like any "pure OCLC link" line followed by a line that starts with a recognized pattern should be doable. Ahasuerus 10:19, 3 July 2017 (EDT)
... and then I find a publication note where the last line consists of an OCLC link and the next-to-last line reads 'OCLC <.....> Table of contents from <carriage return>". Sigh... Ahasuerus 14:15, 3 July 2017 (EDT)

Titles without Pubs report

Can we have this extended to include essays(for now) so we can start clearing those out of the DB? The ones that need to stay for one reason or another will need to get notes to explain so and ignored but there are a lot that need just old fashioned deleting. Thanks! Annie 18:33, 2 July 2017 (EDT)

Done. The data will become available tomorrow morning. Ahasuerus 21:51, 2 July 2017 (EDT)
Cleaned up, it was mostly essays by a few authors that seemed to have had a special dispense (granted by who, I don't know) like Shirley Jackson and Arlan Andrews (and also some quite unknown authors that probably used our system to beef up their bibliography), some leftovers from varianting, some purely electronic texts, some mismatched types (ESSAY for SHORTFICTION), some wrongly split NONFICTION books and some duplicates (under or not slightly different names). We can now try to clean the POEMs. Hauck 03:31, 3 July 2017 (EDT)
Great! POEMs will become available tomorrow morning. Ahasuerus 10:11, 3 July 2017 (EDT)
Done, note a concentration on very few authors (Hardy -see below-, Payack -a special dispense like Hardy?-, Baudelaire -problem with pseudonyming-, Poe -some working or "generic" titles or variants-) and a few leftovers. Ready to go to the next level (SHORTFICTION), perhaps in small batches as I won't try to guess the numbers involved. Hauck 03:58, 5 July 2017 (EDT)
Excellent! Actually, SHORTFICTION isn't too bad, only 1,300 records. I have tweaked the report and they will pop up tomorrow morning. Ahasuerus 13:54, 5 July 2017 (EDT)
A lot of the pub-less stories got nuked when I was assigning languages - because they were not in a publication, they did not get automatic language through any of the usual means so they got in the manual inspection and got deleted there. Thus the much lower number that one would expect (same happened with the poems and essays by the way).Annie 16:13, 5 July 2017 (EDT)

BNF cleanup report tweaked

The cleanup report "Publications with direct BNF links in Notes" has been tweaked to ignore links to Gallica and other non-catalogue parts of the BNF Web site. Ahasuerus 16:13, 3 July 2017 (EDT)

K. S. Hardy pub-less poems question

A huge amount of the poems in K. S. Hardy's account a pub-less (example: this one). What is our policy on that - do we allow them (if they were in printed books or ebooks and not solely online)? I can see that going both ways - if someone bothers to add the non-genre magazines/newspapers they were in, they are eligible (poems are fiction after all and we allow fiction from non-genre sources) but without the magazines, it looks weird. So what do we want to do:

  • Delete the lot of them
  • Add Magazine entries for the magazines/newspapers that contained them and add them as the only content
  • Leave them as they are. (my least preferred but if that is what we want to do, then I am fine with it).

Thanks for any opinions. PS: There are couple more poets like that (and I suspect we will find a few short fiction writers like that as well). Annie 15:40, 4 July 2017 (EDT)

If someone had added it and did not bother to add a note where they appeared, they are getting deleted - if someone cares enough, they need to write notes. But someone did here - thus the question. Annie 15:52, 4 July 2017 (EDT)
IIRC they were added by the author (or its representative, here). I'm going to delete them as our policy (or so it seems to me) was to add non-eligible (either because of their genre or their place of publication) texts at author's level and not as "proper" titles. For example, even if I've written a few essays about SF in non-genre publications, I won't try to include them in the db. I'm also wary of a recent tendency of "aspiring" poets (this is strictly without sarcasm) to massively enter their works in the db (note the length of this page.Hauck 02:39, 5 July 2017 (EDT)
Essays are clear. But if we have a short story in Nature (a non-genre magazine), we catalog these. As we will do any SF story in any non-genre magazine of any type. So I was wondering what we are doing for poems. I have no issue with deleting the lot of them. Annie 02:50, 5 July 2017 (EDT)
Yes, but in the case of Nature there is (in theory) a publication record that goes with the text, it's not the case here IMHO either because the poems are not spec-fic or the publication itself is not eligible. To go beyond this case, the underlying question is "Are POEMs to be treated like SHORTFICTIONs?". This could easily make a few pages at R&S. (For the record, I'd say "no" only on practical grounds as I fear that there are a looooooooot of poems in looooot of "obscure" supports waiting to be entered). Hauck 02:57, 5 July 2017 (EDT)
Another point is that I found the practice of some "self-centered" contributors to enter only their own texts (either as a stub as in this case or as the sole content of the publication like here as I've seen numerous times) instead of taking the time to enter the whole publication quite contrary to our spirit and aims. In Hardy's case, it's quite interesting as some titles were entered as stubs and were (before I deleted them) given as being published in this magazine or that one, BUT the contributor simply didn't take the pain to add the issue. This is just lazyness or self-publicity but it's not acceptable bibliographic work. Hauck 03:02, 5 July 2017 (EDT)
Maybe they did not know that they should because a boatload of those were accepted like that. But then - people will be people, there will always be people with different agenda in a community project... When I end up with one of those incomplete ones, I just edit and add the rest if I can find it. Maybe that will show a new contributor what is the expected behavior. And even if it does not, we do not have a half-way entered publication.
PS: Agree on poems - unless if they are in a genre publication, they are out in my book. Annie 03:31, 5 July 2017 (EDT)

Add AWards From the SFPA

The Science Fiction and Fantasy Poetry association holds an annual contest as well as offering yearly awards for speculative poetry published the previous year: the Rhysling Awards for individual poems, the Dwarf Stars Award for short-short poems, and the Elgin Awards for genre poetry books and chapbooks.

I would like to have the SFPA award, Dwarf Stars and Elgin added to the Awards list —The preceding unsigned comment was added by Akua (talkcontribs) . on 23:05, 4 July 2017 (EDT)

The Dwarf Stars Award and the Elgin Awards look solid and I would have no problem adding them. However, I am not sure what "the SFPA award" means in this context. We already support the Rhysling Award and I am unaware of any other SFPA-administered awarwds. Could you please clarify? Ahasuerus 20:32, 6 July 2017 (EDT)
The SFPA holds an annual contest for sf/f poetry. "The 2017 SFPA speculative poetry contest is open to all poets, including non-SFPA-members. Prizes will be awarded for best poem in 3 categories: Dwarf (poems 1–10 lines [prose poems 0–100 words]); Short (11–49 lines [prose poems 101–499 words]); Long (50 lines and more [prose 500 words and up]). Line count does not include title or stanza breaks. All sub-genres of speculative poetry allowed in any form. Entries will be read blind"Akua 22:10, 6 July 2017 (EDT)Akua
OK, I have added new award types for Dwarf Stars Award and Elgin Award.
As far as the "SFPA Poetry Contest" goes, I am not sure it counts as an award in the regular sense of the word. Its Web page says that it "is intended to raise funds for SFPA, as well as to draw more attention to speculative poets and reward writers of good speculative poems". All poems must be unpublished and and it costs $2 to enter a poem. Ahasuerus 15:21, 10 July 2017 (EDT)

Stephen Crane and the supernatural

On this page, a scholar answers a question about which of Stephen Crane's stories have supernatural elements by listing just three, one of which he considers to have merely "overtones of the supernatural." Do you think this is sufficient evidence to remove Crane's other stories from the database? --Vasha 18:46, 5 July 2017 (EDT)

Ashley and Contento's The Supernatural Index lists 10 stories. I wouldn't recommend deleting any of those. --Ron ~ RtraceTalk 21:08, 5 July 2017 (EDT)
The thing is, Ashley & Contento tend to indiscriminately list the complete contents of a book of "Tales of Terror and the Supernatural" -- because they can't read everything, that's fair enough. I would recommend in that case adding a synopsis and a nongenre mark and a note saying that Ashley & Contento list it even though it's not supernatural. Vasha 21:24, 5 July 2017 (EDT)
If someone reads them and write synopsis for each (or find some) and determines that they are non-genre, a flag and an explanation will be a good idea. I would say that without a reading, it is just one source against another - we need a real human :) Deleting them will cause issues downstream - someone somewhere will decide to add them again. So staying in sync with Ashley & Contento plus notes is probably the best way to go. Annie 21:48, 5 July 2017 (EDT)
Sure, that's fine. Vasha 22:20, 5 July 2017 (EDT)

Stephen Crane wrote a number of parodies of ghost stories in which an apparently supernatural phenomenon turns out to be something ridiculously mundane -- for example "A Ghoul's Accountant" and "The Black Dog." I believe these should be marked non-genre, right? --Vasha 21:28, 6 July 2017 (EDT)

Linking to Smashwords images

Please note that we can now link to Smashwords images. The syntax is similar to what we do for SFE3-hosted images -- see Template:Image Host Sites for details. Ahasuerus 18:42, 7 July 2017 (EDT)

Cool. ···日本穣 · 投稿 · Talk to Nihonjoe 18:50, 7 July 2017 (EDT)

The Whole Science Fiction Data Base 8 question

Hello, I've seen a reference to this: 'The Whole Science Fiction Data Base 8' stating that it provides more detail about a specific book (http://www.isfdb.org/cgi-bin/pl.cgi?257168). Can anyone please tell me what this 'Data Base 8' is and how to access it?

Thanks in advance!—The preceding unsigned comment was added by Voodoomailman (talkcontribs) .

Answered the duplicate post in the Help Desk. --Ron ~ RtraceTalk 22:42, 8 July 2017 (EDT)

LCCN double formats, PV'd publications and the current cleanup project

LCCN have two valid formats: year-number and year(some number of zeroes)number. Both are valid, both can be used for search and linking. In the current LCCN cleanup project, my understanding is that we are moving "as is" - if the editors preferred the 91-38127 format to be visible, this is what we put in the new External ID field - we are not trying to standardize at all (by using the 91038127 format always in the External ID and leaving the 91-38127 in the notes); if they preferred 91038127, that is what we are putting in the field. Especially on PV'd publications. There are a few cases really:

  • Link is pointing to 91038127, the visible part is 91-38127. The new field should be 91-38127 (link ends up in the same place; the visual part is preserved).
  • No link at all, just a string 91-38127, same as the above (actually in this case, I would say that using the 91038127 format is unreasonable.
  • If the visible part is already 91038127, we follow the same principle.

Leaving the link in the notes clutters them - if it is already moved down, why keep the note at all unless if it is tied to more details and needs to stay for clarification? Any opinions on that? Annie 18:58, 10 July 2017 (EDT)

I'm all for removing the note. A difference between LCCNs and the other external identifiers is that LCCNs are usually printed in the books. As an editor/verifier, I'd prefer to have the number formatted as it appears in the book. Assuming editors are likely to have entered them that way, preserving any existing visual formatting appeals to me. --MartyD 21:16, 10 July 2017 (EDT)
Me too (tm) Ahasuerus 21:56, 10 July 2017 (EDT)
There are cases (e.g. The Clocks of Iraz) where a LCCN number is quoted in a book that has no bearing to the present records of the Library of Congress, thus offering no valid link upon entering. I think all links should be checked, and the actual number must be used for linking, as was the case with the old method of entering LCCN numbers in the notes.--Dirk P Broer 21:33, 10 July 2017 (EDT)
We are not talking about the cases where there is a difference between the printed value and the actual one in more than format. Yes, all links need checking - and if it is plainly wrong, the note should explain the situation. But if the LCCN is printed as 62-123, I think this is what needs to be in the External Identifier field - as long as it is the valid one. I strongly disagree that we should enforce 91038127 in the link in such cases - this will make it appear as the value as well as the link now and that is the difference from those old links.
We may have an alternative option - have the external indentifier link be changed when creating the link - doing the conversion on our side (so it emulates the current most prevalent links). But this will be an overkill - LOC already does that. Annie 21:41, 10 July 2017 (EDT)

Similar series titles

This series has a notice at the top that another series has the same name. However, their names are not the same. In fact, the second series is a subseries of the first. Is there a way to suppress these notices on specific series? ···日本穣 · 投稿 · Talk to Nihonjoe 13:39, 12 July 2017 (EDT)

I submitted a series name change for the second one that may fix this. ···日本穣 · 投稿 · Talk to Nihonjoe 13:43, 12 July 2017 (EDT)
Yep - instead of brackets (), use another separator in the series name. They have the notice as the same because the brackets syntax is considered disambiguation. The change helped (different names outside of any brackets). All good nowAnnie 13:46, 12 July 2017 (EDT)
Cool. Any way to fix the sorting in the main series? It currently has them listed 2, 1, 3, 4 instead of 1, 2, 3, 4. ···日本穣 · 投稿 · Talk to Nihonjoe 16:40, 12 July 2017 (EDT)
The problem is that there are two different elements here - the books directly into the series (2) and the subseries entries (the rest). They sort separately and I am not aware of any way to sort them together - it sorts first the books into the series, then the subseries. So... one option is to put the single one that is in position 2 now into its own "main" series and put that main series as #2. Or you can wait to see if someone else has a better idea. Annie 16:49, 12 July 2017 (EDT)
Guess we should wait for Ahasuerus to wander by. :) ···日本穣 · 投稿 · Talk to Nihonjoe 18:08, 12 July 2017 (EDT)
Hmm...found an art book for the series, so I submitted that and changed the series to レア・ガルフォース to match the art book. That should fix it once approved and the new series is placed as #2 in the main series. ···日本穣 · 投稿 · Talk to Nihonjoe 18:17, 12 July 2017 (EDT)
Approved everything you had in the queue - and added the numbering (I think). let me know if this is what we were trying to achieve :) :) Annie 18:24, 12 July 2017 (EDT)
Looks good. Thanks. ···日本穣 · 投稿 · Talk to Nihonjoe 18:28, 12 July 2017 (EDT)

Here's a fun one to sort

Somehow, things got mixed up with this volume:

Volume one should have the following contents:

Instead, the first entry is for this omnibus (which isn't actually a real book). I'm not sure how it happened, and I'm not sure how to fix it since there's an omnibus within an omnibus. ···日本穣 · 投稿 · Talk to Nihonjoe 18:42, 12 July 2017 (EDT)

Use 'Remove Titles From This Pub' and remove the 'bad' omnibus . Once approved, you can delete the omnibus record (since it doesn't seem to be a valid publication) and import the missing novel's title record into the 'good' omnibus. -- JLaTondre (talk) 18:50, 12 July 2017 (EDT)
Submitted ···日本穣 · 投稿 · Talk to Nihonjoe 19:05, 12 July 2017 (EDT)
That's the wrong one -- you want to kick out the 銀河無責任時代 omnibus, not the one that is the same name as the publication :) Annie 19:07, 12 July 2017 (EDT)
Fixed. ···日本穣 · 投稿 · Talk to Nihonjoe 19:11, 12 July 2017 (EDT)
here is your stray title. Do you want to delete it? Seems like that series is missing its #1 now? Annie 19:13, 12 July 2017 (EDT)
Submitted the deletion and the addition of the stray title (I knew about it, since that's how I noticed something weird was up). ···日本穣 · 投稿 · Talk to Nihonjoe 19:14, 12 July 2017 (EDT)
Yeah, you were moving faster than I was typing above :) All looks good now I think? Annie 19:15, 12 July 2017 (EDT)
Almost. I submitted a sort order fix, too, for both in the series. ···日本穣 · 投稿 · Talk to Nihonjoe 19:16, 12 July 2017 (EDT)
Got them less than a minute after you sent them - probably while you were typing. :) Now all should look fine. Annie 19:25, 12 July 2017 (EDT)
Thanks for helping to sort it out. :) ···日本穣 · 投稿 · Talk to Nihonjoe 19:52, 12 July 2017 (EDT)
I remember the anime, but I had no idea that the (very) irresponsible Captain Tylor was so popular! Ahasuerus 20:25, 12 July 2017 (EDT)
And there's more, too. I still have 4 more novel series to add to the pile. ···日本穣 · 投稿 · Talk to Nihonjoe 20:47, 12 July 2017 (EDT)
Also, I wasn't aware there were this many. I knew the anime was based on some novels, but I'd never looked into it before. ···日本穣 · 投稿 · Talk to Nihonjoe 14:52, 13 July 2017 (EDT)
You never know what you may find once you start digging. Back when I started working on the database, I had no idea that so many Kenneth Bulmer books were only available in German (at the time.) Ahasuerus 16:00, 13 July 2017 (EDT)

Jonathan Strange & Mr Norrell

Based on the following links ( WorldCat, LOC, Locus) and other references, the correct canonical title for this title should be "Jonathan Strange & Mr Norrell". The author's intent based on interviews is to use the punctuation of the time, and so the title should not include a period after 'Mr'. I can correct the publication entries for which I am PV, and suggest modifications to other PVers, but I think we should correct the Title Name as well. What's the recommended procedure for this?


WorldCat LOC Locus

—The preceding unsigned comment was added by Taweiss (talkcontribs) . 19:46, 12 July 2017 (EDT)

Changed. Stonecreek 23:42, 12 July 2017 (EDT)

Nominating user Nihonjoe for moderator

See Moderator Qualifications#Becoming a moderator for the nomination process.

I would like to nominate user Nihonjoe (talkcontribs) for moderator. He has been with us for a number of years now, mainly working on Japanese entries and the Wiki side of the database. His talkpage shows excelent communication skills, and hardly any problems with his submissions. I think he meets the Moderator Qualifications and he has accepted the nomination. His language skills are an enormous asset.

Support

  1. Support, as nominator. --Willem 02:10, 17 July 2017 (EDT)
  2. Support, sorry to be late to the party, and discussion below notwithstanding. I don't share those concerns. I've found him to be a good communicator and his submissions and understanding of ISFDB policies and procedures to be solid. I also think beefing up the moderator corps' Asian language capabilities would be an added benefit. --MartyD 21:11, 20 July 2017 (EDT)
  3. Support, Nihonjoe is careful and all of his submissions that I have moderated have been uniformly good.Rkihara 23:06, 21 July 2017 (EDT)
  4. Support, with a small note - I had never seen him submitting an English language item (except when it was needed for a variant for a Japanese one or something like that.) Which does not disqualify him (and I had not been around long enough to know the complete history) but the conventions around capitalization, names normalizations, same-named publishers and so on can be a bit tricky (a lot trickier than they are in any of the other languages) and need a lot of attention when moderating. But the easiest way to learn those is either to edit a lot in that language or start helping people do them properly - and the first one is unrealistic for someone whose interests are in other languages. So when/if he becomes a moderator, I hope he will be careful around English-language submissions. :) Annie 13:06, 22 July 2017 (EDT)
  5. Support, I'm about 65/35 split [positive/neutral] and only because I have not moderated enough submissions for Nihonjoe. I think most were Variants, so the original submissions were already done [while I'm comfortable with french, German, Spanish or Italian I have no experience with cyrillic or pictographic languages]. But that's my lack, not his. I went through past pages of commentary and I will say there is a definite involvement at all levels. That's important. I don't feel the 'lack' of submissions is crucial [I certainly had far fewer when I became a Moderator, but the database was much simpler then - I actually knew what I didn't know]. What one submits also doesn't need to cover everything [I've never done one involving an Award - so maybe both my and Willem's flags should be lowered to half-mast]. I agree with Marty that more language coverage is a plus at the Mod level. And I haven't seen where Nihonjoe has been the slightest bit reluctant to ask if there's something he's unsure of. That's how I learned. --~ Bill, Bluesman 15:29, 22 July 2017 (EDT)
  6. Support, despite a moderate amount of submissions, what I moderated was usually the result of serious, careful work. It being often Asian-centered is no problem, on the contrary : competence in this field seems to be an asset for this db. Linguist 08:16, 23 July 2017 (EDT).

Oppose

  1. Oppose Not for now, as Nihonjoe has (IMHO of course) not enough contributions (7.000). In the present state of the db, 10.000 contributions is a bare minimum (15.000 being the correct number to have learned most of the ropes), the last two accepted moderators were above the 20.000 mark. There is also, at this time, no shortage of moderators. Hauck 02:18, 17 July 2017 (EDT)

Comments/ Neutral

  1. Neutral. Since I was asked to comment, I will post this versus not responding. I would prefer that someone becoming a moderator be familiar enough with the tools to handle this situation (discussion two topics above). However, I don't oppose the nomination. Nihonjoe's work has been good & I highly doubt he will cause any damage. If others are fine with him picking the rest up 'on the job', I'm comfortable that he will remain conscientious and so don't have an objection. -- JLaTondre (talk) 21:03, 23 July 2017 (EDT)


  • Regarding Hauck's comment, there are a couple of moderators who have fewer contributions than me per the list: Albinoflea (~2800), Tpi (~4000). Another has just a few more than me: Kpulliam (~8500). So these seemingly-arbitrary "bare minimum" or "correct number" of contributions seems to be just that: arbitrary. As for the other argument (no shortage), there is no mention anywhere that I can find of a cap on the number of them (or anything similar). ···日本穣 · 投稿 · Talk to Nihonjoe 12:06, 17 July 2017 (EDT)
What don't you understand in the acronym IMHO? Of course it's arbitrary, it's my opinion. As for Albinoflea, I've exactly had the same position (not enough contributions), as for tpi he's with us since 2008 and was made moderator more than seven years ago in different times. On the relationship level, this immediate outburst just confirms my opinion that you're not ready for the job. Hauck 12:50, 17 July 2017 (EDT)
I apologize if you feel my comments were an "outburst", but I wasn't aware I wasn't allowed to make comments. If that's the case, I'll certainly refrain. I understand the IMHO acronym, too, as I've been online since the glory days of IRC way back in 1990. I'm fine if you don't think I'm ready for the job. I was merely pointing out that the numbers you gave seemed arbitrary (to me, even though I didn't use "IMHO" when I made the comment). You are certainly welcome to have your own opinion about anything, and I wasn't trying to imply that you couldn't. Apologies if it sounded that way. ···日本穣 · 投稿 · Talk to Nihonjoe 14:46, 17 July 2017 (EDT)
A little bit of background. Back when ISFDB 2.0 was open for alpha testing in 2006, all testers were automatically made moderators because we were still testing the software. Once the beta phase started in December 2006, many original testers slowly became inactive. Eventually their moderator flags were removed. Some even asked to have them removed because they felt they were no longer on top of things.
Over the following few years a more formal process -- formal requirements, nominations, voting, etc -- was developed. We also added a "demoderatorization" policy for inactive moderators.
As far as the number of contributions goes, I think it's a useful gauge, although necessarily a rough one. As the size and complexity of the database grows, moderators need to be aware of more and more areas which didn't exist in 2006: awards, publication series, external IDs, cloning, cleanup reports, links to third party web pages, various title flags, languages, chapboooks, user preferences, etc. It's possible for an active editor to be very familiar with one part of the database editing process and be a rookie in other areas.
When reviewing the edits made by a potential moderator, I find it useful to look not only at the total number of edits, but also at the breakdown by submission type, which can be found here. In this case, I see that Nihonjoe has created 432 New Award submissions, which suggests that he is probably familiar with awards. Similarly, he has created 318 Title Merge submissions and 765 Make Variant Title submissions. On the other hand, he has created fewer than 10 Title Unmerge submissions, a tricky area.
One thing that we have found useful in the past is identifying the areas which a prospective moderator hasn't been exposed to and having a more experienced moderator work with him to make sure that he or she becomes familiar with them before we turn the moderator flag on. Perhaps it's something that we could consider in this case. Ahasuerus 13:54, 17 July 2017 (EDT)
I'm sure there are far more areas where I haven't done much, too. I don't think a moderator has to be intimately familiar with every area before becoming one, but rather willing to learn and willing to ask questions if they don't know what should be done in a given area. ···日本穣 · 投稿 · Talk to Nihonjoe 14:46, 17 July 2017 (EDT)
True, but there are times when, as the saying goes, "we don't know what we don't know". There have been times when I thought that I knew what the policy was and made edits based on my understanding only to find out later that my understanding was incorrect or out of date. On the other hand, as the system grows, it becomes harder to keep track of all of our features and policies; if we required perfection, we would be left with no moderators. Kind of a tricky balance... Ahasuerus 14:57, 17 July 2017 (EDT)
However, if that's the general consensus, I'm happy to live with it. I wasn't looking to be made one, but Willem said he thought I could do the job, so I indicated that I was fine if he wanted to nominate me. Whatever the decision here, I plan to continue contributing as I can (which is all I've ever done). ···日本穣 · 投稿 · Talk to Nihonjoe 14:46, 17 July 2017 (EDT)
I find that this too is a matter of balance. On the one hand, we obviously do not want new editors to be able to approve their own edits -- even the most well-meaning newbie is liable to make mistakes early on. On the other hand, having moderators approve every trivial change made by a veteran editor is not a very good way to spend out limited resources either: it makes both the veteran editor and the approving moderators less productive. Making qualified editors moderators is in everyone's interests, we just need to make sure that they have been properly "seasoned" :-) Ahasuerus 15:09, 17 July 2017 (EDT)
If people want to work with me in specific areas in which they think I need more experience, I'm open to it. I tend to just work on whatever interests me (though I do have a very large list of projects which seems to never get smaller). So, feel free to pass the salt and pepper and make sure I'm seasoned. ;) ···日本穣 · 投稿 · Talk to Nihonjoe 15:13, 17 July 2017 (EDT)
I think this discussion is going nowhere. Sorry to put you through this Nihonjoe. Perhaps my moderator flag should be removed because I don't do anything in the awards section? --Willem 15:22, 17 July 2017 (EDT)
No worries. I've been through far worse in similar discussions. This one barely registers. I'm fine it continuing if people want to continue it. Either way. ···日本穣 · 投稿 · Talk to Nihonjoe 16:03, 17 July 2017 (EDT)

Request for additional comments

It's been 5 days since the nomination. Normally (and as per the standard process), it would be enough to determine if we have consensus. However, only 4 votes have been cast so far: 3 in favor and 1 against. I worked with Nihonjoe last year, when he was learning the ropes, but I have processed only 14 of his submissions in 2017, so I hesitate to cast a vote.

I have compiled a list of moderators who have approved more than 50 of Nihonjoe's submissions in 2017 and who haven't voted. I will ask them to chime in based on their experience. Unfortunately, one of them is on vacation. Ahasuerus 11:02, 22 July 2017 (EDT)

Outcome

The final tally is 6 in favor, 1 opposed and 1 neutral, so I will go ahead and set the moderator flag on Nihonjoe's account. Based on the comments about his limited exposure to non-Asian publications, I will encourage Nihonjoe to concentrate on self-approvals while he gets more experience in other areas. Ahasuerus 16:07, 24 July 2017 (EDT)

I'll keep Hauck's comments in mind. As for non-Asian contributions, I've submitted quite a few. More recently, I've been focusing on a few long Japanese series (one of the series had around 160 volumes in it!), so most of my recent contributions have been related to those. Thanks to everyone who participated. ···日本穣 · 投稿 · Talk to Nihonjoe 19:56, 24 July 2017 (EDT)
Congratulations from me also... don't be afraid to ask questions, and the beers are in the fridge. :) PeteYoung 06:24, 25 July 2017 (EDT)
As long as some of them are root beers since I don't drink. :) ···日本穣 · 投稿 · Talk to Nihonjoe 10:48, 25 July 2017 (EDT)

Content field for omnibus variants

The way we variant ombinuses is that we do not variant if the content is not exactly the same. At the same time, if you want to see the content of a variant, you need to have it added on the variant itself. Is there a reason why this field is not treated as the Series and SeriesNum fields and to be greyed out on the variant and inherited from the parent? Annie 18:10, 19 July 2017 (EDT)

Let me think about it... How about this scenario: the parent is [/1,2] while the variant is [/1,2+ss]? Is it a legitimate parent-variant pair or should their VT link be broken? Ahasuerus 20:56, 19 July 2017 (EDT)
Well, I do not like to variant different content - if it is a variant, it should be the same content (abridged, translated and whatsnot make a variant, split volumes (these I still do not like but oh well); different fiction elements altogether make new title). Otherwise where do we draw the line? At 1 story, 2 stories? 1 novel? If we take that wiggling to the extreme, we should just variant all omnibuses that contain the same 2 novels - after all the fact that one of them have 1 more novel and the other 2 more novellas and the third 4 more novellas is ignored if we say that 1-2 and 1-2+ss is the same. Or if we decide we will accept that, we need to draw the line somewhere firmly (and to keep in mind that if 1-2+ss is varianted under 1-2 but that will also mean that 1-2+2ss needs to go under 1-2+ss which will mean under 1-2 as well.
Of course if the community disagrees, I will just shut up and go do the variants the way we agree we should (and I had never seen your hypothetical situation so never thought on that - I was just cleaning series from variants and saw it on the omnibus and decided to ask. Annie 21:11, 19 July 2017 (EDT)
I only variant those with the exact same content. Otherwise, it's a different publication. ···日本穣 · 投稿 · Talk to Nihonjoe 00:08, 20 July 2017 (EDT)
I do this on a case by case basis (that is a really precise definition!). When the "bulk" is kept, especially in translation, I make the variant, for example in Ahasuerus' scenario for a translated work, I'd make the variant. As for the content of the field proper, I rarely use it as some title series are quite fluid and what is true at one moment may not be so in the future. Hauck 02:09, 20 July 2017 (EDT)

Award year listings not working for Gemmell Award

Examples: 2016, 2015, 2010. If you view them by category, it works fine: Legend, Morningstar, Ravenheart. ···日本穣 · 投稿 · Talk to Nihonjoe 15:54, 20 July 2017 (EDT)

It is the dates - someone had added them as 2010-06-00 (for example - the one I looked at) and not as 2010-00-00. I fixed one so now one shows under 2010. I will wait for Ahasuerus to see this and confirm before I fix the rest. Annie 16:11, 20 July 2017 (EDT)
Just looked at the help page: "Enter the award year using the YYYY or the YYYY-00-00 format" :) So at least the help page is correct. Maybe we need a check added to make sure that the page does not allow the month and date to be added. Annie 16:14, 20 July 2017 (EDT)
Probably easiest would be to have it be a YYYY-only field and add the "-00-00" on the back end if needed to make the database happy. ···日本穣 · 投稿 · Talk to Nihonjoe 16:45, 20 July 2017 (EDT)
It will do it if you just add 2017 :) The field accepts two formats: YYYY and YYYY-00-00 and stays valid. Because it is called year, I guess the assumption is that people will just add the year so no additional checks were done. I think the validation is the same as a date field but the help page for them is different. Annie 16:48, 20 July 2017 (EDT)
PS: That is why I said that I want to wait for Ahasuerus to chime in :) Annie 16:50, 20 July 2017 (EDT)
Yup. Another idea: just have the database strip out "-MM-DD" for the awards and replace them with "-00-00" when submitted. Might be quicker than changing the form and such. ···日本穣 · 投稿 · Talk to Nihonjoe 18:20, 20 July 2017 (EDT)
Thanks for identifying the problem! I am kind of under the weather at the moment, but I will take a closer look as soon as I get better. Ahasuerus 20:33, 20 July 2017 (EDT)
Get better! This can wait. Annie 20:41, 20 July 2017 (EDT)
Yup, definitely hope you feel better soon. ···日本穣 · 投稿 · Talk to Nihonjoe 01:42, 21 July 2017 (EDT)
Thanks, folks, I am feeling better now. Not at 100% yet, but "mostly functional" (or at least that's what Fixer tells me.)
I have changed the affected Web page to use the award year only, so values like 2010-06-00 should no longer be a problem. While I was at it, I fixed a couple of other bugs. They only affected processing improperly formed award URLs, so fixing them should have no impact on most users. Please let me know if you run into any issues. Ahasuerus 15:44, 21 July 2017 (EDT)
So what is the policy now - dates or year only? If we officially allow dates, we need the help page changed. I know that it will work for dates now but that does not make it correct according to the current rules - thus me asking. Annie 15:50, 21 July 2017 (EDT)
I guess the first question that we need to answer is whether we plan to support awards that are not given annually. For example, Writers and Illustrators of the Future gives its awards quarterly. It's really a contest for amateur writers rather than an award:
but there may be other awards that are given more than once a year. If we were to decide to add them, we would need to redesign our software because it currently assumes that all awards are annual. (Gaps are allowed, of course.)
For now, I think we should keep the current policy, i.e. "YYYY or YYYY-00-00", for award dates. Once we clean up the data (there are 90 award records with months), I can add another layer of validation to prevent editors from entering YYYY-MM-00 and YYYY-MM-DD values. Ahasuerus 16:10, 21 July 2017 (EDT)
I agree. :) I've corrected any that I was approving last few days. Can you get a list of the affected ones so we can fix them? Annie 16:25, 21 July 2017 (EDT)
I have coded and deployed a cleanup report to find them. The data will become available tomorrow morning. I'll go ahead and create an FR to add that extra layer of validation that we discussed earlier. Ahasuerus 17:53, 21 July 2017 (EDT)
There is another example of 2 sets of awards in one year with the 2010 Grand Prix de l'Imaginaire. My assumption is that because their eligibility year was drifting, they awarded a second set of awards in 2010 to harmonize the eligibility year with the calendar year. I originally tried adding those with different months, but quickly discovered the problem seen with the Gemmells. I ended up adding both sets under 2010 with notes on the award record explaining which set it belonged to. I also added an explanation on the award type record. It might be a nice idea to have a note field for the award year where these sorts of things could be explained closer to where they appear. We could also note eligibility periods and time and place of the award ceremony. Just an idea. --Ron ~ RtraceTalk 18:52, 21 July 2017 (EDT)
An interesting example; thanks. I agree that there would be value to adding an "award year note". Unfortunately, our current database layout doesn't lend itself to this addition. Notes are associated with records. At this time the only awards-related records are "awards", "award types" and "award categories". We would need to create a new record type -- something like "award year" or "award ceremony" -- in order to add notes.
After sleeping on it, I think it may be a viable idea. We already need to create an award type and an award category before we enter an award. If we were to add a new record type for "award years/ceremonies", it would be similar to award categories and would be available as a drop-down list on New/Edit Award pages. You would have to define an "award year" before you could enter any awards for it and there would be a note explaining where and when the awards were announced and/or presented. This functionality would also allow us to support non-annual awards like "2015 3rd quarter". Food for thought... Ahasuerus 09:28, 22 July 2017 (EDT)

(unindent) I have created FR 1086 to document what has been discussed here. It's not a high priority; I expect that we will discuss it when it's time to talk about Roadmap 2018 . Ahasuerus 12:45, 24 July 2017 (EDT)

Sounds good. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 13:29, 24 July 2017 (EDT)

"Note to Moderator" field name change

Hello, I find Annie's idea to use the "Note to Moderator" field to "notify" PVs really great. I'm starting to use it intensively as does Annie (of course) and Bill. This may be the long-searched-for way to simplify our notification procedures as this field is accessible anytime in the "My Changed Primary Verifications" link. Of course this way of doing things should be just used for "simple" updates or additions (The process for more problematic cases should stay as it is via the Talk Pages). To make things even clearer, perhaps should the field be renamed along the lines of "Note to Moderator / Notification to PVs" (or something else). Any thoughts on the matter? Hauck 02:56, 21 July 2017 (EDT)

One of the issues is that if there is more than one update in a day or since the last time you opened your Changed verifications, the previous notes get lost - each pub is on the list just once (I believe so anyway - maybe adding all updates to the list there is the way to go?). They are in the submission itself but not in the quick link under "Changed verifications" so finding them is not that easy). When I know I am multi-editing (typos usually), I make sure I repeat both but if I do not know about the change (or it was long enough ago to forget), the lost ones are getting lost. And yes - anything that is not moving fields around, format issues and so on should still be following the old process.
Nope, multiple edits show up on the list, and each change is different. I can't say if Mod notes stay as the ones I just checked on my list didn't have any. --~ Bill, Bluesman 15:42, 22 July 2017 (EDT)
Another concern is that these are supposed to be just for the moderators (so people may add personal information there) - even though they were never really hidden. A rename of the field just for the publications (how often do we get personal information on changed publications? Authors edits - maybe, but on publications, not that much. Plus if we do see one, a rejection WILL keep it safe (I am pretty sure that only moderators and the editor that submitted the original have access to rejected submissions or at least it is not that easy to find them) and the moderator can reapply the change and add a proper note) Annie 03:26, 21 July 2017 (EDT)
So split in two fields? Hauck 03:32, 21 July 2017 (EDT)
Also an option :) But we have editors that put notes to the moderators into the Notes or Synopsis fields (and notes for the edition into the moderators notes) :) So technically, no matter what is done, we will have things that are just in the wrong place anyway. And I am really not that concerned about these on publications - most updates on verified publications won't have personal data. It's just that we tell people this field is safe - so we need changes in that. As long as people know that the field is visible, we should be fine with one field I think.
But I do agree, it is very useful for cleanup operations (and cuts down on the "what did she change now?" questions I hope) which is where I use it mostly (bigger changes get the note AND a post on the Talk page). As long as the multiple edits issue can be resolved, that may be a very effective tool indeed (not that there will be a lot of multiple edits in the same publications very often but corner cases tend to snarl designs and it is a valid case). :) Annie 03:47, 21 July 2017 (EDT)
Extremely useful, for all minor changes. I've actually re-submitted sometimes because I forgot to add a note [by rejecting the non-note version only one shows up on the other PVs page[s] ]. If we make it more visible/renamed/whatever more people will use it. --~ Bill, Bluesman 15:42, 22 July 2017 (EDT)
I seem to have missed the original suggestion / discussion. Searching on "Note to Moderator" turned up nothing and I'm not looking through everything Annie has submitted. Any chance of a link? Thanks. Doug H 11:46, 22 July 2017 (EDT)
I think that the seed of the idea is here. Hauck 12:43, 22 July 2017 (EDT)
There was never a discussion per se. I used to use the moderator notes a lot before I was a moderator to remind myself what I did or what else needs to still be done or to show the moderators what I am changing for a small change (it is very annoying to try to figure out the change in a 6 lines of notes when visually you cannot spot it:) ). Then I realized that I can see the moderator note when I click on the changed verification or in an item in Recent Edits (before I was a moderator) so figured that it won't hurt to add there a summary of what I am changing during my cleanup -- too small to notify (moving lccn or adding a missing tag or changing to a template or adding a transliteration or changing a capital/small letter in a story - that kind of thing). So that is the backstory - a side unintended effect of a feature basically that is very useful on its own. Annie 12:53, 22 July 2017 (EDT)

(unindent) It looks like there are three overlapping issues here:

  • Add a new field to allow entering "Change Summaries". We may want to add this field to all types of submissions, not just EditPubs, since the functionality can be useful in a variety of cases.
  • "Create a history of changes to primary-verified publications by storing a snapshot of the way each verified pub looked like right before it was changed" as per Roadmap 2017. Unlike the proposed "Change Summary" field, the data will be saved automatically at the time the submission is approved. Even if the submitter doesn't enter anything in the "Change Summary" field, the data will be captured. The display software will be able to show two versions of the record side by side with differences highlighted.
  • Ensure the privacy of the "Note to Moderator" field since it can contain private information like e-mail addresses.

If this functionality is deemed higher priority than other outstanding items listed on the Roadmap 2017 page, I can reshuffle the development queue accordingly. Ahasuerus 16:28, 23 July 2017 (EDT)

I do not think that the first two are opposites - if you just change a letter somewhere in the notes (or add a tag), finding it even when you have both versions takes a lot of time and effort. So having a change summary in that case is very useful. Annie 14:13, 24 July 2017 (EDT)
I agree that they are complementary. Changing the software to highlight the differences will help, but a Change Summary field can do a lot more to explain the reasons for the change. Ahasuerus 16:23, 24 July 2017 (EDT)
Something else to think about: Changes in contained titles. We have the publications changes but if something is changed on the titles level (being it the title itself or content), there is no notification. And these are the ones that can really mess things up -- changing a story to a novella when you know it is not true or merging the translations of two stories when they need to stay separate. Any plans to start adding those to the changed primary verifications as well (and/or make them more available)? Annie 16:39, 24 July 2017 (EDT)
As I recall, I briefly considered the issue of EditTitle submissions affecting verified pubs when I was implementing the Changed Primary Verifications report. I agree that it would be desirable to add EditTitle submissions to this report. Title merges may be trickier; I'll have to think about them. Ahasuerus 17:55, 24 July 2017 (EDT)
I think that just highlighting that there had been a merge will be enough if someone wants to look into it - with a surviving note (because the current Recent Edit entry does not show the moderator notes for Merges and Deletes I think. Merges are mostly important in two cases: pseudonyms and translations. In most other cases, they are usually because someone added a book and typed the content instead of importing. On the other hand, this will make the changes page very busy so having two separate levels (show pubs only and show all titles) may be desirable. Just thinking aloud here - may be overthinking some of that. Annie 18:36, 24 July 2017 (EDT)
However - in both cases, we need to think about multiple changes - either by adding ALL changes at the Changed Primary Verifications or by making a list of "here are all changes, click away". Otherwise we are running the risk of me changing substantially the format (fixing links for example) and then someone updating a small to a big letter - leaving both my summary and changed record away from the eyes of the PV. Annie 14:13, 24 July 2017 (EDT)
As Bluesman pointed out earlier, the Changed Primary Verifications page already displays all changes, including multiple changes per day. If you have come across cases where a change to a verified publication didn't appear on the primary verifier's Changed Primary Verifications page, it was a bug and I'd love to hear the gory details! Ahasuerus 16:27, 24 July 2017 (EDT)
I need to remember what the issue was. Maybe if one of the "changers" was the PV? Will post back when I remember what that was all about (and because of that my mind somehow decided that we never have the duplicates. Looking at my own list, we obviously do so apparently I am a bit confused. Sorry! Annie 16:39, 24 July 2017 (EDT)
I think that in the meantime, using the moderator note field is a useful, albeit imperfect workaround. Annie 14:13, 24 July 2017 (EDT)

PulpCovers.com covers

Apparently, 4 years or so ago, the site owner gave permission to use their covers over here. What is the correct process here:

  • Ask the user to send an email from the domain (to verify that it is them?)
  • Just accept it as stated
  • Something else?

And if that is enough, can we have it added? I am holding a submission from them with another link and I explained the situation to them.

PS: And it is a good site to have permission to link to. Annie 12:30, 21 July 2017 (EDT)

I think the permission posted on our Wiki should be sufficient in this case. I have updated the software to credit the site. Approve away! Ahasuerus 13:14, 21 July 2017 (EDT)
Done! Can you add it to the help page as well? Thanks! (over here :) Annie 13:44, 21 July 2017 (EDT)
Added and alphabetized. Ahasuerus 14:35, 21 July 2017 (EDT)

demon.co.uk

Our UK-based spies are reporting that "UK Demon Internet" (demon.co.uk) has undergone a number of changes over the last few months. As a result, many non-business users, including some SF people, have moved their Web pages elsewhere. We have a couple dozen author records which link to Demonic web pages, so it would be good to double check that our data is still valid. Calling for volunteers! Ahasuerus 13:24, 21 July 2017 (EDT)

I will see what I can do today. Annie 14:59, 21 July 2017 (EDT)
The only ones that need attention are the 11 that used to be on fiawol.demon.co.uk (this is the list. The rest are either verified (they are still where we think they are) or updated now (for the ones that had moved). All of these are invalid - but I want to spend some time to try to find where the sites went (if anywhere) and a quick look did not find them. Annie 15:40, 21 July 2017 (EDT)

Template:PublicationFields:ExternalIDs

I've tweaked the layout of Template:PublicationFields:ExternalIDs in order to make it more readable and helpful. I added some descriptions so people will have a better idea about each one. If those familiar with the different IDs can add information on how to find it when viewing a record on each site, that will make the page even more helpful. Also, if there is a Bulgarian name for the SFBG, that should be added to the entry in the table. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 15:09, 22 July 2017 (EDT)

No Bulgarian name really - it is an abbreviation of Science Fiction - Bulgaria :) Known as SFBG - both on the site and in any conversation or site that cares what a site calls itself. Added some notes. I like the new format :) Annie 15:36, 22 July 2017 (EDT)
Glad you like it. ···日本穣 · 投稿 · Talk to Nihonjoe 18:45, 23 July 2017 (EDT)
Personal tools