ISFDB:Community Portal

From ISFDB

Jump to: navigation, search


ISFDB Noticeboards
Before posting to this page, consider whether one of the other noticeboards might suit your needs better.
Help desk
Questions about doing a specific task, or how to correct information when the solution is not immediately obvious.
• New post • Archives
Verification requests
Help with bibliographic, image credit, and other questions which require a physical check of the work in question.
• New post • Archives
Rules and standards
Discussions about the rules and standards, as well as questions about interpretation and application of those rules.
• New post • Rules changelog • Archives
Community Portal
General discussion about anything not covered by the more specialized noticeboards to the left.
• New post • Archives
Moderator noticeboard
Get the attention of moderators regarding submission questions.
 
• New post • Archives • Cancel submission



Archive Quick Links
Archives of old discussions from the Community Portal.


1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · 23 · 24 · 25 · 26 · 27 · 28 · 29 · 30 · 31 · 32 · 33 · 34 · 35 · 36 · 37 · 38 · 39 · 40 · 41 · 42



Contents

Roadmap 2017

As of late, I have been tweaking the software based on user feedback and in support of various cleanup projects. It was a good match while I was recovering from the most recent unpleasantness over the last couple of months.

Going forward I hope to get back into more heavy-duty development work. With that in mind I have compiled a list of software projects that are currently close to the top of my list of priorities for 2017. I'd like to post them and solicit feedback re: their desirability and relative priority. If a description seems too vague or incomplete, please don't hesitate to ask for a clarification. Detailed suggestions/feedback re: individual FRs may need to be spun off as separate Community Portal sections. Additional requests welcome!

Continuation and further development of currently active projects

  • Language cleanup:
    • Assign languages to all titles
    • Assign languages to all authors. Should we use the manual approach that we used with titles? Or can we automate some parts of the process based on the language of each author’s titles?
      I think that for the ones remaining (with 3 titles or less), we can just assign automatic English if all the titles they have are English (after all titles have languages). We will miss a few that are not English but we will also miss a lot of them if we go manually - they will require a lot of research at this point. If a non-English title ever appears, we can always change it. That won't be much different to what will happen now if someone adds a magazine in English and do not change the author language manually after the addition... Annie 20:38, 19 February 2017 (UTC)
      Or if someone is too worried about doing it, going the manual way will always work of course. Annie 05:38, 21 February 2017 (UTC)
      I guess the "low-hanging fruit" would be identifying language-challenged pseudonyms of language-enabled authors and auto-assigning the parent author's language to the pseudonym. Ahasuerus 16:07, 21 February 2017 (UTC)
    • Yes, I think we even talked somewhere about that. If those can be done, this will eliminate a pretty good chunk of what is remaining... Annie 16:25, 21 February 2017 (UTC)
  • Wiki cleanup
    • Move series/magazine-, publisher- and publication-specific etc Wiki pages to the database
    • Move Bio and Bibliographic data to the database
Globally OK.Hauck 20:00, 19 February 2017 (UTC)
Along with this (though it might not qualify as part of your currently active projects), I would like to add, it would be very good to try and untangle the main DB from the wiki DB (particularly with respect to login credentials and sessions, etc.; I have a few ideas on this). This would facilitate upgrading our MediaWiki software which we are sorely behind on (including many security/bug fixes as well as features). Uzume 20:41, 19 February 2017 (UTC)
Unfortunately, my knowledge of the MediaWiki software is very limited. Last we talked about development, Al was going to look into this issue, but he has been unavailable for a long time. I could try and get myself up to speed, but, unfortunately, learning new things gets harder as you get older. Ahasuerus 16:18, 20 February 2017 (UTC)
It is already untangled in terms of sessions - I get asked to login in wiki while my session is fine on the site and vice versa. So it is just the users that are the same. Annie 05:38, 21 February 2017 (UTC)

New projects

  • Create a separate queue for automated submissions, which will enable non-moderators to work on them (and let me concentrate on development). Rather time-consuming to implement. Discission moved here.
  • Internationalization:
    • Support multiple prices per publication.
    • Support multiple publishers per publication. May require more thought re: imprints.
      I wonder if that cannot be solved in a different way - allow books published from a joint venture to show on 3 publisher pages - a combined one and the two separate publishers ones. Or if we go for multiple publisher, we should have a way to see which books are from both publishers at the same time without the need to compare lists manually Annie 20:52, 19 February 2017 (UTC)
      An interesting point. I have never considered it, but the proposed behavior seems desirable. We'll have to think of the best way to implement it. Ahasuerus 02:18, 20 February 2017 (UTC)
      And here comes a live example that I and KarenHunt had been investigating: Братья по оружию. It is a joint venture between the Moscow based Астрель and the Minsk based Харвест with separate ISBNs for both (9785271439186 for Астрель and 9789851815919 for Харвест) and usually all copies of the books carry both publishers and both ISBNs (I've had a few similar ones). If you create a fictitious "Астрель / Харвест" (incorrect it is not an imprint) or "Астрель & Харвест", it will remove the book from the two individual publishers' pages. Adding two separate copies (one per ISBN) is not exactly correct either as it is the same book (ha - this is one of the other projects below - multiple ISBNs). The cleanest seem to be to go for one of them and note the other in the notes. None of those scenarios are very good though. Annie 01:09, 25 February 2017 (UTC)
      And just to illustrate why it is a bad idea to use the double name, this wonderful example of Russian thought has 4 valid ISBNs on the same publication - it is again a joint venture between Харвест and this time another of the publishing group АСТ's imprints/daughter companies - this time Хранитель instead of Астрель. But just to make it funnier, it also got ISBN for the parent as well and an ISBN for the pure Moscow "АСТ Москва" which was consolidating their list. So we will need yet another joint version for this one. As you can imagine, there are more of those. :) Annie 01:25, 25 February 2017 (UTC)
      We had a discussion of Russian publishers a while back, which is what pushed this FR up the list of priorities. There are other publishers who use multiple ISBNs, but Russian language publishers account for the majority of our "problem cases". Ahasuerus 01:34, 25 February 2017 (UTC)
      Well - I saw an example, decided to post it so it is in the thread. Sorry if it was repeating already known data - I do not remember stumbling on this discussion when I was reading archives. Annie 01:52, 25 February 2017 (UTC)
      Not a problem at all! I was just sharing additional background information. Part of my job description :-) Ahasuerus 02:03, 25 February 2017 (UTC)
    • ISBNs (mostly to support internationalization, but will also help in other cases):
      • Support multiple ISBNs per publication
      • Move catalog IDs to a separate field, which will help with pubs that have both an ISBN and a catalog ID, e.g. book club publications.
      • Add two checkboxes next to each ISBN: “derived” and “corrected”. The latter will let us capture two versions of invalid ISBNs, the stated one and the corrected one. Discussion moved here.
    • Add support for translators (as well as other roles like single-author collection editors?) Discussion moved here.
    • Move “Add Authors/Transliterated Titles/etc” buttons to the right to free up screen real estate. This will become more important with the addition of new multiply occurring fields (see immediately above.)
    • Change “Family Name” to “Directory Entry”. If we allow multiple values to support different alphabets/scripts, how will we sort by this field? Discussion moved here.
    • Separate translations from VTs on the Title page. Allow users to limit the list of publications to the ones associated with the canonical title.
      Won't that also allow us to group same language translations? If not, then can we have that as well? I would love to be able to see that there are 30 different language translations without seeing that there are 10 French ones until I really want this information :) Annie 20:52, 19 February 2017 (UTC)
      I don't recall this functionality mentioned in the past. I would suggest creating a separate section to explain what you would like to see and to gauge other editors' reaction. Ahasuerus 16:23, 20 February 2017 (UTC)
      I'd like to see this one. -- JLaTondre (talk) 01:22, 20 February 2017 (UTC)
  • Re-do verifications. This will let us have more than 5 primary verifications as well as multiple transient verifications per publication.
    I'd also like to see this one. -- JLaTondre (talk) 01:22, 20 February 2017 (UTC)
  • Create a history of changes to primary-verified publications by storing a snapshot of the way each verified pub looked like right before it was changed.
  • Support for third party IDs (OCLC, BLIC, LCCN, BNF, ASIN, Goodreads, etc). This will enable Fixer to submit ebooks without ISBNs, an increasingly common scenario. Discussion moved here.
  • Make the Quick Tags list user-definable.
  • Add support for additional title types like PLAY and FICTITIOUS ESSAY (requires discussion to come up with additional values.) Discussion moved here.
  • Cleanup reports:
    • Make additional cleanup reports available to non-moderators
    • Review old/incomplete FRs for cleanup reports, add missing title types
  • Add a “non-genre” field to author records; create a cleanup report to find non-genre titles by non-genre authors. Discussion moved here.
  • Add a disambiguator field to title records (requires discussion)
    Don't understand. Hauck 20:08, 19 February 2017 (UTC)
    Is that for the ability to add the reason for the variant (change in name, change in author, translation, abridgement)? If so, I'd love this one. If not - What are talking about? Annie 20:52, 19 February 2017 (UTC)
    The problem that we are trying to address here is that some authors have identically named titles. For example, H. P. Lovecraft's Summary page shows three "The Lurking Fear and Other Stories" collections published in 1947, 1964 and 1971. They are all different, but you can't tell until you drill down to the title level and check each title's Note field. Similarly, there are 2 version of the story "The Rats in the Walls" (1924 and 1956), 4 "[To Albert A. Sandusky]" poems and 9 (sic!) poems whose title is currently entered as "[To ?]". Analogously, Clark Ashton Smith wrote 4 different "Ennui" poems, 2 different "A Sunset" poems, 3 "sonnet" poems, etc. A "disambiguator" field would be used to capture a brief summary of what's unique about each title, e.g. "1964 version", or perhaps the first line if it's a poem. It would be displayed on the author's Summary page. In addition, we will need to decide whether to use this field to capture other information like "abridged", "revised" or "greatly expanded". There were different suggestions the last time we talked about it. Ahasuerus 16:53, 20 February 2017 (UTC)

Ahasuerus 19:50, 19 February 2017 (UTC)

Additional Proposed Projects

  • Add a "printing rank" field to order multi-reprinted titles without pub dates.Hauck 19:57, 19 February 2017 (UTC)
    Yes, please! And not only for dateless - even if we know the dates, a field for edition and printing will be awesome (so if I have 2nd edition, 3rd printing, I can look for 2.3 or something like that and find it without looking through 200 publications). Annie 20:52, 19 February 2017 (UTC)
    There is a Feature request to "Add a 'printing' field to publication records". It would be relatively easy to implement, but we need to decide what kind of data we want to capture. Unfortunately, "editions" and even "printings" do not always follow the standard numbering scheme, so "2.3" won't work. For example, this 1989 edition of Heinlein's "Have Space Suit—Will Travel" says "First Ballantine Books Edition: December 1977 Thirteenth Printing: October 1989". On the other hand, this 2005 edition simply says "First Pocket Books trade paperback edition [no printing number]". This is very common, so you can't expect "editions" to be sequentially numbered. We could limit this field to numeric printing numbers, but there can be any number of "first printings" from different publishers. It doesn't mean that we won't want to implement this field, but we need to understand its limitations. Ahasuerus 04:07, 20 February 2017 (UTC)
    The 2.3 was mostly an oversimplification for the multiple printing scenarios - not necessary requesting it to be numbers only. But we do need some guidelines for formatting in that field - or we will end up with unsearchable field. It will still be better than the current way to find where we are. Even if it is just printing in the field - the edition is kinda clear from the ISBN unless if it gets reused. Annie 04:13, 20 February 2017 (UTC)
    What!? I cannot put Chinese characters in the printing field? But I know of some publishers that mark their printing this way (OK, I am facetiously joking but it is true). Uzume 04:58, 20 February 2017 (UTC)
  • Add a new title type for excerpts, and one for plays/scripts. Neither of those fit nicely in SHORTFICTION. --Vasha 21:02, 19 February 2017 (UTC)
  • Another useful feature, mainly for links from external site to the ISFDB: stable IDs. Currently, if a title is merged into another its ID ceases to exist in the database. As a result, links from external sites to such a title record will become broken links. The software should instead keep track of merged IDs and redirect to the new ID (there was a discussion about this some time ago but I just couldn't find it). Jens Hitspacebar 21:10, 19 February 2017 (UTC)
  • [Discussion of third party IDs moved here.]
    1. The next item I would like to see is revamping verifications. I am not sure how valuable it is but it has plagued us for a while now. Uzume 21:36, 19 February 2017 (UTC)
    2. Finally, author and publisher directory updates seems like a good item. This could bed sooner but probably will be easier after more internationalization/localization is completed so doing it a bit later might prove less problematic. Uzume 21:36, 19 February 2017 (UTC)
  • There had been some discussion about reviews of magazines here. I think the discussion petered out rather than concluded, but did get into alternate ways of dealing with magazine series. I'm not pushing for it, just reminding. Doug H 21:59, 19 February 2017 (UTC)

Editor review of submissions: "automated" submissions sidebar

Create a separate queue for automated submissions, which will enable non-moderators to work on them (and let me concentrate on development). Rather time-consuming to implement. Ahasuerus 00:10, 20 February 2017 (UTC)

Yes, please :) I know that it is a time consuming change and so on but it should help in the long run... Annie 20:52, 19 February 2017 (UTC)
I am for editor review of submissions (by Fixer, etc.), however I question the need for a separate queue. Uzume 23:02, 19 February 2017 (UTC)

(unindent)I have some reservations about "automated submissions" (but that maybe based on my lack of understanding) referred to above in #Roadmap 2017. Why do we need another submission type like this? It seems to me this FR is just to allow editor reviews of submissions (like a voting system to help moderators close pre-reviewed submissions). Is there some reason some submissions should not allow editor review? Uzume 21:45, 19 February 2017 (UTC)

Because at the moment Ahasuerus is doing all of them. This is the process by which all newly published books are added to the system - so we do not wait for people that have them to add them. The idea is to allow the queues that the automated robot finds to be worked on by other people as well - which includes verifying that it is indeed eligible, cleaning the record based on our standard and so on. It is not about approval of entries from people, it is about adding the entries in the DB so the moderators can approve them. Annie 21:50, 19 February 2017 (UTC)
I do not see a difference with this. It is true bot generated submissions maybe of lower quality but then so are submissions from new users. I do not think we need a new field specifying the origin of the submission. I like the editor review of submissions concept (perhaps with comments and queued results to be approved by moderators). In the general sense, consistent high volume and high quality editor reviews of submissions could be a very good way of selecting prospective new moderators. Uzume 21:55, 19 February 2017 (UTC)
We do not have real bot submissions now - Fixer is manually operated by Ahasuerus, title by title, every month. That idea is to allow these queues to be processed by other people as well. Annie 22:03, 19 February 2017 (UTC)
I understand he is doing significant manual work on his side of bot before making submissions and to lighten his load, we want to be able to have his bot make lower quality submissions with better review from our community (instead of just him) and I agree with and would not argue on that point. However, it seems to me a generalized editor submission review process is a better solution than a one targeted just at this problem (which adds unneeded extra levels and fields for such). Uzume 22:10, 19 February 2017 (UTC)
And how is that different from making more people moderators if there are people that can do reviews like that? :) Plus - the process of accepting edits is already slow, do you really want to add more layers? A new editor will get discouraged fast. Plus - that won't solve the issue with Fixer and the sumissions - no matter who reviews, they need to get into the system one way or another. If you want all that it finds to be just added and someone reject it if it is not genre and/or not eligible, that would add a lot more work on the moderators... A Fixer, non-massaged entry is missing series information, have (or can have) a weird spelling on author and publisher, title (Amazon adds a lot of junk there) and it may need to be merged somewhere. No matter how many layers of whatever we have, if those just get dumped into the regular queues, the real updates will get delayed with days until the pure deluge of those is dealt with. Annie 22:18, 19 February 2017 (UTC)
I never suggested making editor review mandatory—just generalized. I do not understand why the normal queues would cause delay. Submissions are not required to be moderated in any particular order (I understand there are order issue with regard to moderation however). Uzume 22:36, 19 February 2017 (UTC)
No but delaying a new contributor edit while we are dealing with internal ones is a bad idea. And if you have 12000 Fixer submissions in the regular queue, how will a moderator find a new one. And even if they can, working on those ratty ones will be a longer and more involved process than on user-created ones. Thus the separate queues and the ability to do something before they hit the moderators. :) Annie 23:13, 19 February 2017 (UTC)
That sounds to me more like better management of the current queue is needed not another queue. Make the submission queue searchable and a moderator can for example just say "I do not currently want to see submissions by Fixer" (or say any user with a bot flag; this list is not very long Special:Listusers/bot). Uzume 23:40, 19 February 2017 (UTC)
The main difference is allowing editing before submitting - allowing the entry to be cleaned up before it goes into the proper queue for approval. This is not something moderators can do now - they can only accept and then edit after that. If you are saying that moderators should be able to directly edit ANY submission before acceptance, then yes, better management is all that is needed. But I am not sure we really need that. Annie 23:45, 19 February 2017 (UTC)
OK, so you are saying you want a separate queue and allow reviewing editors to edit/transmute the submission as part of the review process? That would be very powerful but also extremely difficult to code (since submissions are actually only semi-structured XML documents). Uzume 00:39, 20 February 2017 (UTC)
Thus the comment above that it will be very complicated :) If you read the FR, this is what it describes. Annie 00:40, 20 February 2017 (UTC)
Maybe Annie means a tiered submission process where editors can review automated submissions converting them to normal submissions or some such. But then in my mind "automated" just means "I want editor review first". Uzume 22:02, 19 February 2017 (UTC)
Not really. The current automated process is; Fixer finds the titles (wonderful), Ahasuerus submits them to the DB (from his name or from Fixer's name). Anything with low priority just stays in the queues until someone gets to them. Or forever. It is not about magazines, it is really about what the robot finds and needs to be processed. And you do not want to add to the moderator load all the "cleanup" that is needed - there is a LOT of work to be done on those submissions. And just dumping them into the DB without cleanup, even with a flag will dilute and make the DB useless fast.Annie 22:09, 19 February 2017 (UTC)
I am not arguing against editor review for Fixer submissions. I am arguing this is more elegantly solved with a generalized process of editor review not specific to Fixer submissions. Uzume 22:13, 19 February 2017 (UTC)
Baby steps. Plus Fixer (and any other robots that can find more entries). It is not Fixer specific really - it is the ability to handle those kinds of submissions in a slightly different way. And I still argue that if a person does the submission, you do not need it to be delayed further. :) Annie 22:20, 19 February 2017 (UTC)
So you are saying the "automated" flag adds another tier delaying such submissions. I do not think that is necessary (and if you read the FR a moderator can directly moderated such without editor review anyway). The point is we do not need the flag can just implement generalized editor review of submissions. Uzume 22:35, 19 February 2017 (UTC)
If someone new does not understand the page, they may click it, sending the submission in a long(er) queue. What we need is a double queue - one that only moderators can deal with (the ones that had been seen by a human and just need the second set of eyes) and one for any "ratty" submission where more people can help with. Which is what this FR is for - allow a new process for submissions. Annie 23:09, 19 February 2017 (UTC)
I am all for editor review of submissions to help with such "ratty" submissions. I do not think we need another queue (chosen at submission time), just better management of the current queue should suffice.

OK, after some discussion...if we are talking about a totally different process where someone or something provides suggested changes (I am intentionally avoiding the word "submission") which are reviewed and potentially generate submissions (a sort of reviewed and potentially modified transmutation from the original suggestion), that is a totally different beast and I am not at all sure it makes sense to leverage the current submission system for this sort of thing. I believe Fixer and other bot like "animals", normally do not make arbitrary submissions but only very specific types (mostly NewPub; someone tell me what other types it can make). As such, it might make much more sense to have separate NewPub (suggestion? submission?) queue. A reviewer could then load such NewPub items into the NewPub editor, potentially make changes, and make the submission (or mark the suggestion rejected). This would greatly simply such an implementation (which would still not be trivial). It might also make sense for bots to mark items in this NewPub queue as completed (or actually delete them) when it noticed the items appear in the main database that match the NewPub suggestion it previously made. For this purpose, there should probably be a way for bots to query this queue (with search requests like "give me all outstanding NewPub items I provided but I have not marked as completed/deleted"). Frankly, I see this as more of a remote bot NewPub queue than an automated submission queue. Uzume 01:24, 20 February 2017 (UTC)

AddPub as well when it manages to recognize where a new publication belong. Fixer founds a few thousand new ISBNs every month. They need to be added to our DB - but they do need a lot of cleanup. What is your proposal - how those say 6 0000 publications get added without the new process and without burning out all the moderators? If Fixer can just dump them in a new queue where someone can go, edit and then submit to the moderators, this is exactly what the FR is proposing - call it automated submission queue, call it "list of possible NewPub/AddPub", call it whatever you want :) So do you just disagree with the name? What you describe here is exactly what the FR is proposing... It is not about changing the current process, it is about adding a new process BEFORE the current one that will allow these ratty ones to be streamlined for the process. Annie 01:39, 20 February 2017 (UTC)
That is not how the FR is written "Change the submission table to support a new submission state, “A”utomated" implies modification of the current submission queue. I do not think that is appropriate if what is really needed is a remote bot queue with limited input types. Uzume 02:11, 20 February 2017 (UTC)
Look at point 10 in the FR - that is what ties it back into the standard process. Annie 02:13, 20 February 2017 (UTC)
I see that now, however, I still think that unduly complicates the submission system with little value. I believe we would be better off with a new different specialized queue than trying to shoehorn this functionality into the current submission queue. Uzume 02:20, 20 February 2017 (UTC)
The linked FR would effectively create a new specialized queue. Editors would then use "automated"/robotic entries in that queue to create regular submissions. Implementing various supporting mechanisms -- like putting the automated submission on hold as described in the FR -- would be somewhat challenging, but I think it should be doable. Ahasuerus 04:58, 20 February 2017 (UTC)
I imagine it would be challenging both on the API server side of things as well on the bot client side of that (what does/should Fixer do with such a remote queue item marked on hold?). Uzume 05:23, 20 February 2017 (UTC)
When Fixer sends a submission via the Web API, he marks the ISBN as "submitted" in his internal database. From that point on he ignores the ISBN.
When Fixer gets the next weekly backup, he updates his internal database with the data from the backup. At that point the status of moderator-approved ISBNs changes from "submitted" to "already in the ISFDB". Moderator-rejected ISBNs remain "submitted" forever. It doesn't really make a difference because Fixer ignores "already on file" ISBNs just like he ignores "submitted" ISBNs. Adding a new queue on the server side shouldn't change this logic on the client side. Ahasuerus 15:16, 20 February 2017 (UTC)

(unindent) If there's a separate queue, that queue will probably back up (esp. once the novelty wears off). If it does back up, we'll end up in a situation where more of its entries turn out to be duplicated by live-editor submissions that were created without consulting that queue. It might be better to get more moderators and let Fixer submissions flow through without any sort of pre-qualification/filtering that's being done now. We are not particularly aggressive about recruiting moderators. --MartyD 12:03, 20 February 2017 (UTC)

Unfortunately, the main problem is not that we don't have enough moderators. The main problem is that processing robotic submissions requires a significant amount of research and TLC. Some moderators have the time and the energy to do the necessary research. Others don't and simply click the "Approve" button if the submission looks OK. This is the main reason why I only submit on Fixer's behalf if:
  • I am 99% sure that the submission (usually an AddPub) is truly OK "as is", or
  • it's a relatively low priority ISBN and it's better to have it on file in a potentially suboptimal state than not to have it at all
Over the years, I have tried a number of strategies to improve the situation, all of them unsuccessful.
The "automated submissions queue" is the latest stratagem in a long list. The idea is that having human editors do the requisite research up front would result in higher quality submissions to be reviewed by moderators. Admittedly, if a moderator wants to pull up the new queue and simply approve everything in it, it won't stop it.
If there is a better solution to this problem, I'd love to hear it. The current situation is barely sustainable as it is and will continue to deteriorate as my productivity declines over time. (Not to mention the increased number of SF books being published.) Ahasuerus 19:15, 20 February 2017 (UTC)
That makes more sense and I prefer that idea. This is why I am sort of against the idea of another queue (particularly if it is shoehorned in the the existing one as the first part of the current FR describes) despite its possible value in being able to modify the entries at review time (it seems like much work where this is the only key value). That said, it might make sense to add generalized editor review for the main submission queue (it could be limited in some way if there are security concerns). Such reviews could help moderators more quickly decide when closing submissions. As Marty points out such things may turn into a novelty (but then same could be said of moderators; look at how many inactive or barely active ones we have at the moment), but it might ebb and flow with new or return editors, etc. Aside from the possible value of possible valuable reviews to speed in moderation, it seems like a better tool to use when hunting for moderator candidates (so though this too might be complex and end up perhaps with smaller value this part seems to me the most valuable part). Uzume 18:24, 20 February 2017 (UTC)
If we go this way, we really should have a prioritization of some type available so that live editor updates do not get backed up when Fixer dumps its submissions. It is very annoying to wait for a moderator to approve something so you can send the next update (especially when you are new) :) Annie 05:32, 21 February 2017 (UTC)

Adding support for translators and possibly other roles

Add support for translators (as well as other roles like single-author collection editors?) Ahasuerus 23:45, 19 February 2017 (UTC)

Ok for the former, no to the latter, why not editor of novels? Hauck 20:08, 19 February 2017 (UTC)
A collection editor usually chose the stories, a novel editor does not chose content in the same way. I think we should allow such editors be somehow recognized. Annie 20:52, 19 February 2017 (UTC)
I am with Hervé here. I would like to added more contribution types supported when we have no way to capture such now but I would be wary of taking what we already have and dividing it down (into different types of authors/editors or different types of artists, etc.). I definitely want to see translators (since our current rules have no fields to capture this short of ad hoc notes). I am curious why we cannot just add translators as authors of translation title records and just reach through the VT to find the original authors instead of adding a new contributor type (we do not need one for artists they just tend to have different types of works; a different language VT work is a different type of work). If we implement such, we could have a cleanup report for translations where the canonical authors (jumping through pseudonyms) match authors of the original work allowing us to convert these over time (translations without translators report). Uzume 23:20, 19 February 2017 (UTC)
So you are proposing the Bulgarian variant of Dune to have as an author the translator? How would you separate on the translator page the ones he did write from the ones he translated? And if a user does not understand out variant system, they can misunderstand who the writer is. Plus this way we are losing the ability to have the actual author name in the language of the variant... Artists are different because they are different works - the translation by definition has two authors - the original one and the translator. Annie 23:24, 19 February 2017 (UTC)
The software can display these differently by reaching through the translation variant and author psuedonyms to properly credit authors vs. translators. Uzume 23:33, 19 February 2017 (UTC)
How about an example. Dune by Frank Herbert is translated as Дюн by Франк Хърбърт, translated by Виолета Чушкова. Which name goes where in your idea? Annie 23:37, 19 February 2017 (UTC)
I am assuming we are referring to T1442269. I am suggesting we just add Виолета Чушкова to this record as an author and the software just credits the translators differently than the authors by chasing the authors of this record from A237335 to A30 (since A237335 is a pseudonym) and whatever number Виолета Чушкова ends up (since that is not currently in the DB) and comparing that to the authors from T2036 (which is the variant translation parent of T1442269) which is A30. The software can clearly detect that A30 is the original author and whatever number Виолета Чушкова ends up is not (and thus is a translator). In a similar vein, cleanup reports could be generated for translations without translators. It should be noted that all of the above data is already required to display and properly link these entries anyway. This proposal just suggests handling that data differently to allow for and credit translators (there is already an FR900 for displaying translations differently; I just want to add translator credit support to that and propagating it to more than just title pages). Uzume 00:10, 20 February 2017 (UTC)
None of your links work by the way - you may want to look at the templates again:) The example was from my library. I get where you are getting with that. But... what about the case where it is not just a translation but also an abridgement. This is when the second author gets added now. And that will make the translator author page a nightmare for the software to work out - in which cases the author is a translator, in which cases he/she is a co-author, in which case an author. Not to mention that the searches for translators will become overly complicated. Annie 00:14, 20 February 2017 (UTC)
Currently our rules rule out using the VT mechanism for any derived (where the contents is significantly different) works except translations so that should not be an issue. In partially translated works and the like there should be no VT parent and whether an author is an translator or not becomes muddled to begin with. I believe it would be best to list all contributors as authors and specify who did what in the notes (much as is done with certain types of authorship now). Uzume 00:26, 20 February 2017 (UTC)
Not true. Abridgements are variants - in a way a lot of translations are abridgements, especially the older ones. As are two parts of a split novel. Revised novels also get varianted... Annie 00:42, 20 February 2017 (UTC)
We did not used to allow VT title records except to change the title or author and then later allowed translations (i.e., changes in the language). If indeed we now allow other derived type VT title records, I would prefer a translation flag marking a VT title record as a translation over adding new contributor role flags (add a field to title records instead of to author records). The problem with using the VT mechanism for generalized derivatives is: what constitutes a derivative? Is something a derivative if it is barely inspired by something else? As you can see this gets rather sticky quickly and the resulting connections are more then the VT system can support (since that would actually require a generalized graph construct). Uzume 01:38, 20 February 2017 (UTC)
We are already using them for that (as much as I dislike it in some cases - split novels for example) - the question is how to make it a bit more manageable. And as for the flags - if a variant is an abridgement AND a translation from different people, how do we determine who is who? We can use additional authors but we will need a flag on the author as well... Annie 01:52, 20 February 2017 (UTC)
You cannot use VT to specify abridgement AND a translation by different people anyway. That would require more than a two title records to express (one would have to link the translation to the abridgement and the abridgement to the original; our VT mechanism specifically does not allow generalized graph structures). Uzume 02:08, 20 February 2017 (UTC)
You cannot have a variant of a variant - so if my Bulgarian book is an abridgement and a translation, it goes under the main work directly. You need one title to express - you just need to add notes. Annie 03:21, 20 February 2017 (UTC)
Exactly, and in the same notes that qualify such you can qualify the contributors as well. If you cannot have the titles linked perfectly (outside of notes) it makes little sense to try and credit people perfectly. Just as you have to pick the parent (even if it is not right) you can pick if it is a derived work or a translation (even if neither of those is entirely right either). If we ever support such a linking then the credits will be able to be right too. I see little reason to hack around credits to solve the real problem. Uzume 04:12, 20 February 2017 (UTC)
And here we ended up on a circle. I cannot see how your idea is easier than a new role (considering all the additional work that will be needed to populate summary pages. And if it is not separate, something will need to be done on the publication and title pages to identify them as translators/whatever they are anyway because new contributors will be confused. Annie 04:20, 20 February 2017 (UTC)
Easier solutions do not always make for better solutions (in fact the easy way often ends up being hard long term). I am not sure how new contributors would be any more confused than by which is the proper variant parent (which cannot be set properly too). Uzume 04:54, 20 February 2017 (UTC)

(unindent) The issue of translator support has been open for close to 5 years now, so I don't expect us to come up with a perfect solution in a few hours. There are various aspects to consider, e.g. the fact that there are awards given to translations rather than to titles.

The reason that I mentioned "roles" is that whatever design we come up with will ideally also support other "roles" like single-author collection editors. The exact roles that we will eventually choose to support are less important than the decision to make the design extensible -- or not, if we find that translators are sufficiently different from other "roles". Ahasuerus 05:11, 20 February 2017 (UTC)

Yes, I was not suggesting the translators issue would be solved anytime soon. I am against dividing our author records into roles however, and just wanted to state that there are potential solutions for the translators issue without depending on the introduction of divided contributor role tags. If we do end up supporting such role tags, I would recommend corollaries for publisher roles (or any other authority control item), etc. Uzume 05:16, 20 February 2017 (UTC)
I don't know how useful it is to researchers, but I like the idea of being able to capture/catalogue various roles. Part of the challenge right now is that the nature of the relationship is embedded in one of the two ends (mostly in the publication/title end, but sometimes in the author end). Thinking about the relationship in terms of a generic "contributor", I could see typed Contributor relationships instead -- the nature of the relationship stored on the relationship. There could be many of these, and a pre-defined set of types. One type would be "Author", another "Editor", another "Translator", and so on. The bibliographic summaries for people could be organized by roles, then title types. To keep things simple and avoid big mistakes, the software could provide some specialized handling for required roles (e.g., it could still present Author (for single works, collection, and omnibus) or Editor (for anthology and magazine)) and then "other contributors" or something like that for everything else (just like we get the Title record but then other Contents). --MartyD 17:42, 20 February 2017 (UTC)
I have been thinking along similar lines for a while now. However, redoing the author logic (and the variant title logic and a lot of other things) would be a daunting proposition. It may be less painful to declare authors and anthology/magazine editors a "special privileged case" and implement a generalized "contributor/role" mechanism for translators, other types of editors, etc, separately. Once the new mechanism is in place and everything is working smoothly, we can look into migrating "special case" authors/editors to the new system.
Granted, a two-step approach is not without its dangers -- I once worked on a multi-billion dollar project which took this route and never completed the second step -- but we may be better off even if we never complete the second step. Ahasuerus 23:59, 20 February 2017 (UTC)
Well, we need to solve it if we really want to claim that ISFDB is international. I do not want to add 3000 works just to have to edit them in a year to get the translators sorted... Just saying :) And I like Marty's idea. Annie 00:04, 21 February 2017 (UTC)

Non-Genre flag for authors

Add a “non-genre” field to author records; create a cleanup report to find non-genre titles by non-genre authors. Ahasuerus 00:04, 20 February 2017 (UTC)

This will open the gates to well-intentioned persons that will want to enter such authors (as with the "graphic" or "non-genre") as it will look possible. Bonus problem: how to define a non-genre author.Hauck 20:08, 19 February 2017 (UTC)
[Discussion of author birthdays and dates of death displayed on the front page moved here.]
I believe adding a genre field to authors should be fairly easy and provide good value (like moderator warnings on submissions when adding non-genre or graphic content to non-genre authors, etc.). @Hauck: all authors are non-genre by default and those "over the fold" get promoted to "genre" and we allow (nearly) all content for them. Uzume 21:36, 19 February 2017 (UTC)
Sure, and promoted by who? We're back to the remarkably precise definition of "over-the-threshold" (or "over-the-fold"). This may sound funny but trying to explain for the upteenth time to another new contributor all these perfectly nebular notions may prove an interesting expreience for those who do not practice it. Hauck 21:55, 19 February 2017 (UTC)
Promotion (and demotion) is via the moderated submission process (changing the author genre/non-genre checkbox). It could be restricted to moderators too (like author canonical renaming, etc.) though I am not sure that is necessary. Education is always useful and we already have the rule. The only difference is having a field to declare such. Uzume 22:58, 19 February 2017 (UTC)
I really do not like the idea. I do not think that we should have non-genre works even by the over the threshold authors but that ship had sailed. And how we will determine who is genre author? If someone has a story/essay here, they are an author with genre works. Anyone that has at least one non-genre work is again an author with genre works. Trying to draw the line in the middle is going to cause a lot of unnecessary edits.
I can think of a few Bulgarian authors that had written both SF and Crime works and are considered masters of both. I would not add their crime works here or call them non-genre because they just happen to also had written outside the genre. Even the ones that under the usual "rules" will be over the threshold. Annie 00:10, 20 February 2017 (UTC)
This is already specified in our Rules of Acquisition. I agree some cases are not clear cut. If you want to challenge the rule (or champion a specific author for promotion or demotion) that is a different discussion than implementing it in software. Uzume 00:20, 20 February 2017 (UTC)
It does not determine what a non-genre author is. Again - where would you like to draw the line? If you want it to be based on what an editor wants to nominate, it becomes meaningless... Annie 02:14, 20 February 2017 (UTC)
What you are saying is the current Rules of Acquisition are meaningless in this regard (and I agree that could be argued) but that is a different (though valuable) discussion. Uzume 02:38, 20 February 2017 (UTC)
No, what I am saying is that the rules determine which works are added but do not qualify authors as genre or non-genre ones (above the threshold is not a determination of that).Annie 02:52, 20 February 2017 (UTC)
This seems a pointless designation. If an author has written a genre story, they are by definition a genre author. If it is attempt to designate authors who write primarily genre stories, what is the point? If it is an attempt to designate authors who are 'above the threshold', that is not the same as authors who primarily write genre stories so the name is deceiving. And as Hauk has pointed out 'above the threshold' is a nebulous thing that is not well suited to a database field. -- JLaTondre (talk) 01:15, 20 February 2017 (UTC)
I am curious why you think this is not suited to a database field? I agree an author could be promoted or demoted to threshold levels over time (an author starts out in one genre and gravitates to another). Should instead every editor and moderator know and potentially reevaluate each author upon every submission? That would make for very complex submission criteria and potentially break the rules of acquisition as different people came to different conclusions at each point in time. Uzume 01:58, 20 February 2017 (UTC)
Everyone has a different opinion of 'above the threshold'. People already come to different conclusions. A database field is not going to change that nor is it going to change that different editors and moderators apply it differently. It would just be a field they can now edit war over. -- JLaTondre (talk) 02:17, 20 February 2017 (UTC)
Edit warring over that single field is considerably more noticeable than generalized edit warring over all the pubs and titles of an author which what we are already at. If there is an edit war over that field it will be noticed and brought to community discussion more easily and thereby a consensus can be reached. I agree we could also likely generate some sort of automated heuristic to flag if an author should be above the threshold (by counting title and/or pub records) but it would not take into account exceptions by consensus. But the proposed flag is meant to hold the current value of community consensus (which obviously can change). Uzume 02:32, 20 February 2017 (UTC)

Derived and corrected ISBNs

Add two checkboxes next to each ISBN: “derived” and “corrected”. The latter will let us capture two versions of invalid ISBNs, the stated one and the corrected one. Ahasuerus 00:12, 20 February 2017 (UTC)

Not frequent enough to clutter the screen, better use the note field instead.Hauck 20:08, 19 February 2017 (UTC)
I agree with Herve on that one. Annie 20:52, 19 February 2017 (UTC)
The proposed check boxes would only appear on Edit/Add/Clone Publication pages. Regular Publication pages would continue to display "(Bad Checksum)" for invalid ISBNs like they do now.
The advantage of having invalid ISBNs captured in the ISBN field vs. in the Notes field is that it enables searches. For example, if you search for the ISBN stated in Alpha 5 (0-345-24140-X), you won't find it because it's only available in Notes at this time. When dealing with invalid ISBNs, we have always had to choose whether to enter the stated version or the corrected version in the ISBN field. The other version was relegated to Notes where it became unsearchable. With the addition of support for multiple ISBNs, we will no longer have to choose -- we will be able to eat our cake and have it too! :-)
As far as derived ISBNs go, there are hundreds, if not thousands, of them in the database, mostly from the mid-1970s when the industry was transitioning from catalog IDs to SBNs and then to ISBNs. It's always made me uneasy that we claim, e.g., that Alpha 4's ISBN is 0-345-23564-9 even though it says "345-23564-9-125" in the book. If we add a check box to the edit page, the main Publication page will be able to indicate that the ISBN value was derived. Ahasuerus 03:15, 20 February 2017 (UTC)
I too agree and believe implementing ISBNs as identifiers (a different FR) will likely solve much of this problem anyway. The key is to differentiate identification (e.g., cover price, etc. which can be erroneous) from identifiers (which probably should not be erroneous). Uzume 00:23, 20 February 2017 (UTC)
I still believe most usages of this can be covered by treating valid ISBNs as identifiers (in the as yet unimplemented identifiers FR) and invalid ISBNs as catalog numbers. Here is some potentially pertinent information on this subject: MARC ISBN. Uzume 03:47, 20 February 2017 (UTC)
Back when I was working on the design of the "third party identifiers" FR, I considered making catalog IDs and ISBNs just another type of identifier. Eventually I decided against it for a number of reasons:
  • Unlike third party identifiers, catalog IDs and ISBNs are stated in publications
  • Also unlike third party identifiers, catalog IDs and ISBNs are displayed on various "Publication Listing" pages
  • ISBN searches are "privileged" in the sense that they are available via the main search box
  • ISBN searches use a special ISBN-10/ISBN-13 conversion algorithm so that a search on an ISBN-10 finds a matching ISBN-13 and vice versa
  • ISBNs can be "derived" or "corrected" as described above
In the end I decided that catalog IDs and especially ISBNs were sufficiently different animals to merit separate places in the database. Ahasuerus 04:17, 20 February 2017 (UTC)
Those are good points, however, an ISBN as an identifier can be converted to a 13 digit number in all cases (and left in original ISBN-10, SBN or other brokenness as stated in the publication in other cases like the catalog number field). When searching ISBN identifiers, ISBN-10s to search for can be converted to 13 digit numbers before comparisons (so there is no issue). It would not be so hard to have a combined search for these and the catalog field(s) (matching original input against catalog fields and any convertible ISBN-13 against the identifier fields). Though very common, all (and sometime any) ISBNs for an edition are not always stated in its printed publications (e.g., I have seen some that only have an GTIN-14 which contains the ISBN-13 and another digit). I am not sure it is worthwhile to implement ISBN as both catalog numbers and identifiers and to accomplish such (but it is something to think about). Another issue is broken punctuation in ISBN. I know sometimes an ISBN is stated in a publication with improperly placed hyphens (I have no idea how we handle this if at all; I am guessing we do not maintain this original statement). Uzume 04:49, 20 February 2017 (UTC)
There is a great deal (2427 lines to be precise) of ISBN formatting logic in this module. It's yet another way in which ISBNs are different from third party identifiers. Ahasuerus 15:20, 20 February 2017 (UTC)

Changing "Family Name" to "Directory Entry"

Change “Family Name” to “Directory Entry”. If we allow multiple values to support different alphabets/scripts, how will we sort by this field? Ahasuerus 02:39, 20 February 2017 (UTC)

Switching to unicode and then sorting by it will be cleanest. Until then I kinda like it as it is (as long at non-Latin entries actually have their transliterations properly and not getting defaulted to the name of the author in English. Annie 20:52, 19 February 2017 (UTC)
I have been looking forward to the DB moving to some Unicode encoding like UTF-8 for a long time (currently it is only pseudo-Unicode by way of using HTML entities). Multiple directory entries solves more than just multiple script/language issues (see ISFDB:Community Portal/Archive/Archive41#Sorting, author directory and nobiliary particles). Multiple scripts can be handled by multiple directories. Uzume 00:51, 20 February 2017 (UTC)
Multiple scripts/multiple directories becomes a bit of an issue with author names with letters from two separate scripts - Macedonian or Ukrainian names for example which combine Cyrillic (somewhat creative on some letters) and Latin letters. Which directory will have these forms? Or do we start a separate directory for each of them? Annie 02:44, 20 February 2017 (UTC)
This is not necessarily their real name anyway. I would suggest making entries for both (all Cyrillic and all Latin even if their real name is a mixture of the two). Uzume 03:03, 20 February 2017 (UTC)
For Ukrainian and Macedonian authors, it IS their real name. And we all hope we will get more international entries, right? Saying that they do not deserve their real names in a directory while we have everyone else's name there will be a bit... uninclusive. Annie 03:19, 20 February 2017 (UTC)
I do not see how that is an issue. The above mentioned "nobiliary particles" discussion proposed having multiple entries based on different cultural conventions in different places. Uzume 03:29, 20 February 2017 (UTC)
So separate directory for each language we support (I am just trying to make sure I understand the idea) Annie 03:37, 20 February 2017 (UTC)
No, not for every language. One per script (all Latin in one, all Cyrillic in another, etc.) implemented as time and demand allows. This means for example English language authors would be found in the same directory as German ones. Another example of directory entries that do not match real names would be Japanese (which would have kana entries) where I do not think we want to consider a kanji Chinese character directory (I have no clue how we would do such for actual Chinese names). Uzume 03:53, 20 February 2017 (UTC)

(unindent) Let's take a step back and examine the uses (current and proposed) of the "Family Name" (soon to become "Directory Name") field. At this time it is used to do 2 things:

Making this field a "multiply occurring field" will make it possible for an author record to appear on multiple Author Directory pages. For example, if we specify "de la Roche" as well as "Roche" as Mazo de la Roche's Directory Entries, his name will appear on two different Author Directory pages. Once we implement non-Latin Author Directories, it will also enable us to make the same author appear in multiple directories.

So far so good. However, once an author has multiple "Directory Entry" values, we will have no way of using them for sorting purposes. Unless we make one of them our "preferred sorting value", I guess. Ahasuerus 03:49, 20 February 2017 (UTC)

Agreed. And as such instead of having a "preferred" tag, why not treat this like transliterations. You have a Family name with "transliterated" Directory names on top. This would allow a mixed (potentially collating) name as a family name (we could have Cyrillic and Latin or kanji and kana in this field) and single script directory names on top. Uzume 03:57, 20 February 2017 (UTC)
I am not sure I understand what "on top", "mixed name" and "potentially collating name" mean in this context. It sounds like you may be suggesting that we keep the current "Family name" field and add a new multiply occurring field for "Additional Directory Entries". The latter would be used for names like "Roche" in the "Mazo de la Roche" example above as well as transliterations. The former would continue to be used for sorting purposes. Is that about right? Ahasuerus 15:45, 20 February 2017 (UTC)

Add support for additional title types

Add support for additional title types like PLAY and FICTITIOUS ESSAY (requires discussion to come up with additional values) Ahasuerus 02:41, 20 February 2017 (UTC)

Not enough candidates to warrant new types.Hauck 20:08, 19 February 2017 (UTC)
Maybe not for fictitious essays; --Vasha 00:52, 20 February 2017 (UTC)
"Fictitious essays" or "fictional essays" are basically "in-universe" essay. They are written as if the events of the work that they are associated with were true. They have prompted many discussions since it's not clear whether they are "essays" or "short fiction". Help currently says:
  • Some books contain fictional essays, purporting to written by a character in the book, as introductions or afterwords. There is no "FICTIONAL ESSAY" title type, so you have to choose whether the title is better described as SHORTFICTION or ESSAY.
There have been repeated request to implement a separate title type for them. Ahasuerus 02:48, 20 February 2017 (UTC)
Well then, if there have been repeated requests, clearly there are enough cases to justify adding the new type. And I will say it definitely sounds useful to me. --Vasha 03:37, 20 February 2017 (UTC)
but for plays, even if there aren't that many of them, the difference between them and SHORTFICTION is so obvious that they need their own title type. --Vasha 00:52, 20 February 2017 (UTC)
I would also like to see a separate title type for plays, but we will need to decide how to handle related cases first. For example, are movie scripts, TV scripts, etc separate types or are they all the same type? If they are the same, should we come up with a more generic term than "play"? Ahasuerus 02:51, 20 February 2017 (UTC)
"script" covers plays, audioplays, tv scripts, and even librettos I think. --Vasha 03:37, 20 February 2017 (UTC)
(And for the other one I would like to see a new type for, excerpts, there really are a lot.) --Vasha 00:52, 20 February 2017 (UTC)
The vast majority of excerpts are works of short fiction. Should we treat them as a separate title type or as a separate "length" value like novellas, short stories and novelettes? Ahasuerus 02:52, 20 February 2017 (UTC)
I'd like a separate length value for these - this will separate them from the real short fiction. Annie 02:54, 20 February 2017 (UTC)
Keep in mind that if we add "excerpt" as a new "length" value, it won't separate excerpts from other works of short fiction on Summary pages. Ahasuerus 03:18, 20 February 2017 (UTC)
I believe you mean it won't currently separate such (the software could be made to do such without huge issues though I am not convinced that is a good idea). I vote for an "excerpt" "length" value. Uzume 03:21, 20 February 2017 (UTC)
I do not like seeing novel excerpts mixed in with the actual short fiction on the summary pages. --Vasha 03:37, 20 February 2017 (UTC)
It does not do it now anyway (See Asimov - the third and the forth are excerpts... If we go for separation one day, we may start talking about separating all lengths from each other anyway. What I meant upstream was that it will allow the excerpts to be seen easier. And allow searching them - and searching without stumbling at them. Annie 03:40, 20 February 2017 (UTC)
Agreed. Though I am not convinced about separating such on author pages. One can create custom pages via search that show just what you want. Uzume 04:17, 20 February 2017 (UTC)
I would be strongly against separating all lengths on author pages. It's not of interest under most circumstances, plus every author has lots of unspecified lengths. --Vasha 04:21, 20 February 2017 (UTC)
I am not exactly vying for it but it might be interesting to consider user preferences allowing one to select which lengths one wants to see (much as language selection is possible this way). Uzume 04:25, 20 February 2017 (UTC)
I like that, it would speed up searching for titles on prolific author pages. Maybe checkboxes for all title types on the author page rather then setting it in user preferences.--Rkihara 18:23, 20 February 2017 (UTC)
The question wasn't choosing which title types to display, but rather displaying ss/novelette/novella separately. It's this that I think would not be good on summary pages. As a preference option, OK maybe. --Vasha 20:55, 20 February 2017 (UTC)
If the intent is to separate excerpts from other SHORTFICTION titles on Summary pages, then I believe they need to be turned into a separate title type. Using "length" values to create separate Summary page sections would go against the basic "one title type = one Summary page section" principle. Ahasuerus 23:43, 20 February 2017 (UTC)
Excerpts are short fiction -- it does not really make sense to have them as a separate type any more than novellas are separate type. At least I cannot see a difference... If we split them in a separate type, then why would not things like half a novel published in a volume go into that same new type? Where do we draw the line? At least as a part of the short fiction, that line is clear. Annie 17:05, 21 February 2017 (UTC)
Yes, sometimes excerpts are carefully selected so that they form a complete short story; more often they actually don't seem complete in themselves. --Vasha 17:39, 21 February 2017 (UTC)

(copied from a side discussion in another section) If excerpt got added to the mix [of length options] it would be hard to "sort" that with the others. Uzume 01:28, 25 February 2017 (UTC)

I think you may be the only one who likes the idea of "excerpt" being a length... everyone else was talking about it being a title type --Vasha 01:32, 25 February 2017 (UTC)
That was not how I read things (but perhaps I misunderstood). As title type seems that seems problematic. As a length, I could usefully apply it to essays and nonfiction (like an excerpt of a bibliographic index, etc.) as well. Uzume 01:43, 25 February 2017 (UTC)
I read it as a support for Length as well :) However to your latest point here: length may be showing when you are working on publications for all titles but if you read the documentation and look at editTitle for example, it is valid only for Short Fiction. Making it usable elsewhere will probably require a lot of other changes (even if it becomes a length). Annie 01:46, 25 February 2017 (UTC)
OK, I can sort of see excerpt as a length, in that I hesitate to assign a length to excerpts -- they don't strike me as "really" being a short story or novelette. But that is the very reason that they seem to me to be a separate kind of thing, not short fiction. Besides, I like the idea of having excerpts be a separate section on summary pages, for which they'd have to be a title type. But maybe I am the only one who wants that. --Vasha 02:15, 25 February 2017 (UTC)

Third party identifiers

Support for third party IDs (OCLC, BLIC, LCCN, BNF, ASIN, Goodreads, etc). This will enable Fixer to submit ebooks without ISBNs, an increasingly common scenario.

Good to have some way of identifying ebooks but most of those you listed are ephemeral. Particularly ASIN. (I don't put ASIN in the notes field of an ebook because there are probably numerous ones most of which will disappear soon.) Goodreads has the advantage that they keep records for out-of-print ebooks (it's their policy not to delete) but they're very unsystematic, their records can get merged with each other, etc. --Vasha 17:49, 20 February 2017 (UTC)
Library IDs (OCLC/WorldCat, BLIC, LCCN, BNF, etc) are generally stable, at least as stable as our IDs. We currently link to them in Notes, which requires crafting URLs manually, an error-prone and time-consuming process. In addition, manually built links can become defunct if/when the third party changes its URLs conventions, as we have seen over the last few years. ASINs may not be quite not as stable as, says, LCCNs, but in most cases they work even years after the data was captured. They are frequently the only way to link to the source of our data. Even more importantly, without ASIN support Fixer is unable to submit ebooks without ISBNs, which are increasingly common. Even major authors publish e-stories without ISBNs these days and Fixer can't do anything about them. Ahasuerus 18:10, 20 February 2017 (UTC)
I would certainly like to be able to use library IDs as identifiers for non-ISBN print books. Having support for that makes sense. But Worldcat doesn't list ebooks -- what about other libraries? --Vasha 18:30, 20 February 2017 (UTC)
Some do, but the intent of this FR is to support third party IDs regardless of whether they are for paper books or ebooks. Ahasuerus 18:46, 20 February 2017 (UTC)
As for automatically importing ebooks, that opens a can of worms. We had an inconclusive discussion a while back about what constitutes an "edition" of an ebook. Personally I would like to see them condensed not too many; the way Goodreads has so many is bewildering and ugly. So if you automatically import them there's have to be manual combining. --Vasha 18:30, 20 February 2017 (UTC)
Fixer has been generating ISBN-based ebook submissions for many years. Submitting ASIN-based ISBN-less ebooks should be no different than submitting ISBN-based ebooks once we add support for third party IDs. Where it will get challenging is if we start using another source of ISBN-less ebooks. Suppose Fixer creates a database record for an ISBN-less ebook whose ASIN is "ABC". 6 months later an editor (or another robot) wants to add a record for a Barnes & Noble ebook whose B&N ID is "XYZ". How can we tell whether it's same version of the ebook, especially if the publisher is not specified and the publication dates are close? OTOH, as long as we stick to ASINs, we should be in reasonably good shape since different ASINs are generally associated with different editions. Ahasuerus 18:56, 20 February 2017 (UTC)
Amazon actually doesn't have every single ebook. Graydon Saunders is a fairly notable example of an author who choose not to offer his books via Amazon because he objected to their terms. --Vasha 20:21, 20 February 2017 (UTC)
Oh, definitely. However, as long as we add these books manually, we are less likely to run into problems. It's the robotic submissions that worry me. I have seen many cases where an ebook was originally added by a human editor without an ISBN -- because Amazon doesn't display ISBNs for ebooks even if they exist -- and then Fixer came along and tried to add the same book using its ISBN. (This particular problem will be alleviated by adding support for third party identifiers.) If we were to start using other online sources to grab ISBN-less ebooks, the problem would become more severe. Ahasuerus 20:35, 20 February 2017 (UTC)

(unindent) I believe the next most interesting item is adding support for identifiers. This is a bit sticky but useful concept. The sticky part comes in where we apply these. I believe you were mostly considering at the pub record level, however, the identifiers in this area are themselves usually defined for the concept of an "edition" (which we have no real records for since our pub records most closely correspond to a printing, save for SFBC). This probably is not a major issue but something to consider. Uzume 21:36, 19 February 2017 (UTC)

True, different catalogs and Web sites create records at different levels. For example, an OCLC record may cover multiple ISBNs. Alternatively, multiple OCLC records may cover the same ISBN. I don't think there isn't much we can do about it. Adding support for third party IDs is the best I can think of. Ahasuerus 19:50, 20 February 2017 (UTC)

Next, there is the issue of untangling ISBNs. Currently, our pub record ISBN field represents an identifying mark of the publication (along with cover price, etc.) and we just sort of play games with it to allow is to link to other sites by using it as an identifier. Uzume 21:36, 19 February 2017 (UTC)

Well, ISBNs were originally supposed to be unique and they are still used as "mostly unique" by the industry, e.g. when ordering books. Of course, as we know, publishers can tweak things like cover art and price while keeping the same ISBN. And sometimes they mess up and reuse an ISBN which they didn't intend to reuse. Still, ISBNs are the only IDs that can be used cross-catalog and cross-site with a reasonably high chance of success. Our use of ISBNs to link to other sites is the way they were intended to be used. It's not perfect -- see the OCLC discussion above -- which is one of the reasons why adding support for third party identifiers is useful.
For example, suppose an editor wants to record that some of our data originally came from OCLC records. At this time the only choices are either to:
  • enter something like "Corrected page count from OCLC 1234567, artist from OCLC 987653", or
  • manually craft URLs pointing to the OCLC records
The second options provides more value but is error-prone and time-consuming, not to mention that URL structures can change over time. Once we add support for third party identifiers, editors will be able to enter OCLC record numbers directly. Ahasuerus 20:05, 20 February 2017 (UTC)

An easy way to untangle this is to run some code on our ISBN fields and move them to ISBN identifiers and keep the original field for verbatim publication catalog markings (including wrong/broken ISBNs, etc.). This way we change the current field to a verbatim publication identification field and move identifiers and external linking to use the newly introduced identifiers (including derived/corrected ISBNs etc.). Then we just have to update documentation to keep the fields separate (I recommend renaming the current field as catalog number and documenting it to apply to cover or copyright page catalog numbers, etc.; we could expand it to allow multiple values but frankly I would say that unneeded and can be covered in the note field). Finally, identifiers are also useful at other levels such as author and/or publisher authority control (e.g., VIAF, LCNAF, etc.), and for tags with regard to subject indexing terms (e.g., LCSH, FAST, etc.). Some identifiers also apply to the work (our title record) level (e.g., Open Library works, Wikidata, etc.). Uzume 21:36, 19 February 2017 (UTC)

Yes, author-based and title-based identifiers are something to consider. Currently we enter VIAF links as "Web sites". If and when their URL structure changes, we will have to change the URLs manually or write a conversion script. If we had author-based third party identifiers, the problem could be resolved in 10 minutes. I agree that they would be nice to have, but I see it as a much lower priority than publication-based identifiers. Ahasuerus 19:32, 20 February 2017 (UTC)
Maybe, but we could for instance move Wikipedia links to go through Wikidata identifiers with links like enwiki/Q982133, frwiki/Q982133 and jawiki/Q982133 for John Norman allowing the links to be translatable (without storing and maintaining all the various links) not to mention other possibilities like enwikiquote/Q982133 (provided the sitelink for such articles exist). Uzume 00:47, 25 February 2017 (UTC)

(unindent) OK, I have been thinking about this -- how about this for an addition to the edit-publication page. Below the ISBN field, the ability to add fields for an indefinite number of other identifiers (a button that says "Add Identifier" or whatever), and next to each of these fields a dropdown list to specify what the identifier is: ASIN, OCLC number, publisher's catalog code, however many options seems useful. (Naturally, the submission would be flagged if you try to submit a number without specifying its type -- and maybe, at least for some of them, could be flagged if it's not a valid format for the type specified.) Would that work? --Vasha 01:52, 2 March 2017 (UTC)

That's actually extremely close to the implementation that I sketched out last year :-) Ahasuerus 03:11, 2 March 2017 (UTC)

Changing the front page

As we discussed earlier, the list of authors born today is becoming longer and longer. It's up to 70+ authors now and will only get longer as we enter more DOBs.

Based on the last discussion, here is what I propose we do:

  • Create a new Web page which will show a list of all authors who were born/died on this date
  • Limit the left column ("Authors Born On This Day") to a list of 10 people born on this day
  • The list of 10 people will include award-winning/nominated authors as well as "marque" authors, i.e. the most viewed authors in the database (2%)
  • Display a link to the new page which shows all authors born on this day (see above) in the left column
  • Replace the right column ("Authors Who Died On This Day") with a list of most recently edited records. It will contain the first 10 values from the "Subject" column of the Recent Edits list.
  • Display a link to the full list of "Recent Edits" in the right column

How does it sound? Ahasuerus 01:36, 1 March 2017 (UTC)

I really do not see the value in this. As discussed, every heuristic has its flaws and though the list is now larger, it is relatively small (in terms of bytes transferred; I concede it takes up consider screen real estate vs. bytes transferred). If the screen real estate is the issue, why not just move those sections down below the "Selected Forthcoming Books" section. That way it is still there but way "below the fold" when viewing the page and one has to scroll to get there (meaning still there but less prominent). I thought I would note approximations of these lists can be generated with advanced searches like: Birth 03-01 and Death 03-01 Uzume 16:49, 1 March 2017 (UTC)
I think a smaller list of birthdays would in fact be more interesting. The long list doesn't catch the eye; people would be more likely to read it if it was shorter, and if the first few names included someone familiar. They'd note the familiar ones and maybe click through to the one or two they didn't know. --Vasha 18:38, 1 March 2017 (UTC)
If we are considering pushing the larger list to a separate page, why not make it more of a date-based author directory (meaning one can see such lists not only for today but also for other days and perhaps have it navigable via a calendar of sorts). We might even have links into these pages (I can possibly foresee links from authors birth and dates to these labelled things like "Other authors born the same day" and "Other authors who died the same day"). Uzume 18:46, 1 March 2017 (UTC)
We can certainly create an additional Web page which will look like a calendar. Clicking on one of the 366 choices will take you to a list of authors who were born/died on that day. Ahasuerus 20:15, 1 March 2017 (UTC)
I love that idea. :) Annie 20:35, 1 March 2017 (UTC)
FR 998 has been created. Ahasuerus 19:23, 3 March 2017 (UTC)
Sounds okay for me, except that I think the list of "Authors Who Died On This Day" would pair more graceously with the list of birthdays. Maybe it would be possible to feature the list of recent edits more prominently (analogous to the list for all birthdays). Stonecreek 20:01, 1 March 2017 (UTC)
I think there is enough real estate on the front page to keep the two author columns and add a third one for recent edits. OTOH, there have been complaints (both here and elsewhere) about the "Death" column being too morbid to appear on the front page. At one point someone even requested a user preference to hide this column. Ahasuerus 20:19, 1 March 2017 (UTC)
Well, as someone that prefers smaller screens on her desktops and laptops and with all the people using tablets and phones (I do the latter to edit now and then but I am not expecting to see everything well), I would be careful with adding more columns. Annie 20:35, 1 March 2017 (UTC)
It would be also nice to have a special representation in case an author's birthday happens to be 50, 100, 150, 250 years ago (regardless of prominence or awards). Stonecreek 20:01, 1 March 2017 (UTC)
That's a good point. We'll have to think of the best way to implement it. Ahasuerus 20:20, 1 March 2017 (UTC)
I like the proposals, especially the new list of recent edits, because that way the occasional user can immediately see if and how the site is still active. The "calendar" mentioned above is a nice idea, however it should be of low priority compared to other new features. Jens Hitspacebar 21:12, 1 March 2017 (UTC)
Here's the way I organized it for the speculative fiction portal on Wikipedia. There are additional items, too, but this should convey the general idea. ···日本穣 · 投稿 · Talk to Nihonjoe 21:23, 1 March 2017 (UTC)
Gets my vote!--Rkihara 18:59, 2 March 2017 (UTC)
That is exactly what I meant and yes it could be expanded to include things like awards (Person X won the A award for work W today/specifc day Y years ago, type of thing), and birth and death anniversaries as well, etc. They could be sorted by age (and possibly grouped in increments like 25 years or so), etc. I do not really see the need for the change to the front page (though it could be reorganized moving such parts down if it bothers people) however if the birth and death dates do get removed (or restricted to notable by some heuristic) from it, I believe it should be either totally disposed of (and just made into search links as I mentioned above) or expanded in the calendar directory fashion. Uzume 20:01, 2 March 2017 (UTC)
I still think that we should have a "representative sample" on the front page. A calendar is a good idea but removing the list from the page altogether just does not sound right :) Annie 20:08, 2 March 2017 (UTC)
You can see how the SF portal is organized here. Maybe that will give some ideas on organization? ···日本穣 · 投稿 · Talk to Nihonjoe 21:33, 2 March 2017 (UTC)
I've seen it. :) I find it overloaded and containing so many things that you cannot find anything. I like our much smaller and cleaner (and with a lot more white space) one a lot easier on the eye and a lot more likely to have something that catches my eye. It needs to be like that I guess - because of the scope but turning ours into something like that will be a bad move in my opinion. Annie 21:49, 2 March 2017 (UTC)

(unindent) Outcome: FR 999 has been created to reflect the originally proposed design. Once it has been implemented, we can tweak what is displayed in the columns. Ahasuerus 19:23, 3 March 2017 (UTC)

Mayan languages added

"Mayan languages" have been added to the list of supported languages. Ahasuerus 21:19, 2 March 2017 (UTC)

Yey! I will go and fix the two authors that need it. Annie 21:43, 2 March 2017 (UTC)

Standardizing "Edit" links

"[Edit]" links were added to the bibliographic pages a few weeks ago. They are currently displayed as regular HTML links in square brackets. At the same time we display the link that takes you to the tag editor ("or manage Tags"), the "VOTE" link, the "view full Note" link and the "View all covers" link using a special "inverted" font. See Garan the Eternal for an example of a title page that uses all 4 links.

Should we standardize the appearance of these links? If so, should we use the regular link format or the inverted one? It's a trivial change either way, we just need to decide what we want these links to look like. Personally I think that the "inverted" font looks spiffier. Ahasuerus 21:45, 2 March 2017 (UTC)

Now that you mention it, having the edit link in the inverted form would look good. But the location is right I think. (BTW, out of curiosity, how often does the "Vote" function get used?) --Vasha 21:51, 2 March 2017 (UTC)
According to the ISFDB Statistics page, 9,311 (or 0.72%) titles have votes associated with them. Ahasuerus 21:59, 2 March 2017 (UTC)
I prefer the standard links - it took me a bit to actually realize that the inverted font meant links when I joined. Annie 21:53, 2 March 2017 (UTC)
As I recall, some other editors have run into this problem a well. Hm... Ahasuerus 22:00, 2 March 2017 (UTC)
Seems quite obvious to me (and also a good way of visually distinguishing functions, versus links to data pages which are ordinary links), to add a data point in the opposite direction. --Vasha 22:18, 2 March 2017 (UTC)
I must be particularly dense. What is an inverted link font anyway? Anyone care to explain (Annie's comment about "links when I joined" does not really help me and I joined ISFDB years ago). Uzume 01:43, 3 March 2017 (UTC)
The links that look like they are highlighted in blue (blue background, white letters). Look at the link from Ahasuerus above - there are a lot of them. Annie 01:50, 3 March 2017 (UTC)
The links that I mentioned earlier use a blue background. The CSS definition is a follows:
.inverted {
   font-size: 90%; 
   background: #00f;
   color:#fff;
   text-decoration: none;
   padding-left: 5px;
   padding-right: 5px;
   white-space:nowrap;
}
I now understand, what is meant is the CSS class named "inverted" which has colors that apparently seem inverted from other links (I do see them as white on blue). Thank you. Uzume 02:34, 3 March 2017 (UTC)

I cannot agree on how spiffy such links appear, however, I do agree it makes sense to differentiate action links from normal navigational data links (which should likely be persistent and externally linkable; action links like edit and vote, etc. require a login and need not be persistent so could be renamed and moved in URL space). Uzume 05:04, 3 March 2017 (UTC)

Two new title types to implement

There seems to have been general acceptance of the idea of creating new title types for scripts and fake essays (what to do about excerpts, on the other hand, remains uncertain). I think they would be easy to implement -- almost all existing examples of scripts could be found by searching notes, and as for the essays, some creative searching could find them too. The only thing left to decide is what to call them. I suggest "script" and "fictive essay" (not "fictional"). --Vasha 23:23, 2 March 2017 (UTC)

"In-universe essay", perhaps? Ahasuerus 23:49, 2 March 2017 (UTC)
It ought to include things like Álvaro Menén Desleal writing a preface for his own collection of stories and signing it Jorge Luis Borges, which was intended and received as a joke between fellow South American spec fic writers. Borges was himself a writer of metaliterary pieces like the "Chronicles of Bustos Domecq", so does that make Desleal's essay somehow, some way, in-universe? --Vasha 00:41, 3 March 2017 (UTC)
The Contento1, Locus, Miller/Contento and FictionMags sites use "facetious article" with the abbreviation "fa". For scripts, they use "play" with the abbreviation "pl". FictionMags specifically states that play is meant to include scripts of films, radio and tv episodes. Their full list of types is here. I haven't done a full compare, but I believe that these databases (three of which are our standard secondary verification sources) essentially use the same set of title types. --Ron ~ RtraceTalk 00:50, 3 March 2017 (UTC)
"Facetious" suggests humorous intent which isn't necessarily the case; for instance, an introduction to the world of a fantasy series written as a supposed excerpt from a guidebook. --Vasha 00:58, 3 March 2017 (UTC)
For reference, "comic book" also suggests humorous (the Japanese word "manga" also has similar comical meaning) but methinks that term is well entrenched in English now (though we tend to use "graphic novel" here). Within some of the works of the Gor series written by John Norman, a pseudonym developed by the real author, are envelope notes. I believe this is mostly within the earlier works of the series but I know one example is Outlaw of Gor (I happened to have a copy on hand for reference) which has a leading "A Note on the Manuscript" and "A Concluding Note on the Manuscript" which are signed by John Norman and explain how he came to get the material from a lawyer friend named Harrison Smith without actually meeting the main character of the novel, Tarl Cabot. Further in that novel, the first chapter entitled "The Statement of Harrison Smith" is purportedly by the lawyer friend. The rest of the novel uses the first person to refer to the main character Tarl Cabot. I suppose these pieces of the novel might qualify under this proposed fake essay/facetious article title/work type, however, I am not convinced we need to separate these novel pieces as separate works (anymore than we denote all the chapters of a novel here). If we do in fact include such such a new title/work type, we need to be clear when it should be invoked. Frankly, I would rather we had another title checkbox or the like (similar to graphic and non-genre) to denote such things instead of a new title type (then it could be applied to any fictional title type that applies). In a similar fashion I prefer making excerpts a length. I see no issue adding a script/play title/work type and see it much as our current poem title/work type. Uzume 02:24, 3 March 2017 (UTC)
I believe there are two separate questions here. The first one is which framing stories and in-universe essays need to be entered as separate titles. The second one is how we want to enter the ones that should be entered separately. The current proposal deals with the second question only.
Making "fictive" a check-box on the Edit Title page is an interesting idea which hasn't been suggested before. Something to ponder... Ahasuerus 03:21, 3 March 2017 (UTC)
I fully agree. The point was to think about things as considering how we want to use something often influences how we want to and ultimately how we implement them. Uzume 05:49, 3 March 2017 (UTC)
Well, for what it is worth I needed a dictionary for the word "facetious" a few months ago when I crashed into our sources for the first time while looking for something. If we use this, a lot of international contributors will be a bit lost - the system is already complicated (EDITOR is already a challenge and it is technically never added manually; this one is something people need to be able to select from a list... I understand that it may be the standard but can we come up with something that does not require someone to look up the meaning of - not everyone had even heard of our sources; and even when they had, understanding what this type is when one had not seen a US magazine that contains them is a bit hard. I like "In-universe essay" - it is clear enough so someone seeing it on a list can recognize a piece of writing as being one of those. Just my 2 cents. Annie 16:59, 3 March 2017 (UTC)
Yes, seriously. "Facetious article" may be acceptable as insider jargon, but since this database is aimed at casual users, we need to pick a less bewildering expression. --Vasha 17:10, 3 March 2017 (UTC)
I still don't see the need to implement a new title type for 'fictional essays': if it's in-universe it is fiction, however 'serious' the overall tone may be, if it is rooted in reality it is non-fictional, and as such an essay (ESSAYS being the little offsprings of NONFICTION, SHORTFICTION the ones of longer works of fiction, i.e. NOVELS). Why make it complicated if it can be quite easy?
A 'script' or 'play' title type would be nice, though. Stonecreek 18:52, 3 March 2017 (UTC)
I agree with Christian about the first type, it's useless. The plays are, IMHO, not frequent enough to warrant a new type (why not also TVSCRIPT, FILMSCRIPT, PROSEPOEM, HAIKU, VIGNETTE, etc...). On a more general theme, please remember that the multiplication of possible choices in our categories (let's not talk about format/binding) makes the task of explaining (and correcting) them more and more difficult for those of us that undertake the task of "speaking-to-new-contributors". I'm not sure that all of the participants of this discussion remember clearly how complicated the ISFDB is for a new entrant.Hauck 19:02, 3 March 2017 (UTC)
I do not think that a "play" type would be at all confusing, and even though it's true that there's only about a hundred examples so far, still they're obviously distinct. As a bonus having this title type would allow doing away with some disambiguators like "Dracula (stage play)".
As for the confusingness of fictive essays, I'm coming around to the idea that having them as a check-option for shortfiction would be best. It's less serious if people don't check boxes that should be checked than if they assign things to the wrong type. --Vasha 20:22, 3 March 2017 (UTC)

(unindent after an edit conflict) Let's take a step back and consider whether "facetious/fictive essay" would be best handled as a separate title type or as a flag similar to the currently existing "non-genre" flag.

Aside from the Title page, which is self-explanatory, I can think of two bibliographic pages which will display the data differently depending on our choice:

  • the Publication page, specifically its Contents section, and
  • the Summary page

If we make it a title type, the Contents section of a Publication page will say something like:

  • vii • Introduction (The Statement of Stella Maberly, Written by Herself) • fictive essay by F. Anstey [as by uncredited]

If we make it a flag, here is what the same line will look like:

  • vii • Introduction (The Statement of Stella Maberly, Written by Herself) • short fiction [fictive essay] by F. Anstey [as by uncredited]

As far as the Summary Bibliography page goes, the difference is that making it a separate title type will result in a new Summary section. On the other hand, if we make it a flag, then the affected titles will appear in the existing "Short Fiction" section with a "[fictive essay]" indicator added.

My current thinking is that a separate Summary page section for fictive essays may be excessive, so I would be inclined to go with a flag. We will then need to decide whether the title type of "fictive essays" should be ESSAY or SHORTFICTION. Ahasuerus 18:56, 3 March 2017 (UTC)

"fictive essay" sounds like "a little bit pregnant" to me. And if we go with that, would someone decide that this is a good flag for a 1950s science article that was essay back then (because it was considered true) but since then had become invalid (so not non-fiction)? Annie 19:06, 3 March 2017 (UTC)
As for whether this type is useful... Well, the Mad Scientist Journal is almost nothing but in-universe pieces. Every issue has an "editorial", "advice column", "horoscopes", and "classifieds" supposedly written by mad scientists. I really don't know what to call those -- they aren't short stories or novelettes. They have to be something else, whether it's a flag or a new type.
Annie's concern would be addressed if this was handled as a labelled subtype of SHORTFICTION. I don't think anyone would shift an essay from nonfiction to fiction just because it lauded Wilhelm Reich. --Vasha 20:02, 3 March 2017 (UTC)
We generally decide whether a work is fiction or non-fiction based on the stated intent. If a story about a kidnapping by aliens is presented as a true account, then it's non-fiction regardless of how accurate it is. In addition, as per our definition of speculative diction, SF includes "works set in a future that is now in the past [and] that deal with technological advances that were futuristic at the time they were published", so I think we are covered there. Ahasuerus 20:25, 3 March 2017 (UTC)
I am mostly thinking about someone not reading the rules and seeing a name in the list that sounds like something else - all of our types are self-explanatory on their own (except EDITOR - but that is a different story). I know that docs clarify it, but we have a lot of those. :) Annie 21:49, 3 March 2017 (UTC)

Submission viewer tweaked

The "submission viewer" page has been tweaked to display the name of the reviewing moderator, e.g. see this Fixer submission. Ahasuerus 16:49, 3 March 2017 (UTC)

Ethereal Tales, #7

Can anyone see any reason why these two: Publication 1 and Publication 2 need to stay separate? It does look as an oversight when the second one was added but if someone can confirm, I can go ahead and merge all of the matching elements (and find out which ones are not matching and why). Annie 17:02, 3 March 2017 (UTC)

Well, no! This really is a double entry. Stonecreek 18:44, 3 March 2017 (UTC)
I'll go clear them up then. And there is a serial hiding in there that the PV'd copy had not handled very well so I will also send them a note on the changes. Annie 19:09, 3 March 2017 (UTC)
I also noticed that the dates of the issue and contents are set to something other than the cover date -- I guess that ought to be checked for all VWCrist's submissions. --Vasha 23:44, 3 March 2017 (UTC)
I found it when one of the authors from it popped as "no language" and "has more than 3 works" (I am systematically exterminating these when they appear) and there was a duplicate in the 4 stories they had. Haven't got to looking at who had added it or why yet. And now you mentioning the name, I saw it somewhere else today as well... And I need to sort out what is a poem and what a story because there are a few inconsistencies - which will require a message to the PV :) Annie 23:58, 3 March 2017 (UTC)

Record numbers on award pages

Award, award category and award type pages have been modified to display ISFDB record numbers. The availability of the "[Edit]" links matches what is displayed in the navigation bar on the left: only moderators can edit award categories and award types. Ahasuerus 20:54, 3 March 2017 (UTC)

Advanced Search - "NOT" changes

Based on the recent discussion of Advanced Search, I have added "is not exactly" and "does not contain" to the drop-down list. We'll see whether it approximates the requested "AND NOT" functionality.

Based on my testing, searches complete reasonably quickly if the results are manageable. For example, an Advanced Title Search on "Author's Name contain heinlein" AND "Title Year does not contain 19" generates a list of 21st century titles by Heinlein. If, however, you were to change the "AND" to an "OR" in this search, it would take well over 30 seconds to retrieve the data. Of course, the results wouldn't be particularly useful anyway.

The good news is that lengthy searches don't seem to affect other users (much) because our server has had multiple processors since January 2016. Ahasuerus 22:03, 3 March 2017 (UTC)

That is awesome. Thanks for implementing it :) Annie 22:42, 3 March 2017 (UTC)
That's great! --Vasha 23:31, 3 March 2017 (UTC)
Thanks!--Rkihara 02:28, 5 March 2017 (UTC)

Length/title type mismatches

Edit Title has been modified to prevent users from entering "Length" values for anything other than SHORTFICTION. New/Edit/Add/ClonePub is currently in progress. Ahasuerus 20:07, 4 March 2017 (UTC)

Done. If you find a title type/length mismatch, please let me know. Ahasuerus 21:32, 4 March 2017 (UTC)

Enchanced CHAPBOOK and OMNIBUS validation

Edit Title has been enhanced to perform additional pop-up validation for CHAPBOOK and OMNIBUS titles. You will get an error message if you enter series or synopsis information for a CHAPBOOK title. You will also get an error message if you enter a Content value for a non-OMNIBUS title. Also, Content values can't start with a slash and have to be less than 32 characters long. Ahasuerus 01:22, 5 March 2017 (UTC)

"Add New Omnibus" has been enhanced to use the same pop-ups as Edit Title. Ahasuerus 17:24, 5 March 2017 (UTC)

Date mismatch warnings

Import/Export submissions now show a warning when you try to import a title whose date is later than the publication date. I hope to add a similar warning to New/Edit/Add/Clone submissions tomorrow. Ahasuerus 23:11, 5 March 2017 (UTC)

ClonePub done. Also, EditPub has been modified to display date warnings for COVERART titles. It's a messy area and I need to tread careful lest I break something. Ahasuerus 23:13, 6 March 2017 (UTC)
Regular title warnings added to EditPub. Reviews and interviews are still outstanding. Ahasuerus 00:05, 7 March 2017 (UTC)
EditPub is done. AddPubs still need to be reviewed. Ahasuerus 20:51, 7 March 2017 (UTC)
NewPubs and AddPubs are done. At this point all Import/Export/NewPub/EditPub/ClonePub/AddPub submissions should warn you if a Contents title has a later date than the publication date. If you see anything odd, please post your findings here. Ahasuerus 21:44, 7 March 2017 (UTC)

Magazines and e-books

Under the current way we record magazines, there is no way to add an e-book version of the same magazine because it will show up on the grid as a duplicate. As a result we have some magazines that just show the duplicates, some where there are two separate series (for what is essentially the same magazine - causing the records to drift from each other when they are not updated together) or just not to have the e-versions recorded. The last option is what we have for the big magazines - which is in contrast to the books records where we are recording down to printings.

I know that some of you don't even want to think of the e-books but they are a reality and are here to stay. So anyone with ideas how to solve that little conundrum we are having? I do not think that keeping them in separate series really make sense (the drifting of content is going to continue), so we need other options. Moving the grid on the title level will require untangling the year records we have now... maybe allowing some intelligence in the grid where an e-book is shown only if there is no printed record and the printed record have links to the e-copy?. Or even allowing a checkbox to "not show in the grid" which suppresses the second copy from clustering but allows it to be available for search (a link will still be needed though - because we won't have the title to make it visible). Annie 16:07, 6 March 2017 (UTC)

Ha, the ebook coterie rears its ugly head again ;-). In this case, I'd prefer to have a different title series for the ebook (as we do with Clarkesworld) this won't clutter the grids, will allow as much drift as possible and will kept the problems specific to each format (e.g. if a text is deleted from an ebook by the publisher as happened IIRC with amazon) separate. We may even use the sub-series concept to link the print and the electronic version to the same parent. Hauck 16:31, 6 March 2017 (UTC)
Well, Analog and Asimov's have the same content so we need a way to make sure that if we keep them separate, changes are applied in both copies. Amazon does not delete stories from the middle of an e-book they did not publish :)Annie 16:45, 6 March 2017 (UTC)
IMHO it's an heavy trend (I don't know if it's the correct english idiomatism, we speak of "tendance lourde") to see a divergence in time between print and electronic as publishers make use of all the possibilities of the latter in term of improved (enhanced) content (it's already happening in other domains, e.g. for this magazine that I follow that has more content in the numeric version). So I bet that we will have to keep ASF or Analog separate at some point in the future. Hauck 17:19, 6 March 2017 (UTC)
True. Although I doubt that the pure text magazines will go that way anytime soon. I guess we will end up having two separate series, technically with the same content (and with links set by whoever enters the e-version) to ensure that they are linked. Or something like that. My main concern is data integrity - mistakes happen and when content is imported get populated and they won't get fixed if the magazines are not connected. Annie 18:27, 6 March 2017 (UTC)
This exactly the situation where moving magazines to be pub series over title series would help (I am not suggesting getting rid of magazine editorial records and series but just that magazines not be defined by them as they currently are with seriesgrid: "Issue Grid", etc.). It would also separate out all the magazine publications on publishers pages (which currently get lumped into pubs_not_in_series: "Publications not in a Publication Series for Publisher"). Defining magazines as pub series would allow for separate magazine runs while sharing the same editorial credits (regardless of possible content changes between runs). Uzume 16:30, 12 March 2017 (UTC)
Let me make sure that I understand. It sounds like you are suggesting that we:
  • Keep the EDITOR titles and the currently existing EDITOR series
  • Continue using EDITOR series to organize magazine records on Summary pages
  • For each MAGAZINE/FANZINE publication, enter the name of its title series in the Publication Series field (the original data population effort can be automated)
  • If a MAGAZINE/FANZINE issue had multiple editions, e.g. there was a electronic version and a print version or there were 2+ different versions published in different countries, disambiguate the publication series name
  • Rewrite the Series Grid software to use the newly created Publication Series data instead of EDITOR records
Is this about right? Ahasuerus 17:20, 12 March 2017 (UTC)
That's wonderful, I love it! --Vasha 20:05, 12 March 2017 (UTC)
Assuming that I interpreted Uzume's proposal correctly, my first concern is about data duplication. Every time the same data is recorded in two different places, it inevitably begins to diverge. Ahasuerus 01:06, 13 March 2017 (UTC)

(unindent) Yes, exactly. If we do not employ some automated mass migration magic (as you suggest), we would need an interim solution until everything is converted over. This would then allow editorial records to be later merged (we do not really need multiple records for multiple print runs unless there is some change to the editorial between runs which in most cases does not occur). For instance, once Clarkesworld (print issues) and Clarkesworld Magazine are converted to pub series, Clarkesworld (print issues) and Clarkesworld Magazine could be merged. Neil Clarke (and others earlier on) would still get appropriate editorial credits but the magazine runs would show up on Clarkesworld Magazine and Wyrm Publishing since they are now a pub series entity. There are some caveats however. For example, currently we allow titles series to link to other title series but this is not true of pub series. It would be complicated to create an automated mass migration of things currently organized like Vargo Statten British Science/Space Fiction Magazine (which is an series/issue grid of a title series with no EDITOR records but other title series with EDITOR records). Other differences to note are that pub series have transliterated names available and titles series to not (probably not an issue for this migration concept). Also currently seriesgrid: "Issue Grid" can be applied to other title series like Gor without much value, however, a pub series based series/issue grid could be usefully applied to pub series like OPTA - Aventures fantastiques. Pub series contain pubs and title series contain titles and as such this conversion would allow each pub in the pub series to also have a "Pub Series #" that could correspond to its issue number (I am not sure of a good way to automate such in a mass migration strategy). We would likely also want to change several other things like magazine searches and directories to find the pub series and not the editorial series, etc. The point is we sort of want it to diverge as they represent different things (the editorial of a magazine is related to but not the same as the magazine and its run itself). Uzume 01:14, 13 March 2017 (UTC)

As for french (and generally european publications series) I don't see the added value of putting them in a grid (vs our present system that allow a view either by chronogical or "numerical" order). Such a publication series (which is not that "special") would give a quite crowded (and IMHO unreadble grid) with #373 appearing at 15 points on the grid. The same can also be said of this american one. Note also that, because of our present knowledge and some publisher's practices, the "no-month" cell of the grid my be a bit overcrowded. Hauck 07:38, 13 March 2017 (UTC)

Updating Policy re: dates of birth

With identity theft becoming more of a concern and data deletion requests becoming more frequent, we need to update ISFDB:Policy re: living authors' dates of birth. Here is the latest major discussion of data deletion requests and other issues related to dates of birth. For simplicity's sake, I suggest that we use the same standard that Wikipedia uses:

If the subject complains about the inclusion of the date of birth, or the person is borderline notable, err on the side of caution and simply list the year.

We can eliminate "the person is borderline notable" part since we don't use notability as an inclusion criterion, but otherwise it seems to be a good fit. Also, since Wikipedia and the ISFDB server are currently subject to the same national laws, it's likely that we'll find out about any legal challenge to Wikipedia's policy before we face a similar challenge.

Based on these considerations and the linked Rules and Standards discussion, I plan to update the Policy page as follows:

If a person complains about the inclusion of the person's date of birth and provides evidence that he or she is the listed person, the ISFDB will replace the date of birth with the year of birth.

I am not very happy with the "evidence" part of the proposed language, but I see no other way to prevent vandalism and frivolous requests. Ahasuerus 16:09, 6 March 2017 (UTC)

OK for me for an american site, if we were based in France the total deletion of personal data on request is in the law. Hauck 16:23, 6 March 2017 (UTC)
Yes, "the right to be forgotten", which exists, in various forms, in a number of countries. If we ever move the server to another jurisdiction, it will be one of the things that we will need to consider carefully. Ahasuerus 16:54, 6 March 2017 (UTC)
That's why what was done without discussion and without my consent on the Moderator Availability's list (adding our complete names) is here considered very bad manners and may be downright illegal. Hauck 16:23, 6 March 2017 (UTC)
I was surprised to see the changes even though I think that they were well-meaning. Moderators generally add their own names to the Moderator Availability template and it's up to them to decide how to list their names. Ahasuerus 16:58, 6 March 2017 (UTC)
Are you talking about my recent changes to Template:Moderator-availability (as per 466661)? You will note, though I rearranged the names (making them more noticeable), I did not add any (most were in the original creation by Nihonjoe on 2015-06-04 as per 392440; some were added later with later moderator changes/adds). Uzume 00:21, 13 March 2017 (UTC)
I was referring to the addition of first names and last names, e.g. "Chris J" became "Chris Jensen" and "Dwarzel" became "Desmond Warzel". Of course, their names can be easily found on other Wiki pages or in the database itself, but it's generally up to each moderator to decide how they want their user name listed. Ahasuerus 01:02, 13 March 2017 (UTC)
I noticed Mhhutchins and Hauck recently pulled their names out but I did not add those, however, it seems you are right, a few did somehow sneak in on that edit (I did not think I pulled those in but as you say they are directly linked from their user pages which are linked in the template). My apologies. Uzume 01:45, 13 March 2017 (UTC)
I'm fine with codifying a mechanism for the removal of DOBs, but I agree the evidence part will be tricky; we will probably also need to have some guidelines as to what we will consider acceptable as evidence. Albinoflea 16:27, 6 March 2017 (UTC)
A couple of years ago an author registered as an editor and asked to have her DOB reduced to YOB. I said I would do that if she could prove who she was and that I would send an email to her website for confirmation of identity. No response, but she made the edit as an editor. I decided to let it go through.--Rkihara 18:49, 6 March 2017 (UTC)
In the case I have at hand, it's the YEAR that's private, and the month and day are publicly available, so the proposed wording wouldn't cover the desired/"needed" change.
I guess I'm more inclined to France's approach as a default: We should agree to remove any private or quasi-private identifying information upon the individual's request. A possible private-vs-public bar could be "can[not] be found via Google search". Since we have no way to prevent anyone from adding it again, and we have no way to record that the information is less-than-full information is recorded at the author's request, we would have to be clear that there's no guarantee of its staying that way (and anything public/published is almost certain to come back, which would be justification for not wasting time removing it, beyond the fact that removing it isn't doing anything to enhance information security). --MartyD 19:30, 6 March 2017 (UTC)
Keep in mind that once we finish migrating Wiki-based publisher and series/magazine pages to the database, we will add a "Note" field to author records. Once it's available, we will be able to add notes like "Date of birth removed as per author's request", which should help prevent DOBs from reappearing. (The {{BREAK}} syntax will come in handy.)
As far as this particular case goes -- the year is private while the month and day are public -- I suspect that accommodating it would quickly lead to, as you wrote, having to remove any biographical information upon request. We'll have to revisit this issue when we move our "Bibliographic" and "Bio" pages to the database, but for now I am not ready to go that far. Ahasuerus 20:02, 6 March 2017 (UTC)
P.S. As far as defining "publicly available sources" goes, it's an interesting line. Wikipedia's policy says:
  • Do not use trial transcripts and other court records, or other public documents, to support assertions about a living person. Do not use public records that include personal details, such as date of birth, home value, traffic citations, vehicle registrations, and home or business addresses.
Which, as far as I can tell, would pretty much limit us to Google and other search engines if we were to follow it. And that's not necessarily a bad thing, although there is a possible wrinkle: some search engines serve different results depending on the user's location. Ahasuerus 20:12, 6 March 2017 (UTC)
Would it be possible to add a switch (basically, a checkbox next to the field when in the editing screen) that turns off the display for that information until a date of death is entered? It could be there for date of birth, location of birth, and legal name, since those are likely the fields someone might be concerned about. That way, the information could still be in the database, but it wouldn't appear to the public unless they were no longer living. ···日本穣 · 投稿 · Talk to Nihonjoe 20:35, 6 March 2017 (UTC)
Is it really worth the effort if the information is easily and freely available on the internet? Maybe we could avoid the issue by posting the links, or citing the references giving POB, DOB, etc., instead of listing it explicitly for living authors? Then the authors can petition the root sources.--Rkihara 21:58, 6 March 2017 (UTC)
I am all for citing our sources once we have the Author Note field. Take the submission that prompted this discussion. The author in question stated that her full DOB and place of birth are not publicly available and sure enough, googling doesn't seem to find anything. So where did our data come from? It may have been a print source like Contemporary Authors or it may have come from some database that us not indexed by Google. It's hard to tell without a cited source.
As far as leaving the field blank and linking to the source goes, I suspect that it would make our data much harder to work with. URLs change all the time even when the underlying data remains the same. Ahasuerus 23:20, 6 March 2017 (UTC)
If it's the submission that MartyD has on hold, I posted the DOB and POB (I keep a short record of authors that I enter DOB and DOD for). The source was Contemporary Authors Online, which lists education, hobbies, and career hightlights.--Rkihara 00:11, 7 March 2017 (UTC)
That's a good example. Contemporary Authors Online is an established biobibliographical source which is used by Wikipedia (see this example) and other sites, but, as far as I know, it's not available without a subscription. I believe its data is made available to educational institutions and subscribers. On the other hand, they offer a free trial, so if someone is interested in a particular author's biographical data, all they have to do is set up a free trial account and look things up. So... Can we say that its data is publicly available? Ahasuerus 02:06, 7 March 2017 (UTC)
I would say that CA is so widely cited and available that it can be said to be public knowledge. It is by subscription, but I have access through several channels, including my local library, which also carries the hardcover edition, and other biographical sources.--Rkihara 04:31, 7 March 2017 (UTC)
Thanks for clarifying, I am sold on the notion that the CA data is "publicly available". Ahasuerus 16:02, 7 March 2017 (UTC)
We could build in the ability to archive online sources (using Web Citation or another archive service). I use this all the time when writing articles on Wikipedia to avoid link rot. ···日本穣 · 投稿 · Talk to Nihonjoe 23:58, 6 March 2017 (UTC)
Looks interesting. Do they let Web site owners delete copies of certain pages the way the Internet Archive does? Ahasuerus 16:42, 7 March 2017 (UTC)
They have this policy. ···日本穣 · 投稿 · Talk to Nihonjoe 19:51, 8 March 2017 (UTC)
I see. It sounds like they too have to worry about dealing with various cans of worms. Ahasuerus 22:18, 8 March 2017 (UTC)

(after sleeping on it) I guess we have a few separate issues here.

The first one is how we want to define the "publicly available sources" which we are permitted to use. I think it would be prudent to explicitly exclude government (courts, taxes, immigration, vehicle registration, etc.) records for living persons. We can let other projects take care of the headaches that they cause. For example, just recently the SFE3 editors spent a fair amount of time exchanging e-mails, including scans of immigration records, with a certain SF writer who had disputed the year of birth listed by SFE3. We bowed out of that dispute by saying that we'd quote whatever SFE3 settled on.

The second issue has to do with documenting our sources of biographical data as well as author requests for data deletion. The plan has always to add a Note field to Author records and then to migrate Wiki-based Bio and Bibliographic pages to the Note field. (We still need to decide whether we want to have a separate "Bio" field as well.) However, the original plan was to wait for the publisher/series/magazine Wiki cleanup project to be completed before adding a Note field. Given the fact that we still have 500+ Wiki pages to migrate and the need to support the ability to document the sources of biographical data, I think we should implement the Author Note field sooner rather than later.

The third issue has to do with accommodating author requests to remove otherwise eligible data. Earlier we seemed to agree that we needed to ensure that the requester was the person whose data s/he was asking to remove. However, I think we also want to allow authorized representatives (Web masters, spouses, etc) to make these types of requests. For example, the person in the previously mentioned SFE3 controversy communicated via his Web master.

The fourth issue is what types of biographical information we should be willing to remove upon request. At this point we have seen requests to delete:

  • the year, month and date of birth
  • the month and date of birth, but not the year of birth
  • the year of birth, but not the month and the date of birth
  • the exact place of birth, but not the country of birth

Should we use the Wikipedia standard ("If the subject complains ... list the year") or should we be more accommodating? What about requests to remove legal names? Ahasuerus 18:09, 7 March 2017 (UTC)

I think that issue #4 could be handled by removing the entire date and giving the online reference which contained the information. The author can hardly object to that, and it redirects the author’s attention to where it belongs - as we're just the messengers.
I would also suggest adding the following statement in bold at the bottom of biographical info, “Some personal information has been redacted at the request of the author or the author’s representatives.” This will also warn off editors who might re-enter the redacted data.--Rkihara 18:30, 7 March 2017 (UTC)
Well, we already support the {{BREAK}} template in Note fields. Once we have the previously mentioned Note field, we could add support for a {{REDACTED_PERSONAL_INFO}} template, which would display the message that you proposed. Ahasuerus 19:01, 7 March 2017 (UTC)
The author may WANT their date of birth to be private, but as we all know with varying degrees of chagrin, once something is on the internet, trying to get rid of it is like trying to mop up a flood with a kleenex. Nonetheless, there's a difference between having the information "out there" and having it on a semi-authoritative site that might be thought of as a reference source -- like Wikipedia, SFE3, or the ISFDB. I can see someone not wanting their info on the latter even when they can't do anything about the former; their info wouldn't be formally public. How sympathetic do we want to be to that? --Vasha 19:31, 7 March 2017 (UTC)
Admittedly it's a fine line. For example, suppose we were to adopt Ron Kihara's proposal immediately above: "issue #4 could be handled by removing the entire date and giving the online reference which contained the information". In the case that prompted this discussion, we would then remove the author's year of birth and add the following note: "Year of birth available in Contemporary Authors. Redacted as per author's request." If someone wanted to steal the author's identity at a later point and googled "[author name] year of birth", Google would helpfully serve our Summary page. The bad guy could then drive to a local library and look things up. Is that something we have to worry about? Where do we draw the line? Ahasuerus 22:26, 8 March 2017 (UTC)
Well, we don't have to specify what was redacted, and it will take more effort to find. If I can find the data (and I spend only a few minutes per), then someone who really wants to find this data wll find it too. I also think if an author wants the year of a birthdate redacted, we should redact it all. It really irritates me to see, e.g., born October, or born October dd, with no birth year. The same with place, but not country of birth.--Rkihara 23:21, 8 March 2017 (UTC)

Omnibus or collection

As I am always getting confused - if we have a book containing 1 novel and 3 stories, is that an omnibus (as it contains a novel) or collection (because the 3 stories are not published independently)? In my mind, a novel in the content makes in an omnibus but our conventions are a bit confusing. Annie 22:30, 7 March 2017 (UTC)

Probably collection. See the OMNIBUS bullet of Help:Screen:NewPub#Publication_Type and its specific Heinlein example. This may be pure Male Answer Syndrome, but I believe the intention behind OMNIBUS is that it collects multiple works previously published standalone (so short stories that appeared only as part of some published collection or in a magazine don't qualify). --MartyD 01:43, 8 March 2017 (UTC)
These days any story can be published on its own - making this a collection of two published works (which we call an omnibus as one of them is a novel). If we want to say that we need at least 2 novels/collections/anthologies/non-fiction (we say at least 1 now) to declare it an omnibus - I am fine with that and I think that was the intent behind this definition. But this is why I asked. Annie 02:57, 8 March 2017 (UTC)

Author Note and Author Bio: one field or two fields?

As per FR 307 and the earlier discussion of biographical data, we plan to migrate all of our "Bio" and "Author" pages from the Wiki to the database proper. Changing the software to support a new field is not hard, I can do it in a few hours. However, first we need to decide whether we want to add:

  • one new field, basically a generic "Note" field similar to all the other Note fields in the database, or
  • two fields: one for general purpose bibliographic notes and another for biographical data, which would match the current Wiki layout

It stands to reason that all Wiki-based "Author" pages will be migrated to the new "Note" field. The real challenge is deciding whether the contents of Wiki-based "Bio" pages can be accommodated by the "Note" field as well. Based on a cursory review of the "Bio" pages, we'll be dealing with the following scenarios:

  • Information about the source of our author data, e.g. "Middle name from Locus". Sometimes it can get complicated, e.g. Bio:A. V. Clarke, and require the use of the {{BREAK}} template. This data will always go into the Note field even if we decide to have a separate Bio field.
  • Additional information about pseudonyms like "A Ziff-Davis house name". Will be migrated to the Note field.
  • Basic biographical information, e.g. "Born Andrei Perlmutter" or "Poul Anderson's son-in-law" or "Vernor Vinge's wife in 1972–1979". Can be easily accommodated by the Note field.
  • More detailed biographical articles like Bio:Irene Maude Swatridge. They can be accommodated by the Note field, but they may require the use of the {{BREAK}} template.
  • Really detailed bio pages, usually added by the author, e.g. Bio:Charles G. Waugh. They are the exception rather than the rule, but they do exist. The {{BREAK}} template will effectively move them to a separate Web page, so I think we can put them in the Note field without causing a problem, but I can see how having a separate field may be useful in certain cases.

Given the above, I am 95% sure that we can get away with having just one Note field. Does this sound reasonable? Ahasuerus 00:09, 8 March 2017 (UTC)

I do not think that there is a clear line between bibliographic and biographic note in the case of authors -- does a note on pseudonyms belong to the bibliographic part or is it part of the biography for example? I vote for one note field only. Annie 00:27, 8 March 2017 (UTC)
I agree one Note field seems adequate. --MartyD 01:45, 8 March 2017 (UTC)
To be frank, I've always found that the "biographical" part falls largely outside our scope. IMHO we're a bibliographical tool and not a place for self-promotion or egoboost and so only the basics should be listed (the existing fields are sufficient for me). As a frequent moderator-of-new-contributors, the "biographical" pages are where there is the more "pruning" to do. Hauck 07:25, 8 March 2017 (UTC)
I agree that author-provided biographies can get out of hand, but first a little bit of background. Back in the 1990s, during the "ISFDB 1.0" era, i.e. pre-moderation, there was an "Author Note" field. It had all kinds of data, much of it not very good. It was a learning experience.
When we started work on ISFDB 2.0, the plan was to rely on Wikipedia to house biographical data. However, we quickly discovered that Wikipedia's "notability" criteria were too restrictive for our purposes: we have tens of thousands of authors who do not have (and never will have) a Wikipedia article. That's when the idea to use our Wiki to store author bios for less notable authors was born. Eventually it led to other issues, which is why the Wiki migration project was started a couple of years ago. Ahasuerus 15:32, 8 March 2017 (UTC)
P.S. The original 589 "author notes" from the 1990s and the early 2000s are still stored in the database although you can't access them. Many of them can be safely dropped, e.g. the following note about L. E. Modesitt, Jr.: "Author of over 20 fantasy and SF books, including the best selling Magic of Recluce series". However, there are a number of more useful ones, e.g. "Son of Fritz Leiber", "Married to SF author Charles Sheffield (1998-2002)", etc. Once the new field becomes available, I will create a cleanup report which will facilitate migrating the relevant ones. Ahasuerus 16:11, 8 March 2017 (UTC)
That is not very different from the other wiki pages cleanup - there is a lot of outdated information in them as well (and a lot of irrelevant one that is superseded by adding books and similar actions. Annie 16:14, 8 March 2017 (UTC)
True, although some older "author notes" are really strange. For example, here is the internal note for Edgar Pangborn: "None yet.<br><b>Sources:</b> TEoSF.". Make of it what you will :-)
Another difference is that we will be migrating three different types of author notes:
  • 589 database-resident notes mentioned above
  • a couple thousand Wiki-resident pages in the "Author" and "Author talk" namespaces
  • a couple thousand Wiki-resident pages in the "Bio" and "Bio talk" namespaces
Ahasuerus 16:29, 8 March 2017 (UTC)
That sounds like a fun project. I've seen stranger in some of the pages I cleaned - some of the 2008 notes were... interesting. :) Annie 16:39, 8 March 2017 (UTC)

(unindent) OK, "Author Note" is now available. Here is what A. V. Clarke's Summary page looks like now. It demonstrates the {{BREAK}} functionality, which can be used in a variety of ways. Ahasuerus 22:04, 8 March 2017 (UTC)

The changes make some pages look pretty stark, for example Bill Thompson's page. It would look better if the author name was in a larger font and bold, and "author:" removed, since it's redundant.--Rkihara 23:26, 8 March 2017 (UTC)
Is the starkness caused by the removal of the "Biography" and "Bibliographic Comments" lines for authors who do not have Wiki-based comments? As far as the author name goes, we currently display it twice: once in the header after the words "Summary Bibliography" and then again on the "Author:" line. It's done the same way on all bibliographic pages, including Title, Publication, Publisher, and Series pages. Ahasuerus 23:46, 8 March 2017 (UTC)
That's basically it. If no language is defined, there's just the author name in small font on one line. Maybe replacing the author line with the header, "Summary Bibliography: John Doe" in the large font would help the appearance? I think it's the white background drawing my attention away from the header on the black background.--Rkihara 00:48, 9 March 2017 (UTC)
Keep in mind that the Bio/Biblio links will be removed from all the other pages Real Soon Now (tm), so this issue will affect more than just the Summary page. For example, consider this series. Once the "Bibliographic Comments" line disappears, the header will consist of just one line. Ditto this publisher etc.
I am going to go ahead and remove these links from all series/magazine/publisher pages that do not have associated Wiki data. Once I do that, we will have a better idea of what the pages look like and will better position to decide how to change their layout. Ahasuerus 01:04, 9 March 2017 (UTC)
Done. I am going to create another section to discuss the layout implications. Ahasuerus 01:42, 9 March 2017 (UTC)
Done. Ahasuerus 01:54, 9 March 2017 (UTC)
And when are we getting a report? :) Annie 22:16, 8 March 2017 (UTC)
Real Soon (tm) :-) Ahasuerus 23:46, 8 March 2017 (UTC)

New cleanup report: "Author Notes to be Migrated from ISFDB 1.0"

"Author Notes to be Migrated from ISFDB 1.0" is now available in the main Cleanup Reports menu.

Please note that the data in the "Old Note" column comes from ISFDB 1.0 (1995-2004) and is not accessible in any other way. The data in the "New Note" column comes from the newly added "Author Note" field.

The report is unusual in a couple of ways:

  • The data is not generated nightly. Every time you access this report, you get a fresh copy.
  • There is no way to "ignore" author records or to remove them from the report. Once the useful tidbits (and there are some) have been migrated from the "Old Note" column to the "New Note" column, the old notes will be deleted from the database and this report will be deactivated.

Ahasuerus 00:58, 9 March 2017 (UTC)

Changes to database-Wiki links

The way Wiki links are displayed on the database side has been changed as part of the Wiki migration project. In the past the links were displayed regardless of whether a matching Wiki page existed. As of 15 minutes ago, the links are only displayed if a matching Wiki page exists. (Once the Wiki migration project has been completed, direct links to existing Wiki pages will be removed and we will use "Web pages" fields to link to the Wiki exclusively.)

Now that 99% of all links are gone, we need to decide whether the resulting layout looks good. For example, here is what the Summary page for an author without a language looks like. Here is an author with a language and no other data. Here is a publisher with a single publication. Do the pages look too sparse/stark? Ahasuerus 01:53, 9 March 2017 (UTC)

I like the wiki links being hidden, but I don't like hiding the empty database fields. I think that could discourages editing for a newer user who may have information, but isn't even aware that the fields are available. It also makes the page look pretty sparse. -- JLaTondre (talk) 02:16, 9 March 2017 (UTC)
Contra JLaTondre, I think it would look terrible to have a list of fields reading "unknown" on every page; and I don't think that it would encourage that much extra editing.
However, the name of the author/publisher/whatever should definitely be larger or highlighted so it stands out from the other elements of the page. --Vasha 02:21, 9 March 2017 (UTC)
I agree with JLaTondre's remarks. I don't like this "new" look (that's personal, of course) and it doesn't give a hint of all what's possible for our contributors. As I understand it, the "ISFDB-wiki" links will be among the web links listed, so the understanding that these specific links are "ours" and so are editable may be a bit complicated for the average user. Hauck 07:16, 9 March 2017 (UTC)
Most of the existing wiki-pages will be converted to the DB field - the only ones that will remain as wiki pages will be the ones that are too long or complicated to move (and I suspect even some of them will get moved sooner or later). Annie 07:32, 9 March 2017 (UTC)
Let me make sure that we are all on the same page. Bibliographic (Summary, Series, Publication, etc) pages display two different types of data. There are "database fields" and "Wiki links". What JLaTondre is proposing is that we display empty database fields. Here is what the previously linked Summary page would look like if we were to display the empty fields:
Author: Kathleen Notestine
Transliterated Name: 
Legal Name: 
Transliterated Legal Name: 
Birthplace: 
Birthdate: 
Deathdate: 
Language: 
Email Address:
Webpages:
Note:
It would be easy to implement, but I don't think it would look better than what we have now. Ahasuerus 14:24, 9 March 2017 (UTC)
As I mentioned previously, the missing fields make it look too stark, somehow deprecated. I can't put into words exactly what it is that bothers me about it. Removing "author:," centering the name at the top of the field in a larger, bold font size, and removing the header "Summary Bibliography:" would do a lot to improve the appearance, IMHO.--Rkihara 16:33, 9 March 2017 (UTC)
One thing that comes to mind is that we use HTML bullet ("unordered") lists to offset "Author", "Language", etc fields. A bullet list consisting of one bullet looks odd. In addition, the offset makes these fields align with the contents areas of other sections like "Short Fiction" or "Poems", which also isn't quite right.
There are a couple of things that we could do about this. First, we could move the "Author Record #" field, which is currently right-justified, to the main bullet list immediately under the Author Name field. That way the list will always have at least 2 bullets. Second, we could move the list to the left so that it would be aligned with section names rather than with section contents. Ahasuerus 17:01, 9 March 2017 (UTC)
That would help. After thinking about it a bit, I think that without missing field headings, what remains doesn't give you anything to focus on. The white background draws my eye in, then it jumps to the line below Showing all translations (can be changed in My Preferences) because it is bold and colored blue at the end. Then my eye wanders around to the author: line above, but it keeps getting distracted by the line below. It's like an outline with plain headers and subs in bold. That's why I think the author should be centered in bold, large font. I have a friend who is a graphics designer and deals with this kind of problem in his work. I'll ask him for his opinion.--Rkihara 17:24, 9 March 2017 (UTC)
Why not change "Author:" to "Canonical Name:" like in the edit form (we already have the author name above in the "Summary Bibliography:" and put the list of works, etc. in a different "Works" (or similarly named) box much like the "Contents" box of publications (which have the publication data in one box and the "Contents", etc. in a separate box below it). It seems to me, this type of delineation would also be useful in other places like title displays (where title data could be in a separate box from the list of publications of such title, etc.). This would help underscore which data belongs to the author record and which were just associated with it. Uzume 19:06, 12 March 2017 (UTC)
I am afraid I don't understand what you are proposing. Ahasuerus 21:21, 12 March 2017 (UTC)
I am proposing to put the list of credited titles on author pages into a separate box clearly delineating which is author data and which is author associated data. For example P348699 currently displays the publication data in a box and the associated contents in a separate "Contents" box below it. Do the same thing with author's works in a "Works" box on author pages (and relabel "Author:" as "Canonical Name:" as it is labelled in the author edit form). I am not suggesting (re)adding empty fields to the display (though that could be done as well). To get technical, right now all author data and associated author data on author pages are in <div id="main">. Publication data are in <div id="content"> and associated publication data are in <div id="ContentBox"> (and <div id="VerificationBox">, etc.). With proper styling, this will clearly separate what can be edited in the author record and what cannot directly be edited there. Uzume 21:50, 12 March 2017 (UTC)
Oh, I see. Thanks for the explanation! Come to think of it, we already have a line ("<hr>") which is supposed to separate author-specific data from the author's titles, e.g. see Poul Anderson's Summary page. However, it is only displayed if we have an author image on file. Let me see if I can make it appear for all authors... Ahasuerus 22:19, 12 March 2017 (UTC)

(unindent) The software has been modified to display a thin gray line between the author-specific data and the list of author's titles on all pages. In the past the line was only displayed if the author had an image on file. Ahasuerus 22:33, 12 March 2017 (UTC)

Yes, something like that (though I still like the separate box delineation better; I notice a similar hr rule is used for non-genre). That does make Kathleen Notestine and A. Testa (the sample authors mentioned in this thread) look a little better. I would still recommend changing the label "Author:" to "Canonical Name:" to match the edit form. Also a similar delineation would be good for publishers (as mention above for Barzakh) and as I mentioned for titles too. Uzume 23:36, 12 March 2017 (UTC)
Well, "Canonical Name" is, as Help states, is "the name under which a particular author's bibliography is organized". For pseudonyms, the displayed name is not the same as the author's canonical name. I agree that synchronizing the field label on the Summary page with the field name on the "Author Editor" page would be good, but I am thinking that we may want to change the latter rather than the former. Ahasuerus 00:12, 13 March 2017 (UTC)
I agree that makes that wording confusing. Perhaps "Author Name:" is better and the Help could be updated to define "Author Name" as "the name under which a title/work is credited" and "Canonical Author Name" could remain as "the name under which a particular author's bibliography is organized" (even though we do not have anything where that is used except perhaps in pseudonyms and variants where we could distinguish "Credited Author Name" and "Canonical Author Name"). Uzume 01:36, 13 March 2017 (UTC)

The Illusion Seekers

Something does look weird here. The Illusion Seekers. Why is this story credited to unknown and then varianted for the proper author? If the reason is because P. F. Costello is a pseudonym then shouldn't instead it go to whoever is pseudonymed for? As it is - it is extremely hard to get a listing for "P. F. Costello" (the show all titles format is awkward and unwieldy for things like that) and this specific story only shows on the unknown list - which is unmanageable.

Is this by design? If so, why? Annie 18:45, 9 March 2017 (UTC)

One more like that The Monarch of Mars - seems to be my day to stumble on them. Some background will really help. Annie 18:53, 9 March 2017 (UTC)
"P. F. Costello" and "John Pollard" were house names owned by Ziff-Davis. Z-D used a lot of house names, most famously "Alexander Blade". Sometimes we know the actual author, other times we don't. We start by creating a title record with the house name listed as the author. Then we variant it to the actual author (if known) or to "unknown" (if unknown.) If and when we discover the actual author, we update the canonical title.
So yes, it's done by design. However, I agree that the format used by "view all titles by this pseudonym" is not perfect. Suggestions welcome! Ahasuerus 19:19, 9 March 2017 (UTC)
Someone coming in the site for the first time will be very very confused. Knowing that these are Z-D house names, can't we have a new author for all Z-D house names and not alias under unknown - which looks just weird... At least this way you can see that it went there because it is a House name. Annie 19:25, 9 March 2017 (UTC)
Well, it's a thought, but let's consider the implications. What about Daisy Meadows, a 21st century house name with hundreds of books to "her" name?
I guess one way to approach this issue would be to create a new author record for "undisclosed house name" and variant all undisclosed titles by "Alexander Blade", "Daisy Meadows", etc to it. Ahasuerus 22:02, 9 March 2017 (UTC)
But Daisy Meadows is not pseudonymed under unknown. I do not have issues with her display - all she needs is an author note (already there - just need moving) and all is set. It is the ones that ended up into unknown (because some books are really unclear) that are the problem. And yes - having a "undisclosed house name" will be preferable to the current "unknown". Annie 22:16, 9 March 2017 (UTC)
Keep in mind that pseudonyms and VTs are handled separately. There are many Daisy Meadows titles whose author we do not know -- e.g. this one -- and whose canonical author is currently "unknown". In this respect "her" bibliography is similar to what has been done with the Ziff-Davis pseudonyms. The difference is that there is no pseudonymous relationship between "Daisy Meadows" and "unknown".
We could certainly remove the "unknown" pseudonym from the Ziff-Davis pseudonyms to match what has been done with Daisy Meadows, but it could have unfortunate side effects. For example, consider John Dexter, a house name used by Greenleaf Publications in the 1960s. At the moment "his" Summary page reads "Pseudonym. See: Marion Zimmer Bradley", which makes it appear like it was an MZB pseudonym. A few weeks ago I received an e-mail from an MIT librarian who wanted to add another "John Dexter" title to the ISFDB and to his catalog. It was our data that had suggested to him that the book was written by MZB.
Thankfully, Robert Silverberg was another SF author who wrote as "John Dexter". Once we add his Dexter titles and link everything, it will become clear that "John Dexter" was a house name. However, we are not so lucky in the case of "John Pollard". If we were to remove "unknown" from the list of pseudonyms, it would look like it was a personal pseudonym used by Howard Browne. I guess it will be less of a concern now that we have the Author Note field. Ahasuerus 22:50, 9 March 2017 (UTC)
I understand how we ended up here. It is just a bit weird the first time you see one of those. Thanks for all the background! Annie 23:01, 9 March 2017 (UTC)
We could perhaps implement a new feature where an author is linked to a publisher instead of another author (as is done for pseudonyms; I always thought the design of keeping author and publisher as separate "authorities" was a dangerous one but it was made a long time ago). Then such house names could be unlinked from "unknown" (but known titles could still linked to the proper authors where available) and they could be shown on publisher pages and vice versa. Uzume 19:17, 12 March 2017 (UTC)
I am afraid I don't understand what it means to have an author "linked to a publisher instead of another author". Ahasuerus 21:17, 12 March 2017 (UTC)
Basically a pseudonym of publisher index/pointer (vs. of another author) field (or add a type field so you know which table to index into and reuse the existing index field). Uzume 21:34, 12 March 2017 (UTC)
Before we discuss the implementation details, I would like to understand what it means to have an author "linked to a publisher" functionally. Ahasuerus 21:46, 12 March 2017 (UTC)
It means it is a pseudonym associated with a publisher as house names commonly are. Is there something I am missing (I thought that was what we were talking about in this thread)? Uzume 23:26, 12 March 2017 (UTC)
Authors and publishers are so different that I am having trouble grasping how they could be linked. It's not like an author, a pseudonym or even a house name has to be limited to one publisher. For example, Alexander Blade was a Ziff-Davis pseudonym, but later on "Alexander Blade" stories appeared elsewhere, e.g. in Palmer Publications, Inc. magazines owned and edited by Ray Palmer and in Greenleaf Publishing Company magazines owned and edited by William L. Hamling. Once the stories fell into public domain, they were reprinted by project Gutenberg and a variety of PD publishers.
Then it gets worse. Much worse. To quote SFE3:
  • The use of house names was at its most extreme and most devious at the Ziff-Davis magazines edited by Raymond A Palmer. They have never been satisfactorily unravelled and all records are now believed lost. In these cases most pseudonyms which became house names were originally used by an individual writer but then became farmed out to others, adding to the confusion. Alexander Blade, for example, the most common house name, was originally used by David Vern (see David V Reed) but soon appeared on work by almost all the regular Ziff-Davis contributors, including the serial "The Eye of the World" (June-July 1949 Fantastic Adventures) written by Don Wilcox. Both P F Costello and Gerald Vance were originally the personal pseudonyms of William P McGivern but were later used by Chester H Geier, Rog Phillips and others. E K Jarvis was the alias of Robert Moore Williams until used by others in the 1950s. S M Tenneshaw was the alias of William L Hamling, and he took it with him when he began Imagination, but it has been used by several other writers. Ivar Jorgensen, sometimes misspelled Jorgenson, was originally the pen name of Paul W Fairman, but in later years it was used by Howard Browne, Harlan Ellison, Henry Slesar and Robert Silverberg, the latter usually in collaboration with Randall Garrett. Silverberg once quipped that he started out enjoying the work of Ivar Jorgensen and grew up to be him. Most of the work under these and other Ziff-Davis house names has yet to be accurately attributed. It is believed that Leroy Yerxa wrote almost the entire December 1943 issue of Fantastic Adventures with stories under house names except for one, Morris J Steele, which was usually the pen name of Raymond A Palmer himself but was here used by Berkeley Livingston.
Ahasuerus 00:54, 13 March 2017 (UTC)

8 cleanup reports for authors

8 author-specific cleanup reports have been added for "Bio" and "Author" pages. The rules for their migration are the same as for all the other cleanup reports. The data will become available tomorrow morning. As of last Saturday, we had 1,698 "Author" pages, 2,147 "Bio" pages, 30 Talk pages, and 156 orphan pages. Ahasuerus 21:59, 9 March 2017 (UTC)

Never had so much fun with clean-up rapports!...Dirk P Broer 21:47, 10 March 2017 (UTC)
For anyone that works these for the first time, don't forget after you move the data to add one of the Deletion templates to the wiki page so that the moderators can remove it. The templates are here Annie 21:50, 10 March 2017 (UTC)
Thanks for the info!--Dirk P Broer 22:44, 10 March 2017 (UTC)
For Fanzines and Magazines, We do not delete the wiki pages until at least stub pages are created for ALL issues references in the wiki (especially when all issues are eligible as it is with the fanzines). So I had to reverse your request for a delete on the fanzine. If you feel like creating the stubs - go ahead. But otherwise we lose the information for existing issues that someone collected. Annie 22:54, 10 March 2017 (UTC)
I created a similar category mechanism for marking wiki spam (see Category:Spam) back in March 9, 2011 and I had it linked to the top of the Moderator noticeboard (I am not sure when it was removed; sadly the wiki history gets purged so though I can look at the history of edits I cannot tell what was actually edited when—just the edit comments survive). This mechanism might be partially superseded, however by the newer deletion mechanism more recently created on October 30, 2016 (see ISFDB:Help desk/archives/archive 25#Mixed up series). That said, it is possible to have multiple deletion candidate categories/subcategories for different kinds of deletions requests. I just now added my category to Category:Deletion candidates (directly without using a template). I am not sure if this will be interpreted as a deletion request or allow it to be used as a sort of subcategory for a different type of deletion requests. Uzume 20:00, 12 March 2017 (UTC)

Templates in Notes?

Now that "Author Note" is live, it occurs to me that we may want to add support for Wiki-style templates. For example, if I type {{A|Vernor Vinge}} here, everyone will see Vernor Vinge and be able to follow the link to the ISFDB record. We can add the functionality to the database side if it is deemed desirable. Ditto for publishers. Ahasuerus 23:17, 9 March 2017 (UTC)

That would be very nice :) Annie 23:43, 9 March 2017 (UTC)
OK, template support has been added. At this point the software supports the following three templates in the Note fields:
  • "A", which does what I mentioned earlier, i.e. {{A|Vernor Vinge}} will display a link to Vernor Vinge's Summary page
  • "OCLC", which links to OCLC, e.g. "{{OCLC|880913738}}" will display a link to this OCLC record
  • "LCCN", which links to the Library of Congress, e.g. "{{LCCN|89040470}}" will display a link to this LoC record
We can easily add support for other sites. The only constrains are:
  • They need to have stable URLs
  • They need to allow linking by record number
We may be able to accommodate more complex cases as well, but they will require additional research. ASINs are next on my list. Additional suggestions welcome!
P.S. In case anyone is wondering, the "third party identifiers" that we discussed a couple of weeks ago will use the same logic when implemented. The difference is that using these templates in Note fields allows more flexibility, e.g. "corrected page count from {{OCLC|880913738}}, artist from {{OCLC|880914329}}". On the other hand, a separate field for third party identifiers will support robots like Fixer. Ahasuerus 19:02, 10 March 2017 (UTC)
Can we add a template for Publishers for the Publishers "Notes" field? That will allow the links that connect different publishers (when they move as imprints between publishers and so on) to be spelled in an easier way. Annie 20:01, 10 March 2017 (UTC)
When you have a chance, If you use Scott Thomas here, you get to the page. If you use the same in a note, it gets elsewhere (see my submission here for an example). Is that expected? How can I point to the correct one in this case? Annie 20:16, 10 March 2017 (UTC)
Intra-ISFDB linking templates use the Regular Search logic. If a search string matches only one author name, then the user is redirected to that author's Summary page. If, on the other hand, a search string like "Scott Thomas" matches the names of multiple authors, then the "Search Results" page is displayed.
Granted, it would be easy to change the intra-ISFDB templates to use the Advanced Search logic which supports the "exact" functionality, but there is a catch. An Advanced Search always displays the Advanced Search Results page even it finds only one matching record. That's even worse than the current behavior.
What we need is new logic that would combine the "exact" functionality of the Advanced Search and the automatic redirect logic of the Regular Search. Alternatively, we could change the Advanced Search logic to use the same redirect functionality that Regular Search uses, but I think some editors objected the last time this issue came up.
So expected behavior. No problem - I was not sure if you are aware of it happening. How about allowing to link by number in such cases? Or will that be too much work and not worth it (standard links work after all so one can use that if they wish)? Annie 20:45, 10 March 2017 (UTC)
Let me see if I can tweak the Regular Search logic to accept a new parameter to support "exact" searches... Ahasuerus 22:02, 10 March 2017 (UTC)
Done. "Publisher" has been added as well. Ahasuerus 23:36, 10 March 2017 (UTC)
As far as publishers go, I originally implemented a "P" template, but removed it before installing the patch. The linking problem was even worse: a Regular Search on "Tor" finds hundreds of matching publisher names. Ahasuerus 20:33, 10 March 2017 (UTC)
What is the advantage of adding third party identifiers if we are already implementing them as markup in notes? It seems redundant to me and the only difference is whether they adhere to a structured data relational model or are semi-structured data like those of document-oriented databases (or object-relational databases). Both are searchable but semi-structured data may be better supported with a different query language (e.g., XQuery, XPath or SPARQL, etc. vs. SQL). The one downside to this new markup is we are effectively creating our own template processor, macro or lightweight markup language (which can be nontrivial) and we might be better served to use an existing one (which could solve other problems too like restricting the types of markup available instead of making raw HTML available which can be unsafe e.g., if I can get it past moderation which might not be hard since it has no real display, I could add a script tag with JavaScript to note records, etc.). I would recommend adopting an extended (with our identifier markup extensions, etc.) version of reStructuredText (good support in Python and it is becoming a ubiquitous de facto standard in places like GitHub, etc.). I realize updating all the existing notes would be a large task but it could eventually prove worthwhile (and we could implement an interim solution supporting both until they are all converted). Uzume 20:57, 12 March 2017 (UTC)
As I mentioned earlier, "the difference is that using these templates in Note fields allows more flexibility, e.g. "corrected page count from {{OCLC|880913738}}, artist from {{OCLC|880914329}}". On the other hand, a separate field for third party identifiers will support robots like Fixer."
Fixer's internal database contains hundreds of thousands of ISBN-less records. Most of them have a third party identifier like an ASIN, an LCCN or an OCLC number. Fixer can't submit them because he has no way of knowing if they are already in the ISFDB database. Once we add support for third party identifiers, it will become possible for Fixer to start submitting them.
As far as inserting malicious JavaScript into Notes goes, does adding support for templates make a difference given that we already support a subset of HTML in notes?
Re: using a third party template processing tool like "reStructuredText", what would be the advantages? As far as I can tell, the software already supports the limited template functionality that we need. Ahasuerus 21:13, 12 March 2017 (UTC)
Fixer (and other bots) can be made to use the new template markup for identifiers when making submissions and can query such via a note search (perhaps with an added API for bot ease of use). Search could be adapted to handle searching for so marked identifiers in notes (to ease searching for, e.g., {{OCLC|###}}, etc.). This could be not much different than the current ISBN search API that I believe Fixer already uses to check if records already exist (with ISBN X, etc.). Uzume 21:30, 12 March 2017 (UTC)
I don't think querying the live database would be viable since there are hundreds of thousands of ISBN-less records in Fixer's internal database. Especially considering how long Publication Notes searches take. A better solution would be to take advantage of the fact that Fixer has access to a modified version of a recent ISFDB backup. It would be possible to pre-parse all publication notes and build a table of ASINs, LCCNs, etc that are referenced in Notes.
In any event, there is nothing preventing editors from adding notes like "Not to be confused with {{OCLC|123456}}, a German translation of this book", which would mislead Fixer into thinking that we already have OCLC 123456 on file.
As a general rule, structured data is better than semi-structured notes, which should be used sparingly. Once we add support for third party identifiers, I expect them to be used in the majority of cases with templates only used when more specific information needs to be recorded in notes. Ahasuerus 22:02, 12 March 2017 (UTC)
I am sure the designers of so called NoSQL databases and RDF data models (as used by Dublin Core, BIBFRAME, RDA, ISBD, and other recent bibliographic models) would disagree with statements like "[a]s a general rule, structured data is better[...]" (many applications are moving away from relational fully structured data and towards other models like directed graph models, e.g., Semantic MediaWiki, Wikidata using Wikibase, etc.). That said, I agree it might not be the most applicable solution for our "note" needs (which as you mention might contain ancillary reference comments about other things). Uzume 22:47, 12 March 2017 (UTC)

(unindent) The following templates have been added:

  • ASIN: Amazon.com
  • ASIN-UK: Amazon UK
  • BL: British Library
  • BnF (also "BNF"): Bibliothèque nationale de France
  • DNB: Deutsche Nationalbibliothek

Ahasuerus 20:41, 10 March 2017 (UTC)

"Publisher" has been implemented. Ahasuerus 23:36, 10 March 2017 (UTC)
It would be awesome if a list of these templates displayed when editing, or a link to a help page that lists all of them with examples of how to use each one? ···日本穣 · 投稿 · Talk to Nihonjoe 00:46, 11 March 2017 (UTC)
Good point! I have updated the Edit pages with a link to the list of supported templates. How does it look? Ahasuerus 01:38, 11 March 2017 (UTC)
Looks good. Are the template case insensitive so {{OCLC|880913738}} and {{oclc|880913738}} produce the same thing? ···日本穣 · 投稿 · Talk to Nihonjoe 07:49, 11 March 2017 (UTC)
At the moment the templates are case sensitive, but I guess there is no reason for them to be. Let me see what I can do... Ahasuerus 16:42, 11 March 2017 (UTC)
OK, all linking templates are now case-insensitive. "OCLC", "oclc" and "oClC" should generate the same links. Ahasuerus 19:53, 11 March 2017 (UTC)
Awesome. That should nip that potential issue in the bud. ···日本穣 · 投稿 · Talk to Nihonjoe 23:20, 11 March 2017 (UTC)
Shouldn't other templates like {{BREAK}} be case insensitive too? --Vasha 07:24, 12 March 2017 (UTC)
It's probably better to make it case-insensitive, although {{BREAK}} is a special template with special limitations. For example, you can only have one in a note. Ahasuerus 15:34, 12 March 2017 (UTC)

Author Note in Advanced Author Search

Advanced Author Search has been updated to support Author Notes. Ahasuerus 23:03, 10 March 2017 (UTC)

Can I search for identifiers in our new note markup (FormatNote templates)? If so, why bother with implementing third party identifiers separately? (see my comments above in #Templates in Notes?) Uzume 21:09, 12 March 2017 (UTC)
Answered above. Ahasuerus 21:14, 12 March 2017 (UTC)

Data deletion policy -- proposed language

Starting a new section since the "Author Note" field is now available and the last discussion is kind of buried anyway.

Based on what we discussed earlier and the original Rules and Standards discussion, I would like to propose the following data deletion language:

  • If a living author (or his/her authorized representative) requests that the ISFDB remove the author's detailed biographical information, the ISFDB will comply after confirming the requester's identity. The ISFDB will remove as much biographical data as needed in order to accommodate legitimate privacy concerns while preserving, to the extent possible, the work of the editors who have compiled the data. A note will be added to the author's record stating that some information has been redacted upon request explaining what type of information has been removed and why.

The idea here is to express our intent and codify our de facto policy while giving editors some leeway when dealing with specific requests. Ahasuerus 00:23, 11 March 2017 (UTC)

I am not sure if we need to add it in the language or just in the rules but I think the notice for the removal should list what had been removed (for example removing a birth day when a birth place/country was never entered should be distinguishable from a request to remove both - so that a different editor knows which fields should not be filled in). Alternatively, we can add a new default value of "withheld" valid in each field (including the date ones) - which will serve the same purpose. Annie 00:40, 11 March 2017 (UTC)
I like this, and I agree with Annie. Otherwise, we'll either have people with good intentions filling in "missing" information or people unwilling to fill in unredacted missing information. --MartyD 12:44, 11 March 2017 (UTC)
OK, I have adjusted the proposed language above. I don't think "withheld" would work in his case because it doesn't support birth dates like "1943-00-00" where the "00-00" part has been redacted. Ahasuerus 20:02, 11 March 2017 (UTC)
Yes, I meant the notion of identifying what was redacted, not "withheld". I think your revised wording does the trick quite nicely. --MartyD 11:52, 12 March 2017 (UTC)
I believe we should also codify that such data can be added back upon the requester's death (it ceases to be a privacy issue then; at what point does such such information eventually become public domain without restriction?; there should be some defined end to the redaction). That way such redacting comments can be removed while updating so marked records as long as I add death dates as well. Uzume 21:19, 12 March 2017 (UTC)
True, but I hope it is covered by the "living author" language in the proposed text. Ahasuerus 21:22, 12 March 2017 (UTC)
Fair enough. I often try to kill off authors from Oldest Living Authors (by researching their deaths and adding the data). Uzume 22:05, 12 March 2017 (UTC)
Me too!--Dirk P Broer 23:14, 13 March 2017 (UTC)

Outcome: ISFDB:Policy has been updated with the proposed language. The affected record has been updated and the author has been notified. Ahasuerus 14:54, 15 March 2017 (UTC)

Change for handling suffixes in author legal names

There has been a small change to Template:AuthorFields:LegalName to allow for capturing of suffixes that are part of an author's legal name, when we have evidence that its inclusion in the legal name is actually the case. It should be added to the end of the normal legal name entry separated by a comma (Lastname, Firstname Middlenames, Suffix). --MartyD 13:36, 12 March 2017 (UTC)

Additional cleanup reports made available to all editors

Additional cleanup reports have been made available to all editors. More to come. Ahasuerus 18:54, 12 March 2017 (UTC)

  • Even more cleanup reports have been made available to all editors. In addition, a display bug which affected non-moderators has been fixed. Ahasuerus 14:45, 13 March 2017 (UTC)

Magazine dates cleanup

Here's an idea for a cleanup project: finding magazines whose date is incorrectly set to the exact date of publication rather than the cover date. Checking all magazines with a day value other than 00 wouldn't catch all problems but would be an improvement. (Manageable if you could ignore at a stroke all issues of a magazine that correctly has days.) Once found, fixing the problem would be a bit of a pain because of having to correct the dates of covers and contents too. --Vasha 20:17, 12 March 2017 (UTC)

Why not have the records reflex the exact date of publication when known (some magazines have issues specified by year and month and then complete publication dates also printed within them and sometimes these are intentionally different; some magazines do not associate year and month with issues and we can only use exact date of publication)? Uzume 22:01, 12 March 2017 (UTC)
Well, it's a very longstanding bibliographic practice here and elsewhere to give the date of a magazine as the cover date; the Hugo awards goes with that too. Usually the table of contents, masthead, or cover will say "May" (ergo, date of YYYY-05-00) or "May-June" or "Summer". Only if such a date is found nowhere in the magazine should the publication date be used. And if the cover date is something like "Summer" or "First Quarter" then the format of the date is YYYY-00-00. That isn't controversial, is it? --Vasha 23:22, 12 March 2017 (UTC)
Note: we should probably change Help:Screen:EditTitle#Year to "Date" to match the edit form. Uzume 22:01, 12 March 2017 (UTC)
Help updated; thanks.
Re: the original proposal, it would be nice to see a few examples of incorrectly set dates. Once I have a better idea of the discrepancies involved, I could run a couple of database queries. That should help us determine how many mismatches we are looking at and what types of cases are common. Ahasuerus 22:43, 12 March 2017 (UTC)
re: exhibit A: Clarkesworld (print issues) (Are all those really printed in October 2012?) and exhibit B: Planet Stories: Adventure House Reprints. Uzume 22:56, 12 March 2017 (UTC)
I was actually referring to this where the date of the January 2017 print issue of Clarkesworld is given as 2017-01-12 (the ebook is correctly 2017-01-00). --Vasha 23:22, 12 March 2017 (UTC)
For reference, Template:PublicationFields:Date used at Help:Screen:EditPub#Date currently states books should use stated publication date (where available) but magazines should use stated cover date (even if there is a stated publication date). This seems strange for reprints and such things. It should be noted, magazines cannot really use editorial title dates for anything useful here because typically magazine pubs are grouped by year of editorial (to force the situation for multiple publication runs of a magazine like reprints and ebooks, etc. we create another series of editorial titles to define "another magazine" run; this is why I prefer pub series be used for magazines but we do not currently do that). Uzume 00:02, 13 March 2017 (UTC)

Yearling Newbery

We have three 'yearling newbery' publishers [1]

  • Dell Yearling Newbery
  • Yearling / Yearling Newbery
  • Yearling Newbery

with 9, 1, and 1 publications. Perhaps Yearling Newbery be considered a publication series of the Yearling imprint, or various Yearling imprints, which may or may not have multiple parent publishers. At the moment none of the several 'yearling' publishers, with or without 'newbery' in its name, has a Yearling Newbery publication series. If these were 100 years older I would make Yearling Newbery a publication series, presumably across multiple renditions of Yearling, and eliminate the three publishers listed above.

Three of the publications by these publishers are primary verified including one by me, transient. --Pwendt|talk 16:30, 13 March 2017 (UTC)

The Strugatsky's 'Midday' future history

There's a COLLECTION that's bound to be a variant of this 'novel'. In the light of the afterword, the listing in 'Bibliographisches Lexikon der utopisch-phantastischen Literatur' and the handling of analogous titles like this one it really seems more apt to see this as a COLLECTION: the various 'parts' were and are published independently; there also is virtually no fix-up involved, the stories only are set in various times. Any comments? Stonecreek 17:22, 13 March 2017 (UTC)

Checking FantLab, I see that 4 of the stories were originally published as standalones. The names of the characters were reportedly changed when the stories were added to this book in 1967. So I guess it's a, um, "limited fix-up"? Ahasuerus 17:56, 13 March 2017 (UTC)
I looks like it was a COLLECTION initially but was later often referred to as a NOVEL, even by the authors themselves, for e.g. in Boris Strugatzki's remarks in Werkausgabe #5 which I just skimmed through quickly. He begins the text talking about the novel ("Roman"). Later he writes there that they initially wanted it to be a novel with a consistent story, but then gave up on that and regarded it as a collection of more or less independent, mostly unrelated stories written at different times and on different occasions (page 833). Moreover, there's also PDF with a complete bibliography of Strugatzki's publications in German at Golkonda's website, and the information on page 26 supports that: it's initially listed as a collection there (or "Episodenroman") but later also as a novel. fantlab.ru also lists it as a collection ("Полдень, XXII век (Возвращение) — это сборник коротких рассказов" => "a collection of short stories" if Google Translate is correct). However, finally, on page 832 in the aforementioned remarks in Werkausgabe #5 there's a hint that the Strugatzkis revised the texts for a Russian edition of complete works from 2001. These remarks are quite extensive and I don't have the time right now to read it all, but it might turn out from these remarks that there's a COLLECTION (early versions) and a revised NOVEL (later versions). Jens Hitspacebar 18:33, 13 March 2017 (UTC)
Based on the available information it seems all publications in our database are COLLECTIONs and I'll change them accordingly. Stonecreek 19:54, 13 March 2017 (UTC)

Verification sources are now available as templates

Our standard verification sources (see Reference:Verification_Sources) are now available as templates. If you enter {{Tuck}}, {{Locus1}}, etc anywhere in notes, the displayed version will show the complete name of the source. It will also be linked to the corresponding ISFDB (or non-ISFDB, as the case may be) Web page. Ahasuerus 18:43, 13 March 2017 (UTC)

Is there a template for {{SFE}}? Chavey 21:44, 15 March 2017 (UTC)
"Clute/Nicholls", "Clute/Grant" and "SFE3" are currently available. Ahasuerus 21:48, 15 March 2017 (UTC)
Thanks. I didn't think to try the "3". Chavey 03:08, 16 March 2017 (UTC)

Displaying images for mixed INTERIORART/COVERART titles

Earlier today I was looking for something else when I stumbled upon the following FR:

  • Currently, when a coverart title is displayed the thumbnails of the coverart images for the parent title and its variants are displayed at the bottom of the title record. Occasionally, a coverart title is a variant of an interiorart title (e.g. interior art was first publication). I'd like to request that thumbnails for the variant coverart titles be displayed on their parent's title page just as they would be if their parent were coverart.

As a point of reference, we have 326 COVERART titles varianted to INTERIORART titles. It would be fairly easy to do what the FR requests. However, first let's make sure that we understand what the change would entail.

Consider this COVERART title, Black Gods and Scarlet Dreams by Michelangelo Merisi da Caravaggio. The linked Title page displays the cover scan which we have on file for the pub associated with this COVERART title. So far so good.

This COVERART title is varianted to the same artist's Medusa, an INTERIORART title. Since it's not a COVERART title, we display either the scans of the covers of the 4 pubs in which this title appears or, depending on the user's preferences, a link to the "All Covers" page.

If we were to change the display logic as per the FR, the Title page for this INTERIORART titles would display the cover scan associated with the COVERART titles -- if and only if the COVERART title is associated with any cover scans. At the same time we would the lose the ability to view the covers of the books in which this INTERIORART can be found.

Is this a desirable trade-off? Ahasuerus 21:57, 13 March 2017 (UTC)

No. Hauck 07:51, 14 March 2017 (UTC)
Can we have both versions (also known as "have your cake and eat it")? Leave the regular logic in place but add a new link for the special case where interior art is the parent to allow the new logic to be shown? Annie 22:56, 15 March 2017 (UTC)

Should entirety of Diabolical Plots be catalogued?

Diabolical Plots pays pro rates, so I've been cataloguing its fiction even though it's web only. I just noticed that the site describes itself like so: "Diabolical Plots is a Sci-fi/Fantasy zine that covers virtually every media related to the genre from books to movies to video games. This site also features regular content related to the craft of writing." Now, despite that self-designation as a zine, it sure looks more bloggish, not organized into issues and with a lot of chatty posts by the editor. I think it's another one to be treated the same way as Tor.com, fiction only-- just because it calls itself a zine doesn't mean we have to treat it as one. What do you folks say? --Vasha 23:08, 13 March 2017 (UTC)

FR 980 - Add checkboxes for importing/exporting COVERART and INTERIORART titles

As per FR 980, the Web pages that let editors import and export contents have been modified to allow skipping COVERART and/or INTERIORART titles. If you encounter any problems, please let me know. Ahasuerus 00:15, 14 March 2017 (UTC)

Title/publication page layout synchronized

The other day Uzume pointed out that the layout of the Publication page had gotten out of sync with the layout of the other bibliographic pages. The former had separate boxes for publication metadata, "Contents" and "Verification Status". The rest of the bibliographic pages didn't.

I have modified the Title page to use separate boxes for the following sections:

  • Title Metadata
  • Variant Titles and Serialization
  • Awards
  • Publications
  • Reviews
  • Bibliographic Warnings

Glory Road can serve as an example of a title which has all of the listed sections. Its variant Het Pad van Roem is an example of a minimalistic title. Please note that various bullet lists and tables have been adjusted to align vertically across boxes. As always, you may need to force a page refresh (Control-F5 in most browsers) to see the new layout.

This patch is experimental. It's hard to explain visual design in words, so I figured it would be easier to tweak the layout of one page and gather feedback. Once we agree on the optimal layout, I will implement the changes across all bibliographic pages. Ahasuerus 20:08, 14 March 2017 (UTC)

I think I like it. Have to look some more to be sure. --Vasha 20:11, 14 March 2017 (UTC)
I like it. It makes it easier to see where each section begins and ends. ···日本穣 · 投稿 · Talk to Nihonjoe 20:57, 14 March 2017 (UTC)
It pushes the publications way too low on the page even for cases where there aren't that many variants or awards... Can we add links in the top section that allows you to jump to a lower one? If this is doable, then I think it will look better. Also - don't "Bibliographic Warnings" belong only to the publications while reviews are tied to titles. It almost feels like the warnings should be moved to a separate page (similar to the BREAK for notes) so that a casual user do not see them... Annie 21:12, 14 March 2017 (UTC)
And one more thing. Looking at Weirdbook #33, it is hiding all empty sections except for that last "Bibliographic Warnings" one - which is empty but still shown. Shouldn't that be treated consistently? Annie 21:17, 14 March 2017 (UTC)
You have a point about publications being low. But I think the variant titles should still be above the publications anyhow. Makes more sense that way. --Vasha 21:28, 14 March 2017 (UTC)
Thus my question of having in-page links so people can jump into what they are interested. I never proposed to change the order. :) Annie 21:43, 14 March 2017 (UTC)
Some minor glitches (all seen on Firefox 52/Ubuntu):
  • the right bracket of the "[edit]" link next to the ID is overlapping with the border on the right. It needs a small margin on the right.
  • the "Awards" box needs a small margin at the bottom (like e.g. the "Variant Titles" already has) because right now it is almost touching the lower border —The preceding unsigned comment was added by Hitspacebar (talkcontribs) .
Also happening with Chrome. All boxes could use a right side margin (see pub table and pub pictures). -- JLaTondre (talk) 22:37, 14 March 2017 (UTC)
Regarding Annie's comment: It would be nice to tighten up some of the spacing. Reduce the divider between boxes, reduce white space before (slightly) and after the Variant Titles, Serializations, Publications, etc. headings. The text size for those headings could be reduced slightly as well. -- JLaTondre (talk) 22:37, 14 March 2017 (UTC)
Working on it... Ahasuerus 22:51, 14 March 2017 (UTC)
I like these suggestions, along with the one above about having "Jump to:" links. If we have those links, it may be good to add "Return to top" links, too, at the bottom of each section. ···日本穣 · 投稿 · Talk to Nihonjoe 22:50, 14 March 2017 (UTC)

(unindent) OK, the spacing has been adjusted. Please use Control-F5 to load the new layout. (There is more white space between the word "Publication" and the table that follows it than in the other boxes; it will take another patch to get everything in sync.) Ahasuerus 00:06, 15 March 2017 (UTC)

Fixed. Ahasuerus 18:53, 15 March 2017 (UTC)

Re: the "Publication" box being too far down, I have a different idea.

At this time the second box from the top contains a list of VTs and a list of serializations (if they exist.) We can reorganize it as follows:

  • Do not display the name of the VT's/serialization's author if it is the same as the name associated with the canonical title. We don't display it on Summary pages and it will free up a fair amount of screen real estate.
  • Take advantage of the freed up real estate and display the VT/serializations as a table with four columns: VTs, translated VTs, serializations, translated serializations. This is something that was requested a couple of years ago, but we had to have the language assignment process completed first.

This will reduce the amount of "stuff" between the box with the metadata and the "Publications" box. Ahasuerus 00:06, 15 March 2017 (UTC)

Nice thought; I'd really like to see how that looks. How will it work on narrow screens though?--Vasha 00:18, 15 March 2017 (UTC)
Well, at most the new table would contain 4 columns. That's not too bad considering that the "Publications" table contains 11 columns. Ahasuerus 00:23, 15 March 2017 (UTC)
I like the new layout but one minor nitpick. The "Other Titles" table has entries with the heading "Date", however, clearly these are not complete dates but rather just the year from the (variant) title records. Uzume 23:30, 2 April 2017 (UTC)
That's a good point. I have changed the headers to read "Year" instead of "Date". Ahasuerus 19:58, 3 April 2017 (UTC)

Bibliographic Warnings

There is a fair amount of history here. Back in 2006, i.e. when the first version of the "ISFDB 2.0" software was released and submission moderation was enabled, our data was very (and I mean very) dirty. For example, one of our original robots had run amok and imported thousands of "out of scope" records from library catalogs. Many other records were missing even the most basic bibliographic data.

During that era, title-level bibliographic warnings were useful because they let you identify potential problems with multiple publications at a glance. However, as data quality improved, the usefulness of bibliographic warnings declined. These days, if a publication record doesn't have some bibliographic information, chances are that there is a good reason for that.

All of the above explains why:

  • bibliographic warnings are now disabled for unregistered users
  • registered users can disable bibliographic warnings under User Preferences
  • the logic behind bibliographic warnings has been repeatedly fine-tuned, e.g. to ignore missing ISBNs for books published before ISBNs were introduced

Having said that, as long as the user chooses to to view bibliographic warnings, I think it is useful to display the word "None" if the logic finds no missing/suspicious data. Ahasuerus 00:39, 15 March 2017 (UTC)

New day, new thing to learn. :) Maybe reverse the default for them? I'll admit I never realized I can fully disable them but then I had not looked at this page for awhile (and when I did, I am not sure I knew what all those are so I just left it to show everything). :) Thanks for the explanation! Annie 01:09, 15 March 2017 (UTC)

2017-03-15 downtime

We are getting low on disk space, so it's time to purge Wiki history again. All but the last 50 versions of each Wiki page will be deleted.

The server will be unavailable between 10:30am and 10:50am server time. It's possible that it may take even longer. Ahasuerus 14:06, 15 March 2017 (UTC)

Everything should be back up. Ahasuerus 14:44, 15 March 2017 (UTC)

Templates for SFE3 and FantLab

SFE3 and FantLab are now supported as templates -- see Help:Using Templates and HTML in Note Fields for details. Ahasuerus 14:27, 15 March 2017 (UTC)

If you are taking requests for other languages, SFBG (http://www.sfbg.us/publisher/code , http://www.sfbg.us/pubsequence/code and http://www.sfbg.us/book/code) will be very useful for anything in Bulgarian (which I am still planning to start working on - I guess waiting to add Translator support will be waiting too long). The codes are stable (and once this is incorporated, the owner of the site is willing to send lists of changed codes if that ever happens (usually for disambig reasons and really rarely). Annie 19:02, 15 March 2017 (UTC)
Done. And yes, translator support is the big elephant in the room. I am currently working on less impactful FRs while considering various design approaches. Ahasuerus 19:26, 15 March 2017 (UTC)
Consider faster (she said looking innocently). :) Annie 19:28, 15 March 2017 (UTC)
Unfortunately, I am still waiting for medical nanobots to be perfected. However, the latest combination of drugs seems to give me more energy, so I guess the answer is "Maybe" :-) Ahasuerus 19:40, 15 March 2017 (UTC)
I have a weird idea about translators... why not create a template for adding a translator to a note - that for now will just print the names with some prefix but then when we figure out how we will support them can automatically start forwarding to where they belong (author pages, new pages, whatever)? Worst case scenario - they never link but at least they are uniform enough and thus convertable if we decide something else. If you go for that, it needs to be a three part one and two part one (Translator|name|transliteration) and just (Translator|name). Or we can leave the transliteration alone and just have the name as credited in the book. Annie 19:53, 15 March 2017 (UTC)
It would be easy to create a new template (let's call it {{Tr}} for now) that would have the same functionality as {{A}}. It would work well for certain translators like Brian Stableford and Michael Kandel who are also authors in their own right. It would be less useful when the translator hasn't written any SF. My primary concern is that it could confuse some users who, after clicking the link, would get something like "A search for 'F. Lancel' found 0 matches". Let me think about it some more... Ahasuerus 20:22, 15 March 2017 (UTC)
Don't make it a link initially - at least it will be standardized and can be changed later to a link. Baby steps - no matter how you implement the support, having all translators noted the same way cannot hurt and will allow automatic conversion. Annie 20:37, 15 March 2017 (UTC)
My idea is to basically prepare for either linking (so change the template, the already entered will work with no change) or for moving to a separate field/relationship - in which case a predefined format will help. It is the same as making a rule except that with a rule we will need to be careful for punctuation and what's not - this will clean up the view so we know where the name ends, will ensure that multiple translators are entered consistently and will allow counting to be done in the DB (and even determine when we have a lot of repetitions). Not best use of templates - but best way I can think of to prepare the data (and after all the cleanup I did in the last months, prepared data (the Italian magazines translations for example) is better than having all kinds of weird conventions and manual parsing and reading long messages to find the info you need. Annie 20:56, 15 March 2017 (UTC)
Oh, I see. It's a thought. Let me see what I can do... Ahasuerus 21:20, 15 March 2017 (UTC)
OK, I have created a new template, "Tr", on the development server. If you enter {{Tr|Brian Stableford and Rob S. Pierre}}, the note field will display "Translated by Brian Stableford and Rob S. Pierre". Any additional information like "as by Jean Valjean" will have to be added manually. It's far from perfect, but it can be viewed as the "low hanging fruit" of this thorny issue. Is this what you had in mind? Ahasuerus 22:02, 15 March 2017 (UTC)
Yep. Any chance to allow {{Tr|Елена Павлова|Elena Pavlova}} that gets populated as "Translated by Елена Павлова (Elena Pavlova)"? If not, will {{Tr|Елена Павлова (Elena Pavlova)}} work? Then all we need is a decent set of conventions (always use "and" for multiple translators for example) and we will at least have the data look the same (and downstream can keep playing with it).Annie 22:16, 15 March 2017 (UTC)
{{Tr|Елена Павлова (Elena Pavlova)}} should work. Actually, rereading what I wrote above I see that I was unclear. I should have written "Any additional information like "as by Jean Valjean" can be added manually as part of the template parameter". Thus {{Tr|Brian Stableford and Rob S. Pierre as by Jean Valjean}} will be displayed as "Translated by Brian Stableford and Rob S. Pierre as by Jean Valjean". Of course, you can also choose to put "as by Jean Valjean" outside of the scope of the template. Sorry about the confusion! Ahasuerus 22:28, 15 March 2017 (UTC)
OK... The problem with this approach is that this will make it very hard to just lift and put directly into a field/point to an author automatically. The multiple translators probably will need manual intervention anyway (unless if someone adds the template twice but for transliterations I kinda want to make it easy to just get the second part, ignore the third during a migration - so it can be automated. And adding it outside of the template defeats the purpose of trying to standardize.
Here is an example. A lot of stuff was translated through Russian into a lot of the Eastern European languages. So a statement such as "Translated by Име(Ime) from the Russian." will now become "Translated from the Russian. {{Tr|Име}}. If I add (Ime) in brackets or after the statement, we won't be able to run a script that says: "Look for "{{Tr|Име}}". If found, delete from notes and use "Име" to add to new field/link to author. No manual cleanup needed. Allowing the triple format, will allow you to just ignore the last part if we are linking to authors links (as we will have transliteration there already and it may be different from what will be here - as it may be common name and not exact transliteration) or keeping it with us if we are moving that to a separate section. :)
Now - if that is too complicated, I am fine with what you have now. I am just thinking of the best way to allow automation to help us when we need to change those - which is why I also want the template to start with. Annie 22:50, 15 March 2017 (UTC)
[After sleeping on it.] I think my main concern is that any kind of automation would be tricky to pull off, especially given the fact that our design hasn't been finalized yet. There are all kinds of cases that will require manual review, e.g. "Translated by Michael Kubiak (chapters 1 to 7) and Frauke Meier (chapters 8 to end of novel)" or "Translated by the author and Cecil Hemley" (real life examples.) For this reason I view the "Tr" template as a tool that will make it easier to enter and find translation credits rather than as a tool that will facilitate automation. Ahasuerus 20:20, 16 March 2017 (UTC)
That's fine. At least it will allow an easy report on them for manual migration. And yes - there will be corner cases but if used carefully, it will allow some automation. Maybe. I just think that we are overthinking the corner cases - most of those credits will be a very straightforward single name. Anyway - still better to have it than not - even if it is just to structure the statement better and allow easier finding of cases where it is there. And it should be easier to recognize when it is just a name and when it is more (manual move of the ones that are not just names leaving just names behind as we did with languages/English for example. So are we getting "Tr" as a template? :) Annie 20:26, 16 March 2017 (UTC)
Done -- see the announcement below. Ahasuerus 20:55, 16 March 2017 (UTC)
More seriously though - I will start with the Bulgarian authors I think, giving it some more time. Thanks for adding it. Annie 19:28, 15 March 2017 (UTC)
One for the Deutsche Nationalbibliothek would also be useful. --Vasha 19:43, 15 March 2017 (UTC)
It's cleverly disguised as "DNB" ;-) Ahasuerus 19:46, 15 March 2017 (UTC)

Charles Robinson, Charles Robinson

Our Author record 88489 conflates two illustrators

The header data including all 5 webpages belong to Charles Heath.

So do all the pre-1970 titles and the 1998, 2000, and 2003 COVERART.

LC Catalog credits young Charles for all of our 1970 to 1982 COVERART book titles except the 1976, not found there.

There are some more 1970s/80s titles, INTERIORART only.

I suggest that young Charles gets all 1970s/80s titles.

--Pwendt|talk 19:21, 15 March 2017 (UTC)

I found more info on young Charles Robinson [2] and added him at WikiData. Here I will now begin to disambiguate him as "(I)" --so that Charles Heath retains numeric id 88489. This should proceed publication by publication if/where that works to cover titles uniquely. Right? --Pwendt|talk 19:59, 17 March 2017 (UTC)
7 PubUpdate in the queue as I depart. The remaining 1970s/80s titles (6) appear in multiple publications (5) or belong to Charles Heath (1 or 2). --Pwendt|talk 20:59, 17 March 2017 (UTC)

Title page changes -- VTs

The VT/Serializations section of the Title page has been changed. VTs' authors are no longer displayed if they are the same as the parent's authors. If authors are displayed, they use the standard "as by" syntax and square brackets, e.g. see Glory Road and Inferno. Ahasuerus 21:19, 15 March 2017 (UTC)

Phase 2: The way VTs are displayed has been altered as per our earlier discussion. The new section is called "Other Titles" and contains a table with up to 4 columns. The columns are as follows:
  • Variant Titles
  • Translations
  • Serializations
  • Translated Serialization
"If This Goes On —" provides an example of a table with all four columns. "Coventry" shows a more common scenario with only 2 columns. Glory Road has 3 columns. To Sail Beyond the Sunset has only one column.
These changes are still raw and may need tweaking. For example, this section is currently called "Other Titles", but only because I couldn't think of another term that would encompass all possible permutations. Ahasuerus 23:47, 15 March 2017 (UTC)
I like it a lot. Maybe use "Other versions" instead of "Other titles"? Annie 23:50, 15 March 2017 (UTC)
Looks good! To make it look more consistent with the other tables the <li> elements should be removed and instead an even/odd style added to the rows (same way as in "Publications" table). Also some padding of the table cells left and right would be good. Jens Hitspacebar 23:57, 15 March 2017 (UTC)
For even better readability the year and language could be put into separate columns, with the main table header spanning two or three columns and the second row containing the column header for each row. Jens Hitspacebar 00:02, 16 March 2017 (UTC)
OK, the bullets have been removed. I'll see what I can do about adding alternating background colors tomorrow morning.
I am not sure about having separate columns for years and languages: the worst case scenario would be 10 columns (2+3+2+3) in the table. I'll have to toy with the table layout. Ahasuerus 03:05, 16 March 2017 (UTC)
Very nice, I like it.--Rkihara 03:12, 16 March 2017 (UTC)
As for my suggestion to split it up into columns: the worst case scenario with 10 columns might indeed look a bit cluttered. Some brainstorming: what if the four "Other Titles" tables would be converted into two different sections containing two tables each - i.e. a grouping of the tables. There are two grouping possibilities:
  1. Grouping by "is it a serialization or not?": One section called "Variants" with tables "Variant Titles" and "Translations"; and a second section called "Serializations" with tables "Serializations" and "Translated Serializations";
  2. or grouping by "is it a translation or not?": One section called "Same-language Variants" with tables "Variant Titles" and "Serializations"; and a second section called "Translations" with tables "Translations" and "Translated Serializations".
As a result it wouldn't look so cluttered. The cost for that would be usage of more vertical screen space and - depending on the amount of data - more scrolling until you get to the "Publications" section. I'm actually not sure right now if that's indeed a better solution, just thinking aloud, but I like #2 better than #1. You'd also get rid of the "Other Titles" term that way. Jens Hitspacebar 09:24, 16 March 2017 (UTC)
It was one of the approaches that I considered earlier this week. As you said, the cost would be losing some of the vertical space that we have just recovered. My plan is to polish the current implementation and then review the alternatives. Ahasuerus 16:33, 16 March 2017 (UTC)

(unindent) Italics removed, background color added. Looking into adding more columns. Ahasuerus 16:19, 16 March 2017 (UTC)

the background colors should be the light shades used for the puolications. --Vasha 17:30, 16 March 2017 (UTC)

(unindent) The "Other Titles" section has been changed as per Hitspacebar's original suggestion. Date, language and title/author are now three separate sub-columns within each column. Some examples:

Re: background colors. As Vasha pointed out, there is a discrepancy between the way "Other Titles" and "Awards" are displayed (darker colors) and the way "Publications" are displayed (lighter colors.) Upon closer examination, it turns out that the "lighter colors" scheme was originally added due to a programming error. Oops! However, we have been using it as part of the standard "Publications" table for about a year now and there have been no complaints. On the other hand, there are plenty of other tables which use the original, darker, scheme starting with the Forthcoming Books page. I guess the question is whether we want to standardize the color scheme and, if so, which scheme we want to pick. Ahasuerus 19:38, 16 March 2017 (UTC)

Did you loose the language in the translated serializations somewhere? :) Annie 19:44, 16 March 2017 (UTC)
Fixed! (Apparently "0" and "1" are not the same thing. Who knew?..) Ahasuerus 19:54, 16 March 2017 (UTC)
I like the darker ones better - just provides better contrast. Annie 19:44, 16 March 2017 (UTC)
Looks better and less cluttered than I feared when I came up with the idea. Actually, I like it a lot better this way. As for the colours: I'd prefer the lighter ones of the "Publications". The darker ones look so... dark :) Jens Hitspacebar 20:09, 16 March 2017 (UTC)

(unindent) Had been looking at some art titles (for example) and the new columns make the view there a bit... weird. I wonder if it is time to have a different view for art titles: 2 columns (one for cover art, one for interior - that is a more meaningful distinction than language here, both having language and year columns.). Any thoughts on that? Annie 20:20, 16 March 2017 (UTC)

Well, COVERART titles can't be serialized, so the way things work now, they can't have more than two columns in the "Other Titles" section. Perhaps we should add a third column for INTERIORART reprints? Ahasuerus 20:30, 16 March 2017 (UTC)
P.S. And, conversely, a "reprinted as cover art" column for INTERIORART titles? Ahasuerus 20:31, 16 March 2017 (UTC)
Well, that would be nice! Stonecreek 21:13, 16 March 2017 (UTC)
My point is that the language distinction is less interesting than the type distinction here. Yes - that will be nice. :) Annie 21:18, 16 March 2017 (UTC)
OK, I have added new columns for various INTERIORART/COVERART combinations, translated and otherwise. Here is what the originally linked COVERART title looks like now. Ahasuerus 22:50, 16 March 2017 (UTC)
And here is an example with all 4 columns for art over here just in case someone is interested. I like it :) Annie 23:05, 16 March 2017 (UTC)

I know it has been mentioned that the "Other Titles" and "Awards" (as well as "Bibliographic Warnings" but that is after) boxes push the "Publications" box somewhat far down the page. As a possible workaround for most users, JavaScript could be added to hide the content until opened (like what is sometimes done on pages with a "+" character button). I realize this would not work for our JavaScript-less users (e.g., text browsers which I believe we still support to some degree for viewing but not editing) but it might help for the vast majority of users. Uzume 00:06, 3 April 2017 (UTC)

A [more] button is a good idea.--Wolfram.winkler 07:29, 5 April 2017 (UTC)

Display parameter for templates in notes

In the course of this discussion, the question arose: would it be desirable to have the option to display a linked record (the discussion concerned authors, but it's not limited to them) with a different link text than the page title? I think yes, and in particular it would be good to be able to display disambiguated authors without the disambiguator which is not very informative. Ahasuerus, if I'm summarizing him correctly, thinks that for the sake of consistency and avoiding any possible confusion, the link text should always be the exact page title. --Vasha 03:30, 16 March 2017 (UTC)

That's right. I hope that "Written by Stephen King (I)" would be more informative and less likely to mislead than "Written by Stephen King". It should alert users to the fact that we have multiple "Stephen Kings" on file and prevent them from jumping to unwarranted conclusions. Ahasuerus 04:07, 16 March 2017 (UTC)
I prefer to see the name as is in our DB as well. Annie 21:22, 16 March 2017 (UTC)

"Tr" template added

A new template, "Tr", has been added -- see Help:Using Templates and HTML in Note Fields for details. As discussed earlier, it's nothing fancy, but using it should make migrating translators to the new system -- once we have it in place -- easier. Ahasuerus 20:55, 16 March 2017 (UTC)

I just tried it, but the template doesn't get parsed. Jens Hitspacebar
Hm, it works on the development server. Investigating... Ahasuerus 21:25, 16 March 2017 (UTC)
Fixed. Apologies, human error. Ahasuerus 21:34, 16 March 2017 (UTC)
Thanks a lot! BTW: I'm on the verge of starting to enter all the nominations for the category "best translation" of the Kurd Laßwitz Prize. Question is: is translator support, which would be able to link the translation to the award, more or less around the corner, i.e. should I wait for it? Otherwise I'd start entering them as "Untitled awards" and rework them later once translator support is done. No pressure, just asking :) Jens Hitspacebar 21:47, 16 March 2017 (UTC)
At this point we have gathered the requirements, which include support for "awards given to translations". They are not final yet, but I think we are pretty close. However, the design hasn't been finalized yet -- which can be a big can of worms -- so I wouldn't say that translator support is around the corner. However, I expect that it will be the next major project that we will tackle (emphasis on "major".) Ahasuerus 22:55, 16 March 2017 (UTC)

Titles with title date before the date of the earlier associated publication

(text moved from this Rule and Standard discussion):

I see that there are 2,098 title records that include the phrase "originally published", "originally appeared", or "originally printed". Ideally, none of those would be necessary because you could just look at the title record and SEE where the first appearance was. But it would take an awful lot of work to achieve that ideal. Chavey 03:03, 17 March 2017 (UTC)

Well... if we all agree that this is what we want to do, 2K are not that many. Plus it can take as long as it takes. Half of them will be as easy as creating the record based on the note; some will need research. Still not undoable. :) Annie 03:22, 17 March 2017 (UTC)
I am afraid there are many more permutations. For example, take "A Bottomless Grave" by Ambrose Bierce. It was first published in 1888, but our earliest pub appeared in 1958. The note field reads "First published in the San Francisco Examiner, February 26, 1888.". There are over 12,000 titles like that although some of them already have the first edition pub on file.
Or take "The Young King": the note doesn't explain where the story first appeared.
OK. So we have more than we thought and it may be a huge project to work through them. There will be some that will never be reconciled (we may know it was in a magazine in 1904 but no clue which issue) - not a reason not to mark them in some way that will allow someone with time and will to do research to reconcile them one day. And the ones that we know the history of? It is not undoable. Just because we may not be able to get them all, should we just give up and just say "oh, well, this train left the station, let's leave it like this"? :) Annie 04:10, 17 March 2017 (UTC)
Oh, I am not saying we shouldn't do it. I am just pointing out that there are some really big and juicy worms in this particular can :-) We'll have to think of ways to create at least somewhat manageable cleanup reports. Ahasuerus 04:16, 17 March 2017 (UTC)
Alphabetically. Decade per decade (starting from current and going back - the very old ones will be the pain to figure out). Random 300 per day. Some other new inventive mechanism :) And then will slowly get fixed. If we can identify the majority of them, reports will be the least of our problem I suspect. Plus - I am willing to speculate that from the 12K you are finding at least some have dates just messed up and need adjustment and not a new pub. Or we have the pub and it just need its content. :) Annie 04:24, 17 March 2017 (UTC)
I'll need to write a query to check how many titles predate their earliest pub. Ahasuerus 03:56, 17 March 2017 (UTC)
That will be an interesting list to look at. You will need to add pub-less titles as well - the ones that the report for pub-less skipped and anything that the report caught but was ignored by a moderator. Some of the ones that are like that (with just a note) are hiding there. Annie 04:10, 17 March 2017 (UTC)

(unindent) Preliminary results excluding publess titles (which are handled by a different cleanup report) and 0000-00-00 pubs:

+----------+--------------+
|      218 | ANTHOLOGY    |
|      700 | COLLECTION   |
|     2891 | COVERART     |
|    20530 | INTERIORART  |
|        8 | EDITOR       |
|    10038 | ESSAY        |
|      423 | INTERVIEW    |
|     5677 | NOVEL        |
|      312 | NONFICTION   |
|      122 | OMNIBUS      |
|     4911 | POEM         |
|     1646 | REVIEW       |
|       51 | SERIAL       |
|    33174 | SHORTFICTION |
|      205 | CHAPBOOK     |
+----------+--------------+

Ahasuerus 04:27, 17 March 2017 (UTC)

These are all titles where the date is before the newest pub in their list? Something does not sound right at all... Some of the containers are because of early ebook that is still missing (unknown ISBN I suspect) but these numbers are way too high. Or am I missing something? Annie 04:36, 17 March 2017 (UTC)
It looks legit to me. Here are some randomly accessed titles: an interview, a review, a serial (an improperly entered date, easy to correct), an EDITOR record, a novel, a poem. Ahasuerus 04:50, 17 March 2017 (UTC)
Time for a cleanup report I guess (with an option for moderators to ignore?)), type by type until we hit the big ones? They need either a note on the discrepancy or it to be resolved. Now that I know about them, they are bugging me - a bibliographic DB should make sense -- and that thing does not... (note to self - next time don't ask questions like that...) :) Thoughts? Annie 05:00, 17 March 2017 (UTC)
Chiming in to say, that when I was going through adding contents to 2016 anthologies/collections last December-January, there were a LOT of reprint stories to be added where we didn't yet have them in the database. If I'd stopped to add the first publication for each one, I wouldn't be finished yet. I just did notes. So yeah, 33,000 short stories to do actually sounds believable, unfortunately. --Vasha 06:22, 17 March 2017 (UTC)
I should start by mentioning that we already have this FR to add a cleanup report which would "verify if the minimum of the publication dates of a title is equal to the date in the title data".
Amusing. As usually my FRs were ignored, I've learned quite early to stop using them. Hauck 18:10, 17 March 2017 (UTC)
Well, at least now there is the warning in the Edit results so it should reduce the number of those (if editors and moderators care enough to resolve the issues that is). May be a few years later but it is a step in the right direction IMHO. Annie 18:27, 17 March 2017 (UTC)
Spot checking the data, I see a few different scenarios. First, we have (apparently) incorrectly entered dates. For example, the current date of the previously linked interview is "1997-00-00", but the first known publication date is "1998-03-00". If the interview was recorded in 1997 and published in 1998, we need to change the title date to 1998-03-00. Similarly, I have seen many introductions which were dated based on the author's signature ("Robert Silverberg, New York City, August 1971") rather than on the date of its first publication. We also have a number of serialized novels whose title date is currently set to the date of the first serialization, e.g. Pellucidar. As per Help:Use of the SERIAL type, "when cataloging novel length fiction, the date of the original serialization is not used and the date of the first book publication is used instead. This is an old (and arguably unfortunate) bibliographic convention, and we follow it." In most cases they should be easy to fix.
Second, we have titles which first appeared in "publications" which we typically do not enter. For example, as per Wikipedia, Tolkien's "Bilbo's Last Song" "was first published on a poster containing illustrations by Pauline Baynes in 1974, the year after Tolkien's death."
Other cases are more straightforward. Of course, some publications will be hard to find due to age, rarity, etc. Ahasuerus 17:45, 17 March 2017 (UTC)
Yes, there will be unresolvable ones(such as Wikipedia) but most of them can be resolved - easily or not. I am just thinking that if we keep ignoring these, the number of titles will multiply and a few years down the road, we will be staring at too many titles to tackle... And in the meantime we have a mix of notes (that cannot be searched well to see how many stories were published in a magazine), wrong dates (that really need fixing) and no notes at all where our data is plain wrong or where it may be correct but noone knows why. I do not expect the DB to be always correct (human beings and all that) but when we know about errors, why not fix them? I have similar misgivings for anthologies and collections (and magazines) with no content but lack of data is a bit better than wrong one:) Annie 18:27, 17 March 2017 (UTC)
I am sure we all agree that errors should be fixed. We just need to come up with the best way to approach the process. For example, suppose we add a new cleanup report. The first time it runs, it finds an 1888 title with no pubs prior to 1896. An editor checks the usual bibliographic sources but doesn't find anything. Should we just "ignore" the title record? Or should we create a new template for these types of situations? Something like {{Unknown1stAppearance}} which would be displayed as "The publication in which this title first appeared is currently unknown" and which would be used by the cleanup report to ignore the title? Ahasuerus 21:26, 17 March 2017 (UTC)
Sorry for going on and on and trying to convince the already convinced :) I was thinking of a new template (which is why I mentioned "marking them" somewhere earlier). This way someone can still figure them out later and we will know that we had looked at these and fixed errors - and they are still standing because we cannot find the bibliographic sources... yet. Annie 21:57, 17 March 2017 (UTC)
What about webzine publications, won't those have to remain as notes? --Vasha 22:17, 17 March 2017 (UTC)
We do not set the date for the webzine-first stories based on the webzine first date though, do we? At least my reading of the rules is that we do not (except for the special cases like tor.com that we index which we do index). If we do, then the whole "we do not add webzines" is getting a bit... weird. :) And we need to start talking about that outside of this conversation :) But yes - if it is out of scope here, we will either need to add as non-genre (magazines, newspapers, books) or note (the poster example above for Bilbo). Webzines... hm... comes down to what we count as first date for the story I guess. Annie 22:23, 17 March 2017 (UTC)
Well, if we decide that a web-first publication should not determine the date, then we nonetheless definitely should make a note mentioning that publication. However, I can tell you, from reading the credits/copyrights for all those anthologies I just added, that it is nearly universal for those credits to state a first publication if it's in a webzine; copyright dates are a bit funnier, since some people re-copyright all their stories when they republish them in collections, but if they use earlier copyright dates, then they will always be the date of the web publications. Shouldn't we fall in line with such a universal practice?
One thing that is not consistent is whether such credits pages mention early publications on the author's personal blog or site. Sometimes; more often not. (I know this because I'm in the habit of trying to check these credit pages by googling, and thus I find unmentioned web publications.) --Vasha 22:39, 17 March 2017 (UTC)
Oh, I know. As I said - I think we need to start talking about the web-based stuff again sooner or later - the world is moving, some of our rules do not make much sense in it :) Part of the reason we do not use copyright dates is because it is not always based on a publication. And in the self-publishing world, it gets even trickier (copyright of when they created it and not when published - I was chasing a few of those the last few days). But then "first publication" becomes an interesting concept. And we need a better definition of it for cases like this. Annie 22:46, 17 March 2017 (UTC)
Ooh, I don't envy you trying to identify the first appearance of self-published things! Often, if they've been removed from sale, not much trace of them remains; and if they are still on sale, they may well bear a date on Amazon or Smashwords that has no relation to when they were first sold. Your best hope is if the author provided a good bibliography on their site. Here's offering you some virtual aspirin for the headaches... --Vasha 22:54, 17 March 2017 (UTC)
Well... I am doing some due diligence but if something is untraceable, I use whatever date I can find and note where the date comes from. If one day an older one is discovered, it will get fixed. This whole conversation started because of the dual practice of saying "First published in London News, 25 May 1934" vs creating a record for "London News, 25 May 1934" as non-genre and adding the story there depending solely on how the editor feels about it. In one case it is searchable and we can see how many stories "London News" published; in the other - not so much. It just got a bit bigger because of the way the DB works :) Annie 23:18, 17 March 2017 (UTC)

Amazon Templates stopped Working

The new Amazon templates have stopped linking. They worked great on the first day then stopped.--Chris J 20:33, 17 March 2017 (UTC)

Sorry, it was an unintended side effect of the last template change. Things should be back to normal now, e.g. see this pub. Ahasuerus 21:01, 17 March 2017 (UTC)

tor.com stories and chapbooks art

We have the series of chapbooks over here and the webzine entries here. The stories are properly merged. The art - not so much :) From the few I looked at, the covers of the chapbooks is the same as the art on the site (which we had indexed). Any objections to varianting them properly (both are always visible - one in Amazon, one in tor.com so we will be able to see and make sure they are the same indeed)? Or am I missing a reason not to? Thanks! Annie 22:19, 17 March 2017 (UTC)

Certainly, why not? --Vasha 22:48, 17 March 2017 (UTC)
PS: and it is varianted and not merged because it is covers on the chapbooks :). My initial message does not make that clear. Annie 22:57, 17 March 2017 (UTC)

Publisher page changes

The layout of the Publisher page has been aligned with the recently tweaked layout of the Title pages. Here is an example of what a publisher with a limited amount of data looks like. And here is what a publisher with a lot of data looks like. Ahasuerus 14:13, 18 March 2017 (UTC)

Synchronizing the layout of bibliographic pages

Based on the discussion of the Title page that we had a few days ago, I have further synchronized the layouts of the "content boxes" on the Title, Publication and Publisher pages.

Please note that the layout of what's inside these boxes is still somewhat inconsistent and will be revisited later, but at least the relative position of the boxes, their margins, white space padding, etc are now the same. I plan to make the same changes to all the other bibliographic pages shortly. Ahasuerus 19:35, 21 March 2017 (UTC)

Publication series done. Ahasuerus 21:02, 21 March 2017 (UTC)
Regular (title) series done although the Series Grid page for magazines is still outstanding. If you see anything unusual, please let me know. Ahasuerus 01:59, 22 March 2017 (UTC)
Series grid done. Ahasuerus 19:50, 22 March 2017 (UTC)
Author pages done. Let's see if these changes help with the previously reported "starkness" problem -- see Kim Stanley Robinson, Bob Smith and Bob Smith (I) for examples of long, medium and short Summary pages. Ahasuerus 20:46, 22 March 2017 (UTC)
I like the look of having the head-information and the bibliography in separate boxes on the author pages. --Vasha 21:20, 3 April 2017 (UTC)

New: Trackers to help keep up with current short fiction

I've created two wiki pages (with a lot of help from Annie-- thanks again!) that are intended to help with adding newly-published (not reprint) short fiction to the database: the Magazine Issue Tracker and the Anthology and Collection Tracker. They are schedules of expected forthcoming publications that can be used on a monthly basis for database updates; they also show which issues and volumes have contents records.

Please take a look and tell me what you think of the design, and of the "How to Use" information at the bottom of the pages. Improvements will be gratefully accepted. --Vasha 19:53, 21 March 2017 (UTC)

Pretty cool. Thanks for assembling! My two suggestions:
Put a legend of the colors at the top of the table, for quick reference.
Maybe specifically label as 2017? And what will happen when 2017 rolls over to 2018? It will be here before we're ready... Albinoflea 03:40, 23 March 2017 (UTC)
I would think that in 2018, the current tables with data will be moved to an archived page (just so the remaining 2017 ones can be finished) and this one will remain empty again for "Current Year". Vasha, what do you think? Annie 03:45, 23 March 2017 (UTC)
Good thoughts. I will put in a header that identifies this iteration as 2017, but keep the page title generic. And as Annie says, at the end of the year we can put something like a box with links to archived pages at the bottom, and change the header to 2018. --Vasha 04:28, 23 March 2017 (UTC)
Changes made; how's that? --Vasha 04:47, 23 March 2017 (UTC)
I've also lightened up the background colors. --Vasha 05:21, 23 March 2017 (UTC)

Format question: dos vs tp

I had been adding some entries here and it strikes me that most of the records are set to "tp" instead of "dos". As it is a double cover (cover on the back and on the front), double-novel (*novel 60s style anyway), it looks to me like the definition of "dos" and that overwrites the more generic "tp" (anything that is not already covered and is paper)... Am I misunderstanding what "dos" is supposed to be used for? Or do most of these need a format conversion? Annie 23:53, 22 March 2017 (UTC)

Yes, my understanding is that "dos" would take precedence over "tp", but I'm guessing that since the two novels aren't rotated 180º to each other (at least from the cover examples shown) that these might not technically be valid "dos" candidates. At least that's my take. Perhaps the documentation should be clarified. Albinoflea 03:26, 23 March 2017 (UTC)
Hm, I did not think about the rotation (never seen one of the double novels that are usually used for an example) and you are probably right. I think that the help page needs a bit of an update - may be obvious for most people around here that they need the rotation but it tripped me. Thanks! :) Annie 03:43, 23 March 2017 (UTC)
Only if it's rotated and a part therefore readable from the back, or if there are two backs glued together it's a dos. That's how I understood it so far. The Dos-à-dos binding Wikipedia article has a good description. Jens Hitspacebar 11:21, 23 March 2017 (UTC)
That is explaining a rotation on the other axis from what I was thinking. Still not what Armchair fiction are doing though so I will fix those. Now you made me want to go and find one of the Ace doubles just so I can look at the thing... Annie 17:55, 23 March 2017 (UTC)

David Gemmell Award

Can we add this award to the available awards in the database? It's a popular vote (people vote on the website) set of awards given out each year (since 2009 for the Legend Award, and 2010 for the other two) for fantasy books. There are three specific awards:

  • Legend Award - awarded to the fantasy title judged the year’s best, as chosen by open vote
  • Morningstar Award - awarded to the the author judged to have made the year’s best debut in fantasy fiction, as chosen by open vote
  • Ravenheart Award - awarded to the artist(s) responsible for the year’s best fantasy book cover art, as chosen by open vote

The short name for the award is "Gemmell Award". The website for the award is http://www.gemmellawards.com/. ···日本穣 · 投稿 · Talk to Nihonjoe 21:32, 23 March 2017 (UTC)

Their online voting system is somewhat unorthodox, but I recognize the people who administer the award as established professionals. Hopefully they have added safeguards to prevent multiple voting and such. Otherwise the awards look legitimate and I have no objection to adding them. Ahasuerus 00:43, 24 March 2017 (UTC)
Okay. Let me know when it's been added and I'll populate it. ···日本穣 · 投稿 · Talk to Nihonjoe 19:01, 24 March 2017 (UTC)
Not seeing any objections to adding it. This one should be fairly quick to add the content since it hasn't been around for too long. Thanks in advance, Ahasuerus. ···日本穣 · 投稿 · Talk to Nihonjoe 18:33, 5 April 2017 (UTC)
Done, including all 3 categories. Ahasuerus 18:56, 5 April 2017 (UTC)
Awesome, I'll fill them up over the next couple days or three. ···日本穣 · 投稿 · Talk to Nihonjoe 19:44, 5 April 2017 (UTC)

Add language request

Can Akkadian be added as a language selection? I'm concerned with The Epic of Gilgamesh. Thanks, Ldb001 14:20, 24 March 2017 (UTC)

Akkadian is an ISO-2-recognized language (language code akk), so it shouldn't be a problem. Let me double-check with Linguist to see if there is anything else we may need to add to support Gilgamesh. Ahasuerus 14:28, 24 March 2017 (UTC)
No problem with Akkadian as a language. But remember that the original language of The Epic of Gilgamesh is Sumerian, which is utterly different… Linguist 14:33, 24 March 2017 (UTC).
Should we add both Akkadian ("akk") and Sumerian ("sux") then? They are both ISO-2-recognized languages, so it wouldn't cause any issues. Ahasuerus 15:09, 24 March 2017 (UTC)
Might as well. As the French idiom goes, ça ne mange pas de pain (there's no harm in it) :o) Linguist 15:19, 24 March 2017 (UTC).
Done! Ahasuerus 15:45, 24 March 2017 (UTC)
Thank you for taking care of that so quickly! My friend wikipedia tells me that the two surviving tablet sources of the epic proper are in Akkadian and Old Babylonian (which is called a dialect of Akkadian). Some older Sumerian associated material also exists. I'm sure that modern texts draw from all sources to make as complete a version as possible, but because the Akkadian source is more complete, for shorthand Akkadian seems to be a good choice. I have no knowledge of what proportions of what sources have been used for this translation.
I don't know protocol for including a hypothetical (lost) original form which may have been in Sumerian. At any rate the Akkadian/Babylonian is apparently the oldest version of the epic that exists. Thank you again for the language support, Ldb001 22:53, 24 March 2017 (UTC)
I have handled these sorts of situation by making a variant parent title without any supporting pubs. You can then add notes to this parent title records (and if the pub ever is found it can be added later). This would make things like an English translation also exist under the missing Sumerian parent record too (regardless what it was really translated from; just make name good notes when there are translations of translations). Uzume 23:50, 2 April 2017 (UTC)

Possible new reports

Would it be easy to create "award eligible" reports that listed:

  • All the novels that came out in the previous year
  • All the novellas that came out in the previous year
  • All the novelettes that came out in the previous year
  • All the short stories that came out in the previous year

And so on, for whichever categories we would cover? It might be a good way to collaborate with the Hugo, Nebula, Dragon Award, etc., crowd to provide an easy set of reports that showed this information. What do you think? ···日本穣 · 投稿 · Talk to Nihonjoe 17:23, 29 March 2017 (UTC)

You can run an Advanced Title Search on Year=2016 and Title Type=NOVEL and Language=English (or any other language) to get a report of novels first published in 2016. Ditto for novellas etc. The report is limited to 100 titles per page though. Ahasuerus 18:13, 29 March 2017 (UTC)
Yeah, that limitation is why I thought it might be useful to have a system-generated report that is run once a year instead. ···日本穣 · 投稿 · Talk to Nihonjoe 19:31, 29 March 2017 (UTC)
Except that it won't be run once a year because we add more stories daily - I added more 2016 ones a few days ago for example. Annie 19:50, 29 March 2017 (UTC)
(after edit conflict)
We are constantly adding/merging/deleting titles, so a report that is run once a year would quickly get out of sync with the rest of the database. On the other hand, it wouldn't be a particularly resources-hungry report (it takes less than a second to compile on the development server), so we could make it available as an on-demand report.
The main report page will presumably include the following drop-down lists:
  • Year, limited to 1900-current year
  • Title type/length, including every title type and every length types (SHORTFICTION would include all short fiction)
  • Language, including "all"
At 5K-10K lines per report it will be a bit unwieldy, but probably better than paging through the same information 100 titles per page. We also have lists of Most-Reviewed Titles by year and by decade. Ahasuerus 19:59, 29 March 2017 (UTC)
So, is there enough support for the proposed page to justify an FR? Ahasuerus 21:28, 30 March 2017 (UTC)
I support the idea. ···日本穣 · 投稿 · Talk to Nihonjoe 22:50, 30 March 2017 (UTC)
Oh well - why not. However - I have a question on the proposed implementation: what year will be a story shown in if it is a German story from 1989 but published in English in 2014, new name in 2017 in English? Awards-eligibility-wise, this should be 2014 for English but that will mean looking at a variant date - and finding the correct variant based on the language. And then the same would be nice for other languages. So if I filter by German and 1989, I expect to see the story; (English, 2014) should also show it and the 2017 variant is never accounted for. Is that the plan? :) Annie 23:10, 30 March 2017 (UTC)
Award eligibility rules can be complicated. For example, the Hugo rules say:
  • Works published in prior years outside of the USA are eligible if they were published for the first time in the USA in the current year.
Other awards may have different caveats and quirks. I doubt we can account for all of them, especially considering that it is a moving target. My plan was more modest -- simply provide a list of the titles that match the entered criteria. It will be up to the user to tweak the search criteria depending on his or her needs. Ahasuerus 23:41, 30 March 2017 (UTC)
That's fine as long as in my example above, English, 2014 pulls the story but English, 2017 does not - despite having both as variants :) Actually for what I would like to use that, that is not a problem either way - but for what the conversation started as, it does make sense. Can we add a "translated" flag as a column (or even a filter) to indicate which of the found stories are not originally in the language you filtered on? Annie 23:57, 30 March 2017 (UTC)
We could do what "Show All Titles" does: add a column for the parent title and perhaps another one for the language of the canonical title if it's different from the variant's. Ahasuerus 00:15, 31 March 2017 (UTC)
Sounds good. Annie 00:53, 31 March 2017 (UTC)
OK, FR 1024 has been created. Ahasuerus 15:37, 31 March 2017 (UTC)

Perry Rhodan help needed

Can someone that understand the series of Perry Rhodan figure out which of the series should the three listed here go and add them to their proper series (and single title record as these are from the same year). Thanks! Annie 18:10, 29 March 2017 (UTC)

Noone knows where these three fit? :)Annie 18:01, 31 March 2017 (UTC)
I used to have to find this on my own (as I said, data consistency is not a favorite sport here), by analogy with this, I would have entered "Perry Rhodan (2. Aufl.)" (coherent with #PR2 catalog number) and then merge by year.Hauck 18:11, 31 March 2017 (UTC)
But we should not need to do that when there are a few people that actively add these and understand the series... Oh well - did not realize that you were fixing those on their behalf - I will fix them then. Thanks! Annie 18:39, 31 March 2017 (UTC)
Merge submitted, moderator needs to approve. Annie 18:49, 31 March 2017 (UTC)
And all done. Thanks.Annie 22:20, 31 March 2017 (UTC)

Make Variant - submission review changes

The "review" page for Make Variant submissions has been changed. The following lines have been added:

  • Length
  • Juvenile
  • Non-Genre
  • Graphic
  • Novelization

When varianting to an existing title, any mismatches will generate a warning. Ahasuerus 21:26, 30 March 2017 (UTC)

Note that we still haven't decided what to do when a text "changes category" in translation (e.g. "upward" when translated from english to french or the other way from german to english). Hauck 09:08, 31 March 2017 (UTC)
Well, the ss/nt/nv division only applies to English anyway. Maybe other languages shouldn't have length set? --Vasha 14:21, 31 March 2017 (UTC)
Maybe we should not even list titles not in english. Hauck 17:03, 31 March 2017 (UTC)
Why would it only apply to English? Bulgarian has 3 different words for these as well for example (as does Russian and I suspect most other languages). And online bibliographies split the stories in the 3 groups. So the division is very much applicable for non-English as well. Annie 16:49, 31 March 2017 (UTC)
PS: And even where novelette is somewhat of an artificial construct (classically the split is in 2), the other 2 had always existed. I am strongly against not listing the type in other languages - a novella and a short story are as different as a novel and a short story, regardless of the language. Plus the standard English split had become a de-facto standard in the SF field anyway. Annie 17:13, 31 March 2017 (UTC)
OK... So what is the word length of the three categories in Bulgarian? --Vasha 16:50, 2 April 2017 (UTC)

Additional ISBN warnings

The software has been updated to display yellow warnings if:

  • the entered ISBN contains 10 characters and the publication appeared after 2007, or
  • the entered ISBN contains 13 characters and the publication appeared prior to 2005

Ahasuerus 23:35, 30 March 2017 (UTC)

Excellent... Thanks! Albinoflea 03:32, 31 March 2017 (UTC)

Bibliographic warnings: "missing page count"

Title pages have been modified not to display "missing page count" warnings for webzines, audio books and digital books. Ahasuerus 17:29, 31 March 2017 (UTC)

Title type/publication type validation

The 4 publication-specific edit forms -- NewPub, AddPub, ClonePub and EditPub -- have been updated to check for EDITOR mismatches. Duplicate EDITOR titles, EDITOR titles in non-magazine/fanzine pubs, and missing EDITOR titles in magazine/fanzine pubs should result in pop-up error messages from now on. You may need to do a full form re-load (Control-F5 in most browsers) for the new functionality to become available immediately.

If everything works out and there are no issues, I will add similar checks for other publication/title mismatches starting tomorrow. Ahasuerus 02:10, 1 April 2017 (UTC)

It turns out that the pop-up validation part of the new code didn't play nice with older browsers. I have disabled it pending an investigation. You will still get a yellow error message if you try to create a submission with an invalid or missing EDITOR title. Ahasuerus 15:29, 1 April 2017 (UTC)
The pop-up validation code has been rewritten using older, hopefully backward compatible, features. When entering/editing the next publication, please force a full page re-load (Control-F5 in most browsers.) If you encounter any issues, please let me know. Ahasuerus 16:54, 1 April 2017 (UTC)

(unindent) Additional validation has been added. CHAPBOOK publications are now required to have one (and only one) CHAPBOOK title and at least one SHORTFICTION/POEM/SERIAL title. Ahasuerus 23:34, 1 April 2017 (UTC)

Will it allow the chapbook to be added (NewPub) without a story inside (so it can be imported later)? Or will a story always need to be added and then merged? Annie 00:36, 2 April 2017 (UTC)
At this time the validation logic requires at least one SHORTFICTION/POEM/SERIAL title for new CHAPBOOK publications. However, it would be easy to change if the import route is common. Ahasuerus 01:45, 2 April 2017 (UTC)
I much prefer to import than to merge (all those reprints and kindle singles and what's not these days - and if we will ramp up on the ebooks adding, there will be even more of these) but if the agreement is that there are too much issues if it is left this way, I will change my usual routine. But we will need a "short fiction duplicates" report - at least chapbooks with no content have a report (I think?) and this will help with spotting the second steps not being taken. Annie 02:36, 2 April 2017 (UTC)
Now that you mention it, it makes sense. Let me revert the problematic part of the last patch... Ahasuerus 03:38, 2 April 2017 (UTC)
Done. Ahasuerus 15:40, 2 April 2017 (UTC)

(unindent) ANTHOLOGY, COLLECTION, NONFICTION, NOVEL, and OMNIBUS titles are no longer allowed in CHAPBOOK publications. Ahasuerus 16:07, 2 April 2017 (UTC)

NOVEL titles are no longer allowed in MAGAZINE/FANZINE publications. A reference title (i.e. a title of the same type as the publication record) is now required when editing/cloning publications. Ahasuerus 21:43, 2 April 2017 (UTC)
The "reference title" requirement has been streamlined for Clone/Import/Export submissions and should no longer generate spurious errors. Ahasuerus 13:24, 3 April 2017 (UTC)
Putting them in as SERIAL (complete novel) seems odd to me -- I know there must be a reason. What's the backstory? --Vasha 22:52, 2 April 2017 (UTC)
That's right, there is a reason (or perhaps multiple reasons) behind the current approach -- see this Help section for the gory details. Ahasuerus 13:24, 3 April 2017 (UTC)
Thanks, that makes a lot of sense. So, I decided to look for novels that have their title date erroneously set to the date of a serialization, and quickly found one, A Yank at Valhalla. Anyone else want to have a go at continuing the search? --Vasha 14:38, 3 April 2017 (UTC)
What about this, a serial never printed in book form. Should the date be "unpublished"? --Vasha 14:43, 3 April 2017 (UTC)
You are right, the use of dates for SERIALs and their parent titles has been somewhat inconsistent. The reasoning described in that Help section has one significant flaw: it distorts Author Biblio pages by messing up the order in which the author's works appeared.
Consider Jack Williamson's Summary page. At this time the first title in the "Novels" section is The Green Girl, which was first serialized in 1930 and published in book form in 1950. The reason it appears first is that someone set the NOVEL title's date to 1930. On the other hand, the title date of the Golden Blood is set to 1964, the date of the first NOVEL publication, even though it was first serialized in 1933.
My current thinking is that there is got to be a way to accommodate the two sets of concerns using some software trick, but I haven't been able to come up with a good design so far. Ahasuerus 19:00, 3 April 2017 (UTC)
I cannot attest to this being a good design but we could have parent titles always be empty and all pubs be under a variant. This way the parent could be the real original publication date (regardless of publication method) and the child novel title record the original novel publication date (and of course serials always represent their dates since they are basically one-to-one with their pubs unless there is a true reprint). And other variants are still possible (name differences and translations, etc.). I do not like the "unpublished" parent concept for novels only ever published in magazines. Uzume 05:40, 4 April 2017 (UTC)

2017-04-01 downtime @7:10pm

The server will be unavailable between 7:10pm and 7:12pm server (US Eastern) time. Ahasuerus 22:53, 1 April 2017 (UTC)

The server is back up. Ahasuerus 23:32, 1 April 2017 (UTC)

Title merge bug with transliterated titles

I have been doing some title merges (for Japanese magazine editorial records making them "annualized") and I have noticed an issue related to transliterated titles. Instead of letting me pick which title record to take transliterated titles from, it automatically just forces me to take them all (which is not right and now I have to go back and edit the titles individually to remove these incorrect transliterated titles). Can this be fixed? Thank you, Uzume 14:16, 4 April 2017 (UTC)

It's not a bug, it's a feature :-) That's how all multiply occurring values like Web pages and transliterated names are handled when merging title/author/publisher records. If one or more of the multiply occurring values is in error, they need to be removed before or after the merge. Ahasuerus 15:27, 4 April 2017 (UTC)
That is not really good for titles. If I have titles "XXX" and "YYY" with transliterations, why would I want to choose one of those titles but have to take transliterations for both? Uzume 16:15, 4 April 2017 (UTC)
When merging multiply occurring values, numerous permutations are possible. Sometimes you may want to keep all of them. Other times you may want to keep the values from just one record. Or you may want to keep the values from 2 out of N records. Or you may want to keep selected values from record 2 and 3, but drop all of the values from record 1 as well as selected values from records 2 and 3.
It's possible to support all of these options by adding check-boxes for individual multiply occurring values, but it's a low priority. Ahasuerus 18:00, 4 April 2017 (UTC)
Transliteration is for what is now the title. So why would we want to keep a transliteration from an old now non-existing title? And when Magazines are concerned, you never want the individual issues transliterations... Annie 18:03, 4 April 2017 (UTC)
Let me use an example. Suppose I want to merge two Bulgarian title records. The titles are identical, "Лъвът"/"Лъвът". The first one has the following transliterations: L"v"vt and Luvuvt. The second one has Luvuvt and Lavavt. The way the software works, merging these titles will result in the following transliterated values: L"v"vt, Luvuvt and Lavavt. (The two occurrences of Luvuvt will become one.) Is this the desired behavior? Ahasuerus 18:14, 4 April 2017 (UTC)
That is fine, combining the different transliterations do not harm anything when the two titles match to start with. Not so much if you are merging Знаме, Януари 2016 and Знаме - 2016. When you merge that, the Знаме - 2016 will now have also a transliteration as Zname, Yanuari 2016. Maybe make a special exception for EDITOR records? Annie 18:28, 4 April 2017 (UTC)
It would appear that the underlying problem is not so much the title type but rather the fact that the about-to-be-merged titles are different. It's possible to change the software to let editors select one set of transliterated values when titles are different. However, there are various scenarios to consider. For example, suppose I want to merge title records A, B and C. They all have transliterated values associated with them. Records A and B have the same title while C's title is different. What would be the desired software behavior in this case? Also, I am concerned that introducing additional exceptions may make life even harder for new editors. Ahasuerus 19:36, 4 April 2017 (UTC)
Yes, but it is just more likely to happen with an EDITOR :) Maybe keep it easy - if at least one of the titles is different, force the editors to select the same way as they select a title. If there is a single title, then just combine all existing ones? Or even combine the transliteration with the title - if a title need to be selected, its transliteration comes with it automatically. Annie 20:35, 4 April 2017 (UTC)
Why not let the editor select such? If you have five titles to merge give me six choices. I can pick which one or have the complete merge (methinks it is overkill to allow partial merges, etc. though that could be accomplished with checkboxes). Uzume 21:27, 4 April 2017 (UTC)
After sleeping on it, I think the last approach makes the most sense. It's user-friendly and consistent. I have created FR 1028 which covers all multiply occurring fields, not just transliterations. Due to the way merges work under the hood, it won't be a trivial change, but it should be doable. Ahasuerus 15:47, 5 April 2017 (UTC)
To make matters works the submission record appears like it chose one of the title to take transliterations from when in fact they were all merged (when the submission is committed). Thank you, Uzume 14:22, 4 April 2017 (UTC)
I am not sure I understand. Could you please provide an example? Ahasuerus 15:27, 4 April 2017 (UTC)
Unfortunately, TitleMerge submissions become useless to look at after they are committed by a moderator. Uzume 16:15, 4 April 2017 (UTC)
Do you recall the sequence of events and the exhibited behavior? If you could describe it, I could try to recreate the problem on the development server. Ahasuerus 19:38, 4 April 2017 (UTC)
I just merged 2-3 Japanese editorial titles where each had multiple (2-4) existing transliterations. Since you are saying currently, complete merge of multi-value fields like transliterations is expected behavior, I am wondering how that is displayed upon submission (and in looking at the submission in the queue before moderation). Uzume 02:30, 5 April 2017 (UTC)
When merging multiply occurring fields, all cells use the background color for "retained" values. Ahasuerus 16:45, 5 April 2017 (UTC)

PLAY title type

Is it time to implement the PLAY title type yet? I just added half-a-dozen more of them today. (A cappella Zoo loved to publish mini-dramas.) --Vasha 15:02, 4 April 2017 (UTC)

No, I'm not sure that a consensus was reached. And, if we should restage the play, I'm still completely against this move. Hauck 15:06, 4 April 2017 (UTC)
I am sort of ambivalent, however, Hauck, why are you against it (I do not see it much different than POEM)? Uzume 15:16, 4 April 2017 (UTC)
Well, somewhere in the mists of the wiki, I gave some arguments. In a nutshell, I see the possible inflation of types as not desirable. Half-jokingly I proposed also HAIKU, SONG, FILK, CARTOON, TVSCRIPT (all types that we're presently listing under diverse categories) etc... My main objection to this complexification is the difficulty to convey such subtilities to new contributors (here I speak for myself as I'm regularly welcoming and guiding new users). I'm also also worried by the potential frictions that may arise with such detailed categories (I remember some heated debates on prose poems, are they SHORTSTORY or POEM?). Hauck 15:29, 4 April 2017 (UTC)
Yes, I agree that multiplication of types would be confusing and make it harder to assign things. (I can never remember what Locus & Contento mean by all those 20 abbreviations they use for categories.) That is one reason why, when adding PROSE POEM was proposed, I decided that I was against it. But I would make an exception for PLAY. That is a very clear and obvious category. There are a few cases which might, just possibly, be confusing, because the scenario for a movie is occasionally written like a short story. But if it says on it "a movie scenario", then it's a play. All right, admittedly sometimes you may have a story in dialogue form. But if it's not explicitly identified as a play, then it isn't one. --Vasha 15:47, 4 April 2017 (UTC)
I don't think that we have enough plays to justify a new type - especially because there are two types of plays (prose and verse) and there is a difference between a short (2 pages one) and a long (novella length one) - now keeping them with the other text styles, allows the distinction. I would not be opposed to a template that can be added to the notes that make it easier to show/find them. Annie 16:28, 4 April 2017 (UTC)
After hearing Hauck's argument, I agree we do not need a PLAY type. I would be open to a generic script type that could include plays as well as any other types of scripts we might run into and would be applicable under the rules of acquisition. Uzume 21:30, 4 April 2017 (UTC)
That was what I meant anyhow -- a title type covering prose and verse dramas, tv scripts, movie, radio, audio dramas, librettos. Last time the subject was discussed, the question of what to call it came up, and it was pointed out that locus indexes all those things under "play", so it might be good to use the same term. But "script" or "drama" would be OK. Vasha 21:42, 4 April 2017 (UTC)
I support "SCRIPT", as it would cover stage, television, radio/audio, and film scripts. ···日本穣 · 投稿 · Talk to Nihonjoe 21:55, 4 April 2017 (UTC)
And if the consensus is against a title type, a template in the notes would not be a bad idea. Inserting the words "This work is the script of a drama" maybe? That sounds somewhat odd but it's understandable. Vasha 22:07, 4 April 2017 (UTC)
And there would also be some plays that are of NOVEL length; and IMHO it wouldn't be desirable to separate this two. Stonecreek 07:20, 5 April 2017 (UTC)
Still opposed to the idea of SCRIPT (I've just thought about some pure SHORTFICTION texts that use the format of scripts without having been concieved to be filmed or aired, where should we put them?), the template is a good idea (or the simpler mention in the note field). Hauck 07:23, 5 April 2017 (UTC)
Right! Terry Bisson did a few of those things that are made solely out of the text of conversations. Stonecreek 07:30, 5 April 2017 (UTC)
That's why I said this category only applies to works that are definitely identified as the script to some sort of drama production. Anyone want to tweak the wording to that proposed template to make this more clear? (I M coming round to the idea that a template would be better than a title type.) Vasha 16:13, 5 April 2017 (UTC)
And who makes the decision? If it looks like a play, an editor will add it as a play even for those Bisson ones - and then a moderator will need to realize this is the case and fix. The less confusing we keep all that, the less likely we end up with everything being set in its own way :) Plus with a template, counting those will be easy and if we see a lot? Easily fixable :) Annie 16:31, 5 April 2017 (UTC)

(Unindent) How about {{SCRIPT}} to insert the words "This work is a script for a production." Vasha 16:51, 5 April 2017 (UTC)

NDL via ISBN

Can I request the "National Diet Library" be added to the "Other Sites" navbar with the URL: http://iss.ndl.go.jp/api/openurl?rft.isbn=%s&locale=en (where "%s" is the ISBN)? This will remove the need to link to the NDL in many notes. The URL works with any correct ISBN-10 or ISBN-13 (hyphenated or not). I should probably also consider requesting other significant Japanese places like Amazon JP (there are many missing Amazon sites now; you can see how to link to them at ISFDB:Book sources which I updated for ISBN access via the wiki; see it in action with this example ISBN 4-15-010175-2), CiNii, and Webcat Plus but I can save that for another time. Thank you, Uzume 23:16, 4 April 2017 (UTC)

A number of editors have expressed concern about the "Other Sites" section of the navigation bar getting too long and making it hard to scroll down to the "Editing Tools" section. We may want to address this issue before we add even more links to it.
For example, we could try to convert the bullet list to a drop-down list. The list would be similar to the drop-down list main search box. Once you select your Web site of choice and click "Go", the software will take you to that site. The only catch is that it may be difficult to make it appear in a new window.
Another approach would be to hide the "Other Sites" by default and display a small arrow in their place. When the arrow has been clicked, the "Other Sites" list will appear. Ahasuerus 00:06, 5 April 2017 (UTC)
We could have a set of buttons with drop-down lists, e.g., an Amazon button with a site selector drop-down to select which Amazon site. The buttons could be styled to look like links so they could be in a menu and less obtrusive. Though I am against using target="_blank" you can put that on forms to make them open in a new window upon selecting a submit button (but the same can be had with ctrl+click as on links). The more interesting advantage of using a form is that the form would essentially be a redirection form (you submit it and it redirects you to the site) and as such we could track the usage of such clicks as they round-trip through the server. I am not sure I like this idea but it has interesting implications. Uzume 12:12, 5 April 2017 (UTC)
I support adding the NDL, and I think the drop-down list is a good idea. Instead having to click "Go", what about just having the list be a set of links? That way, you can pass target="_blank" through as a part of the anchor tag, and that will tell it to open in a new tab or window (depending on the browser preferences of the user). ···日本穣 · 投稿 · Talk to Nihonjoe 01:26, 5 April 2017 (UTC)
Menus can be created via JavaScript and/or pure CSS. I am not sure how that may or may not affect text browsers (we do still support such right?). The list could also be made smaller via abbreviations and acronyms (not sure that is the best idea but those who use such links often would obviously know). My personal recommendation would be to use CSS to hide large sections in a menu that appears upon mouse over or keyboard focus (e.g., hover over ISBN and menu appears and click destination choice). But this speaks to the possibilities of a full interface redesign where we could do the same with many longer menus, etc. I am not a fan of forcing links to open in a new window (i.e., target _blank). In most browsers the same can be had via ctrl+click and lets the user choose. We could also use another strategy. Instead of directly supporting such via the web application we could link to the wiki's ISBN handling page. It would require another click on a separate page but the bonus is the list could be very long with little issue (and the linking DB updated via wiki semantics). Anyway, that is why (discussion is needed) I only requested the one NDL change...for now. Uzume 01:59, 5 April 2017 (UTC)
There is a pre-existing FR to add Amazon CN, ES, IT, IN and JP to the list of supported sites. I have been hesitant to implement it for the reasons outlined above, so it's not just NDL.
As far as the implementation side goes, as long as we use "pure CSS" menus, we should be fine. The problem with using JavaScript on bibliographic pages is not just text browsers, it's the millions of people who browse the internet with JavaScript disabled by default for security reasons. We get a fair amount of traffic from search engines like Google, so we need to ensure that random visitors get a coherent view of our data even if they have disabled JavaScript. Ahasuerus 04:57, 5 April 2017 (UTC)
JavaScript can work just fine (many large organizations are using it) but you are right it needs to degrade gracefully in situations where it is unavailable in order to maintain accessibility (so we should not be adding significant content to the page in AJAXy ways but we can implement non-significant user interface content). And while we are on the topic of accessibility, we might also want to consider not just JavaScript vs. no JavaScript but also things like mouse vs. keyboard (or even screen readers, etc.). Pure CSS menus can have issues for keyboard users (depending on how they are implemented). From what I have read there are a few solutions. The older way is to implement mouse hover menus in CSS and hide it from keyboard access making keyboard access go through a more cumbersome (but still available) set of links elsewhere (via links to anchors farther down the page or on a different page). The newer way is to use WAI-ARIA role hinting and degradable JavaScript. For our purposes, I think our best bet might be to make the menus be farther down the page (like they basically are now) and put in anchor links to those (for accessibility) and then use JavaScript to relocate the menus to popups over the anchor links with mouse hover (and possibly keyboard focus), etc. for ease of use for users with JavaScript (which includes all editors as JavaScript is required for editing I believe). I would need to find a good example somewhere. Uzume 11:16, 5 April 2017 (UTC)
If you are concerned with space right now, one thing can already be done. You can kill Shelfari which is gone as it merged with Goodreads (which does not need the strange capital "R" in the middle). Uzume 13:53, 6 April 2017 (UTC)

(unindent) As per the discussion above, I have made the following changes:

  • Removed Shelfari
  • Changed "GoodReads" to "Goodreads"
  • Added NDL
  • Added Amazon JP
  • Added the Library of Congress
  • Alphabetized the list of sites under My Preferences/My Web Site

Ahasuerus 19:56, 6 April 2017 (UTC)

Thanks! I really appreciate that (I believe those were manual table updates as memory serves). I have a few comments though. Why did you choose to use ISBN-10 linkage with LCCN and NDL? They do not require such and the ISBN space now includes more than just 978 (mostly just some French works but it could change more). I know why it was done for Amazon JP (and the rest of the Amazon sites) since we are linking to their detail pages via ASIN (which is a 10 digit string which happens to match ISBN-10 in the vast majority of cases) instead of their search results (which can sometimes differ if using an ISBN-13). Also as an FYI, the European Library may be the next of our "Other Site" links to get the axe as though their links for the time being still seems to work, they have publicly announced they have shut-down their service as of the end of last year (which so far methinks means they have stopped taking updates but I expect it will go entirely offline at some point). Our BiggerBooks.com linkage (http://www.biggerbooks.com/bk_detail.aspx?isbn=<ISBN-13>) seems to be broken but can be fixed using: http://www.biggerbooks.com/book/<ISBN-13>. Oh and you can probably now kill the text "At least one Amazon site needs to be selected since ISFDB links to Amazon-hosted images." (and the code that ensures that) from mywebsites preferences as we now have image host credit links under the images. Uzume 21:19, 6 April 2017 (UTC)
Good points, I have updated the links. Too bad about the European Library. There isn't much we can do about it aside from keeping an eye on things.
As far as Amazon goes, it was something that Al requested some years ago. I think it may have something to do with Amazon's linking requirements, but I am not sure. Ahasuerus 01:03, 7 April 2017 (UTC)
I imagine the linking requirements are likely met by the added image credit links we implemented sometime back. It does not make much sense to depends on ISBN links when not all such images refer to books that even have ISBNs. Thanks, again. Uzume 07:25, 7 April 2017 (UTC)
I am not 100% sure, but I think Amazon's requirements are more along the lines of "If you link to our image, then you should also link to the book-specific page so that the user could navigate to our site and potentially buy the book." Ahasuerus 13:27, 7 April 2017 (UTC)
If that is true, I image we are in violation for any of our pub records that use such images for publications before 1966 (when SBNs were first created). I did a few searches (is is hard to search for dates before a given date with the web interface; I did not grab the DB and run an SQL query) and there are quite a few (there are more than 300 such from the 1950s alone). Uzume 21:47, 11 April 2017 (UTC)
True, although they are probably not too worried about 40+ year old books. Once we have support for third party identifiers (including ASINs), we will be able to link pre-ISBN books as well. Ahasuerus 15:44, 13 April 2017 (UTC)

Cleanup: Cover art for nongenre books

Discussion moved to R&S. --Vasha 20:36, 5 April 2017 (UTC)

Elizabeth Goudge & Pamela Cleaver

An editor has proposed deleting the non-genre works of Elizabeth Goudge and Pamela Cleaver as both being below the threshold. Given the variety of opinions of that term and the fact that each author has more than a handful of genre works, I'm listing it here for community input. If there is no input within the next day or so, I will accept the deletions. -- JLaTondre (talk) 19:27, 5 April 2017 (UTC)

I think Goudge has enough works to be considered "above the threshold". I'm iffy on Cleaver. ···日本穣 · 投稿 · Talk to Nihonjoe 19:43, 5 April 2017 (UTC)
I'm not sure all of Goudge's works we have listed are truly genre (several of her collections don't look like it from descriptions). -- JLaTondre (talk) 20:53, 5 April 2017 (UTC)
I'm not acquainted with their works, but if Goudge's works aren't what they seem, they should be deleted, as should the nonfiction works by Cleaver. Stonecreek 20:58, 5 April 2017 (UTC)
FantasticFiction says of Goudge: "Elizabeth Goudge was an English author of romance novels, short stories and children's books." Notice that there is no mention of fantasy (so far as I know, she never wrote any SF). I have always thought of her as a borderline speculative fiction author, and that's certainly not what she's known for. SFE lists 6 of her books as genre; we list 34. I'm fairly confident that SFE is closer to the mark than we are.
Pamela Cleaver is a children's author, who teaches writing for children. She views herself as a writer of romances for adults, and spec fic for children. (Her non-fic book on writing for children includes suggestions for how to write science fiction for children.) On her home page "Short Stories" link, essentially all of the content is about her published science fiction. For some of us here, her focus on herself as primarily a spec fic writer would put her "above the threshold". I tend to think of "above the threshold" as being someone who is well-known to the SFF community, has been influential on other authors, or might have SFF fans who collect them. Cleaver would fail on all of these counts. Chavey 03:41, 6 April 2017 (UTC)
The books of Goudge's that the Encyclopedia of Fantasy lists are The White Witch for adults and The Little White Horse, Smoky-House, Henrietta's House, The Valley of Song, and Linnets and Valerians for children. From the descriptions of her other books I have been finding, that seems about right, with the addition of the children's animal fantasy Serena the Hen and the Christmas story The Well of the Star a.k.a. David the Shepherd Boy. However, in some of her books she made heavy use of superstitions and legends as thematic elements or as narrative interludes -- a prominent example being Island Magic where not only is island life vaguely wrapped in myth, but a character's premonition plays a significant role. If we want to include those borderline cases, it would be difficult to figure out which ones are far enough over the border. As for her shorter work, The Fairies' Baby and other stories seems to have consisted either mostly or entirely of fairy stories in the mode of Cicely Mary Barker. She probably wrote quite a few other fantasy short stories; it would be possible to go through this biography and identify some of them. Vasha 17:18, 8 April 2017 (UTC)
Based on responses, I have accepted the proposed deletions. I have also added a note to Goudge's summary page regarding the non-genre issue. Please edit that note if you have suggestions on better wording. -- JLaTondre (talk) 18:26, 8 April 2017 (UTC)

Is Hermann Hesse really genre?

I wouldn't consider him in that way and thus delete his non-genre works. The majority of his works seems to lay outside our boundaries, but I'll appreciate any input before taking any action. Thanks for your thoughts. Stonecreek 20:48, 5 April 2017 (UTC)

Well, he is in the Encyclopedia of Fantasy with several paragraphs of discussion. Clute dicusses Steppenwolf as a work of fantasy, and others of his novels as marginally fantasy. And The Glass Bead Game is a famous utopia or conte philosophique. --Vasha 21:23, 5 April 2017 (UTC)
I don't think of him as a genre author, but SFE3 says:
  • [Demian, Siddhartha and Der Steppenwolf] in which Jungian depth psychology, Indian mysticism and Weltschmerz are perhaps overpalatably combined; these and others of his novels can be read – unwisely – to emphasize any fantasy elements, for at their core they are meditations on transcendence. ... [collections] assemble from various sources much of his short work, mostly fantasy, though some parable-like sf venues are evoked, not unpalely.
so we have to be careful. I think it would be best to keep his borderline SF works, but it would also be helpful to add notes explaining what kinds of speculative elements they contain. Ahasuerus 21:25, 5 April 2017 (UTC)
Obviously, I should better have asked if he's above the threshold, and to decide if we should keep his non-genre works. I'm asking because I'm reading through a copy of Knulp, which 1) is a COLLECTION in the first place, and 2) has no speculative content so far (after two of the three items - I'm guessing there'll show up no more).
At least over here there are dozens of other COLLECTIONS, OMNIBUSES & ANTHOLOGIES that feature one or the other of his works. Would we really like to catalogue all of them, even if there's no spec. fic. in them? Stonecreek 03:58, 6 April 2017 (UTC)
Although The Glass Bead Game is clearly genre, that is AFAIK, the only truly genre book by Hesse. Steppenwolf and Siddhartha are spiritual journeys where the visions the characters see are primarily spiritual visions, not the kind we associate with fantasy. In some cases, such as Craig Strete's "A Sunday Visit with Great-Grandfather", the spiritual vision is clearly meant by the author to be fantasy. But that's not Hesse's intent in these novels. It's fair to call them borderline genre, but ONE true genre book doesn't make Hesse a genre writer. Chavey 04:12, 6 April 2017 (UTC)
Thanks, Darrah! That's congruent with what the various secondary sources state. As for his shortfiction: quite a long time ago, I read through his collected stories, and I remember that there were only a few fairy tales, but the vast majority of his shortfiction was clearly non-genre. Stonecreek 15:14, 6 April 2017 (UTC)

Additional cleanup reports made available to all editors - 2017-04-05

Even more cleanup reports have been made available to all editors. Enjoy! Ahasuerus 21:14, 5 April 2017 (UTC)

"Other Sites" changes

I have made some experimental changes to the "Other Sites" section based on our discussion the other day. The links are now hidden by default. Users can access them by moving the cursor to the words "Other Sites", which have a big down arrow next to them. The rest of the functionality, including the ability to select and deselect sites on the My Web Sites page, remains the same.

Looking at the results, it occurs to me that we could move the list next to the "ISBN" field. The only reason we originally added it to the navigation bar was that we didn't want to clutter the main area. However, now that the list is hidden by default, it should no longer be a problem. We'll just display the words "Other Sites" and an arrow to let the user know that more information is available. Ahasuerus 00:00, 7 April 2017 (UTC)

Those are some interesting changes. Even if the external ISBN based links disappeared for some users (e.g., non-JavaScript capable ones), I would content they are not significant site content but just ease of use content (valuable but not significant). On the other hand, if links between pubs and titles etc. were not accessible by all users (regardless of their browser capabilities), I would consider the site broken (because they are more than just valuable they significantly define what we are about). Uzume 07:34, 7 April 2017 (UTC)
I should clarify that the changes are CSS-based and make no use of JavaScript. Even if you disable JavaScript for isfdb.org, your browser should display the page correctly -- as long as it supports CSS 2. Ahasuerus 13:24, 7 April 2017 (UTC)
I like the change. However, the links of the opened "Other Sites" menu now all have a bold font and the menu looks quite crammed as a result. I think it'd look better if they had no bold font-weight like the links in the other sections do. Jens Hitspacebar 18:56, 7 April 2017 (UTC)
Oh, right. Sorry, I didn't notice the bolding and the bullet change. It should be fixed now. Ahasuerus 19:42, 7 April 2017 (UTC)
This thing works as a charm on things where you can hover (aka my laptop) but there is no way to open them from my IPhone where hovering is a bit... impossible AFAIK (and I suspect other touchscreen may have similar issues). Annie 19:46, 7 April 2017 (UTC)
Hovering works, at least on my Fairphone (Android 4). Just tap on "Other sites" and the section opens. Close it by tapping on an area outside the "Other sites" section. Jens Hitspacebar 19:57, 7 April 2017 (UTC)
Well - I did not post without trying :) Not the same on Iphone 5s apparently :) Single tap does nothing; double tap is zoom in/out. No way to open the menu. Annie 20:01, 7 April 2017 (UTC)
(after edit conflict)Oh. I didn't occur to me that touchscreen devices didn't support hovering. I have installed a fix which is supposed to help, but I can't test it since I don't have an internet-capable touchscreen device at the moment. Could you please give it another shot to see if it worked? Ahasuerus 20:02, 7 April 2017 (UTC)
Still does not work here. Unfortunately I have the Iphone 5s only here so cannot try on another one Annie 20:10, 7 April 2017 (UTC)
Maybe it's a browser (Safari) problem and a different browser might help? I'm using Firefox on my phone but also just tried Android's stock browser (which is Google Chrome I think) and it works there too. Jens Hitspacebar 20:15, 7 April 2017 (UTC)
Apparently Apple has a special way of handling hovering. I have installed yet another fix -- hopefully Annie will be able to test it later today. Ahasuerus 20:33, 7 April 2017 (UTC)
Yeah, it is the built-in Safari. Still does not work on it though. :( Wonder if it is not because it is an older phone (although its OS is upgraded to 10.2.1 - I am seeing an upgrade available though so let me try to upgrade that and add another browser eventually and then we can keep troubleshooting. Will take awhile though - I am on my way to Europe in a few hours and planes are not a good place to play with your phone ;) Annie 21:01, 7 April 2017 (UTC)
Bummer. Oh well, one patch at a time... Ahasuerus 21:14, 7 April 2017 (UTC)
No hurry but to align with our templates we could also add the British Library to the national libraries we link to. See details here. Uzume 18:52, 12 April 2017 (UTC)

Smashword images

I image the answer will be "no" but for some reason I feel the need to throw it out there and ask this anyway. We have rights to deep link images from certain sites as enumerated here: Template:PublicationFields:ImageURL. Among them is Amazon. They grant such a right to their affiliates of which ISFDB is one. Smashwords seems to link it cover images from the DNS space of cloudfront.net. Can we use such image URLs since apparently cloudfront.net belongs to Amazon (as evidenced by this set of whois records)? It would certainly make adding the indie ebooks Smashwords publishes easier to add (they seem to allocate ISBNs to their ebooks unlike Amazon Kindle). Uzume 07:49, 7 April 2017 (UTC)

Cloudfront is a content delivery service sold by Amazon. Smashwords still owns the content, not Amazon, and they pay Amazon by the amount of data accessed. While a minor cost, Smashwords would still end up footing the bill if we linked to them. -- JLaTondre (talk) 10:59, 7 April 2017 (UTC)
Yes, I did find Amazon CloudFront. Uzume 13:12, 8 April 2017 (UTC)

Cleanup reports - performance

We have a lot of nightly cleanup reports. They are kicked off at 1am server time and take around 20-30 minutes to complete. While the reports are running, the system is sporadically unresponsive.

Not all reports are created equal. Here is how long each one takes to compile on the development server:

  • Report 1 took 9.875000 seconds to compile
  • Report 2 took 9.329000 seconds to compile
  • Report 3 took 6.177000 seconds to compile
  • Report 4 took 0.390000 seconds to compile
  • Report 5 took 0.515000 seconds to compile
  • Report 6 took 0.156000 seconds to compile
  • Report 7 took 0.484000 seconds to compile
  • Report 8 took 3.401000 seconds to compile
  • Report 9 took 0.639000 seconds to compile
  • Report 11 took 1.794000 seconds to compile
  • Report 12 took 0.078000 seconds to compile
  • Report 13 took 0.078000 seconds to compile
  • Report 14 took 2.403000 seconds to compile
  • Report 15 took 1.513000 seconds to compile
  • Report 16 took 17.893000 seconds to compile
  • Report 17 took 0.764000 seconds to compile
  • Report 18 took 0.734000 seconds to compile
  • Report 19 took 0.390000 seconds to compile
  • Report 20 took 3.385000 seconds to compile
  • Report 21 took 1.435000 seconds to compile
  • Report 22 took 0.094000 seconds to compile
  • Report 23 took 0.093000 seconds to compile
  • Report 24 took 0.421000 seconds to compile
  • Report 25 took 0.000000 seconds to compile
  • Report 26 took 0.000000 seconds to compile
  • Report 27 took 0.094000 seconds to compile
  • Report 28 took 0.094000 seconds to compile
  • Report 29 took 1.045000 seconds to compile
  • Report 30 took 0.281000 seconds to compile
  • Report 31 took 0.359000 seconds to compile
  • Report 32 took 4.461000 seconds to compile
  • Report 33 took 21.310000 seconds to compile
  • Report 34 took 11.045000 seconds to compile
  • Report 35 took 0.234000 seconds to compile
  • Report 36 took 1.154000 seconds to compile
  • Report 37 took 0.764000 seconds to compile
  • Report 38 took 22.989000 seconds to compile
  • Report 39 took 0.218000 seconds to compile
  • Report 40 took 2.077000 seconds to compile
  • Report 41 took 1.326000 seconds to compile
  • Report 42 took 3.120000 seconds to compile
  • Report 43 took 0.296000 seconds to compile
  • Report 45 took 3.292000 seconds to compile
  • Report 46 took 0.755000 seconds to compile
  • Report 47 took 43.652000 seconds to compile
  • Report 48 took 3.246000 seconds to compile
  • Report 49 took 1.591000 seconds to compile
  • Report 50 took 1.274000 seconds to compile
  • Report 51 took 0.203000 seconds to compile
  • Report 52 took 35.721000 seconds to compile
  • Report 53 took 0.093000 seconds to compile
  • Report 54 took 8.672000 seconds to compile
  • Report 55 took 0.843000 seconds to compile
  • Report 56 took 0.265000 seconds to compile
  • Report 57 took 0.234000 seconds to compile
  • Report 58 took 1.903000 seconds to compile
  • Report 59 took 1.872000 seconds to compile
  • Report 60 took 1.888000 seconds to compile
  • Report 61 took 1.887000 seconds to compile
  • Report 62 took 0.749000 seconds to compile
  • Report 63 took 3.389000 seconds to compile
  • Report 64 took 1.389000 seconds to compile
  • Report 65 took 0.579000 seconds to compile
  • Report 66 took 0.141000 seconds to compile
  • Report 67 took 0.670000 seconds to compile
  • Report 68 took 3.885000 seconds to compile
  • Report 69 took 38.446000 seconds to compile
  • Report 70 took 12.942000 seconds to compile
  • Report 71 took 0.702000 seconds to compile
  • Report 72 took 0.218000 seconds to compile
  • Report 73 took 0.016000 seconds to compile
  • Report 74 took 0.842000 seconds to compile
  • Report 75 took 0.266000 seconds to compile
  • Report 76 took 0.015000 seconds to compile
  • Report 77 took 0.000000 seconds to compile
  • Report 78 took 0.078000 seconds to compile
  • Report 79 took 3.370000 seconds to compile
  • Report 80 took 20.749000 seconds to compile
  • Report 81 took 0.031000 seconds to compile
  • Report 82 took 0.000000 seconds to compile
  • Report 83 took 0.421000 seconds to compile
  • Report 84 took 226.592000 seconds to compile
  • Report 85 took 0.115000 seconds to compile
  • Report 86 took 2.371000 seconds to compile
  • Report 87 took 5.320000 seconds to compile
  • Report 88 took 34.535000 seconds to compile
  • Report 89 took 0.655000 seconds to compile
  • Report 90 took 0.000000 seconds to compile
  • Report 91 took 0.718000 seconds to compile
  • Report 92 took 4.883000 seconds to compile
  • Report 93 took 11.871000 seconds to compile
  • Report 94 took 1.092000 seconds to compile
  • Report 95 took 1.077000 seconds to compile
  • Report 96 took 0.000000 seconds to compile
  • Report 97 took 4.691000 seconds to compile
  • Report 98 took 0.062000 seconds to compile
  • Report 99 took 36.972000 seconds to compile
  • Report 100 took 0.374000 seconds to compile
  • Report 101 took 0.265000 seconds to compile
  • Report 103 took 163.226000 seconds to compile
  • Report 104 took 0.000000 seconds to compile
  • Report 105 took 0.117000 seconds to compile
  • Report 106 took 0.004000 seconds to compile
  • Report 107 took 5.184000 seconds to compile
  • Report 108 took 0.108000 seconds to compile
  • Report 109 took 0.216000 seconds to compile
  • Report 110 took 0.002000 seconds to compile
  • Report 111 took 16.063000 seconds to compile
  • Report 112 took 0.289000 seconds to compile
  • Report 113 took 0.019000 seconds to compile
  • Report 115 took 0.681000 seconds to compile
  • Report 116 took 0.000000 seconds to compile
  • Report 119 took 0.413000 seconds to compile
  • Report 120 took 0.000000 seconds to compile
  • Report 121 took 0.016000 seconds to compile
  • Report 122 took 0.015000 seconds to compile
  • Report 123 took 0.546000 seconds to compile
  • Report 124 took 1.014000 seconds to compile
  • Report 125 took 1.014000 seconds to compile
  • Report 129 took 0.999000 seconds to compile
  • Report 126 took 1.014000 seconds to compile
  • Report 127 took 1.029000 seconds to compile
  • Report 128 took 0.999000 seconds to compile
  • Report 130 took 1.061000 seconds to compile
  • Report 131 took 0.998000 seconds to compile
  • Report 132 took 0.998000 seconds to compile
  • Report 133 took 0.998000 seconds to compile
  • Report 134 took 0.999000 seconds to compile
  • Report 135 took 0.998000 seconds to compile
  • Report 136 took 1.014000 seconds to compile
  • Report 137 took 1.030000 seconds to compile
  • Report 138 took 1.077000 seconds to compile
  • Report 139 took 1.092000 seconds to compile
  • Report 140 took 1.076000 seconds to compile
  • Report 141 took 1.934000 seconds to compile
  • Report 142 took 1.139000 seconds to compile
  • Report 143 took 1.778000 seconds to compile
  • Report 144 took 1.077000 seconds to compile
  • Report 145 took 0.640000 seconds to compile
  • Report 146 took 19.422000 seconds to compile
  • Report 147 took 0.203000 seconds to compile
  • Report 148 took 19.266000 seconds to compile
  • Report 149 took 19.235000 seconds to compile
  • Report 150 took 19.281000 seconds to compile
  • Report 151 took 27.362000 seconds to compile
  • Report 152 took 19.204000 seconds to compile
  • Report 153 took 19.344000 seconds to compile
  • Report 154 took 19.282000 seconds to compile
  • Report 155 took 19.234000 seconds to compile
  • Report 156 took 19.328000 seconds to compile
  • Report 157 took 19.298000 seconds to compile
  • Report 158 took 19.360000 seconds to compile
  • Report 159 took 19.297000 seconds to compile
  • Report 160 took 19.266000 seconds to compile
  • Report 161 took 8.315000 seconds to compile
  • Report 162 took 19.532000 seconds to compile
  • Report 163 took 19.500000 seconds to compile
  • Report 164 took 19.453000 seconds to compile
  • Report 165 took 19.609000 seconds to compile
  • Report 166 took 19.469000 seconds to compile
  • Report 167 took 29.827000 seconds to compile
  • Report 168 took 2.995000 seconds to compile
  • Report 169 took 0.687000 seconds to compile
  • Report 170 took 0.671000 seconds to compile
  • Report 171 took 0.686000 seconds to compile
  • Report 172 took 0.687000 seconds to compile
  • Report 173 took 0.686000 seconds to compile
  • Report 174 took 0.686000 seconds to compile
  • Report 175 took 0.671000 seconds to compile
  • Report 176 took 0.686000 seconds to compile
  • Report 177 took 0.686000 seconds to compile
  • Report 178 took 0.687000 seconds to compile
  • Report 179 took 0.686000 seconds to compile
  • Report 180 took 0.687000 seconds to compile
  • Report 181 took 0.671000 seconds to compile
  • Report 182 took 0.686000 seconds to compile
  • Report 183 took 0.718000 seconds to compile
  • Report 184 took 0.827000 seconds to compile
  • Report 185 took 0.764000 seconds to compile
  • Report 186 took 1.310000 seconds to compile
  • Report 187 took 0.952000 seconds to compile
  • Report 188 took 1.482000 seconds to compile
  • Report 189 took 0.094000 seconds to compile
  • Report 190 took 0.015000 seconds to compile
  • Report 191 took 4.602000 seconds to compile
  • Report 192 took 1.139000 seconds to compile
  • Report 193 took 23.541000 seconds to compile
  • Report 194 took 0.624000 seconds to compile
  • Report 195 took 0.733000 seconds to compile
  • Report 196 took 3.416000 seconds to compile
  • Report 197 took 3.417000 seconds to compile
  • Report 198 took 0.312000 seconds to compile
  • Report 200 took 4.333000 seconds to compile
  • Report 201 took 0.009000 seconds to compile
  • Report 202 took 0.016000 seconds to compile
  • Report 204 took 7.272000 seconds to compile
  • Report 205 took 0.001000 seconds to compile

One thing to keep in mind is that not all reports are run in the numeric order, e.g. report 180 may be run after report 200.

Here are some things that we could do to minimize the impact of these reports on the system:

  1. Optimize the reports to take less time to run. Although it may be possible to improve some reports significantly, it will take a fair amount of development time to tweak the code.
  2. Eliminate some reports. Once we wrap up the Wiki migration project and break the "lexical match" links between the database and the Wiki, we should be able to disable a few dozen reports.
  3. Move the most resource-intensive reports to a weekly schedule. The problem with this approach is that some logically connected reports, e.g. certain reports that facilitate the Wiki migration project, will end up on different schedules.
  4. Change the frequency of the "nightly job" to run once a week or perhaps twice a week.

My current thinking is that we could start with approach #4, then work on #1. #2 will happen a few months down the road once the Wiki migration projects has been completed.

Does this sound like a reasonable approach? If so, should we run the reports weekly or twice a week? Ahasuerus 18:02, 7 April 2017 (UTC)

Is there a way for the report to be run on the development server and just posted on the live one (update data, run report, post back on the live site)? Annie 19:23, 7 April 2017 (UTC)
It's technically possible, but, unfortunately, it's not feasible given how the development server works. Ahasuerus 19:47, 7 April 2017 (UTC)
Too bad :( It was just an idea. Annie 20:05, 7 April 2017 (UTC)
If not - then the above approach sounds like the most logical one. Weekly should be fine I think - Sunday is not a very busy day from what I had seen in terms of edits so it may be a good day for them :) Or you can run some statistics and find a calm time. Annie 19:23, 7 April 2017 (UTC)
6 PM US eastern time? Americans are making or eating dinner, Europeans are going to bed. --Vasha 23:08, 7 April 2017 (UTC)
Currently I experience issues with the site daily at around 6:30 am US/Pacific (currently daylight time so UTC-7) for about five minutes. Uzume 02:24, 8 April 2017 (UTC)
That's when the daily backups run. As you said, they take under 5 minutes, which isn't too bad. The nightly job takes over 20 minutes and is getting to be a nuisance. Ahasuerus 02:31, 8 April 2017 (UTC)
What about spacing them out? Even over a couple of days? Since many are small, that should reduce the perceived impact. If it goes to weekly, can the clean-up report page be changed so that each report button shows the remaining number of issues (and perhaps even hide the button if zero left)? That would be much more convenient than having to click each one to see if they have been resolved yet or not. -- JLaTondre (talk) 12:24, 8 April 2017 (UTC)
I was thinking of having a message at the top of the reports that say something like "X hours until report is regenerated" or similar. This would become useful if the reports are spaced out and/or run at different times (like some weekly, etc.). Uzume 13:08, 8 April 2017 (UTC)
If you send me pointers/source for the long-running reports, I'd be happy to take a look at them. --MartyD 12:27, 8 April 2017 (UTC)
Great! The majority of cleanup reports are in this module. Transliterations-related reports are in this module. Each query has a comment like "Report 85: Non-Latin authors with Latin characters in legal names", so it should be reasonably straightforward (hopefully.) BTW, I suspect that the main culprit is "and not exists". Ahasuerus 14:33, 8 April 2017 (UTC)

(unindent) I am happy to report that thanks to Marty's mad SQL skills 4 of the worst offenders have been brought to heel! :-) Ahasuerus 03:27, 16 April 2017 (UTC)

Great work: nightly_wiki.py nightly_update.py Uzume 19:33, 18 April 2017 (UTC)

Problem with Template:Author Image Data

It doesn't link correctly to Japanese names (such as 村田蓮爾). See here and try the link in the template. It will send you to this page, which is an invalid page. Somehow, it's not passing the correct escape characters for the kanji to Template:A. If the database could be adjusted to allow UTF-8 characters in the URL, that would correct the problem as it wouldn't need to use escape characters. There may be other ways to fix it, too. ···日本穣 · 投稿 · Talk to Nihonjoe 04:59, 8 April 2017 (UTC)

This is a problem in the Wikimedia software (we use a very old version) on the wiki side; not in the database. I have updated Template:Author Image Data to have a NameId field that can be provided in addition to the Name. If present, it will use that to link which bypasses the problem. I updated Image:Range Murata.jpg as an example. -- JLaTondre (talk) 12:39, 8 April 2017 (UTC)
This is an issue with Template:A (which Template:Author Image Data uses) and the fact that it is using anchorencode and not urlencode (see mw:Help:Magic words). We could use urlencode but as JLaTondre stated we are using a very old version of MediaWiki (see Special:Version) and ea currently requires spaces in names to be converted to underscores (not pluses as our old version of urlencode supports). A further issue is that the wiki uses UTF-8 while the DB application still uses ISO-8859-1. To properly link to the DB with an author name string would requires one to first XML encode the Unicode and then URL encode. Currently there is no viable option available. Probably the best solution would upgrade MediaWiki (so we can use urlencode with WIKI argument (so spaces are converted to underscores) and upgrade the DB application to serve content in UTF-8. Until then we continue to use the anchorencode hack for ASCII linking and can work around the issue for Unicode via author record ID linking instead. Uzume 13:58, 8 April 2017 (UTC)
There is also a possibility with updating the interwikimap and using interwiki linking syntax (which should do "WIKI" style urlencoding with spaces as underscores even in this older MediaWiki version) but we would still have to handle the Unicode issue. Currently the database tables have not been updated to handle Unicode (in any encoding) and we are using XML entities all the way into the database itself so the DB actually contains &#26449;&#30000;&#34030;&#29246; for 村田蓮爾 and the DB application just depends on the browser to handle the XML entity encoding (displaying and in sending via forms). We could serve UTF-8 HTML in the web application with a large amount of conversion into and out of the DB or do a single huge conversion and change the tables and all the table data to UTF-8. Uzume 14:31, 8 April 2017 (UTC)
Yeah, we really need to figure out how to update to the most recent version of the Wiki software. Too bad Al doesn't really do anything anymore. ···日本穣 · 投稿 · Talk to Nihonjoe 18:40, 9 April 2017 (UTC)
I have looked into this before and would be happy to do the work but I do not have access to the server to deploy it even if I get to the point where I have tested it extensively locally first. I believe we would have to make incremental jumps as there have been a number of changes along the way. The first minor jump would be to move to 1.12.0 (and get off the 1.12.0rc1). Next, I would likely target the last 1.12: 1.12.4. Then I can start looking at what it takes to move to considerably newer versions. Uzume 03:48, 10 April 2017 (UTC)
Well, it's up to 1.28.1 as of just a couple days ago, so we've got a ways to go. ···日本穣 · 投稿 · Talk to Nihonjoe 05:32, 10 April 2017 (UTC)
Yes, but I researched it once and I believe with a little planning and testing, some larger jumps can be safely made. Some will certainly require some tweaking before we consider taking them on however. FYI: in recent times they have made more releases so the number has moved more (not all releases are equally large in changes). Uzume 21:22, 11 April 2017 (UTC)
Well, maybe you and Ahasuerus can work together to do some testing and whatnot. I know a little, but I've never done it on a system with this much entered into it. ···日本穣 · 投稿 · Talk to Nihonjoe 21:28, 11 April 2017 (UTC)

Primary verifications -- work in progress

Please note that I am in the middle of redoing primary verifications. The first few patches should have no impact on the way the data is displayed, but if you see anything unusual (aka a bug), please let me know. Ahasuerus 00:12, 9 April 2017 (UTC)

The demise of SFF Net

As some of you probably know, SFF Net shut down on March 31. Our author pages have some 150+ links to the author bios which used to be hosted by SFF Net and are now defunct. We will need to re-link them to the last known snapshot taken by the Wayback Machine. For example, this page is a snapshot of Kevin O'Donnell, Jr.'s biography. Looking for volunteers! Ahasuerus 02:49, 10 April 2017 (UTC)

I can do it after this week. Some of them may have found new hosts for their sites, too, so each one would need to be researched to make sure. ···日本穣 · 投稿 · Talk to Nihonjoe 05:34, 10 April 2017 (UTC)
That is definitely true. I already updated some (even replacing them in cases where they obviously did move to different hosting; the ones that are now dead are somewhat of a bigger problem) but I am not sure I made much a dent in the list. Uzume 21:24, 11 April 2017 (UTC)

Variant Reviews

I recently noticed author pages do a strange things to variant review titles when the reviewed authors are cited differently but the reviewing authors are cited the same. Here is an example: T1956242 and T1282608 on A123788 (both are reviewed by "Edward Cox" and review the title "The City of a Thousand Gods" but in one the authors are "Marge B. Simon and Malcolm Deeley" and in the other "Marge Simon and Malcolm Deeley"). You will note that T1281964 looks good showing the review as having been reprinted. Uzume 13:58, 12 April 2017 (UTC)

Since Marge B. Simon is a pseudonym of Marge Simon, shouldn't the VT direction of the two reviews be reversed? Ahasuerus 15:12, 17 April 2017 (UTC)
It looks like someone merged them (which is probably not really a problem since it only changes the how the review credits the author or the reviewed work and not the actual reviewer), however the display in T1281964 is a little strange listing the review and that it is reprinted and then also listing the reprint below that. Uzume 17:41, 17 April 2017 (UTC)
Yes, it looks like a bug. I have created Bug 654 to record the problem and will take a closer look later. Thanks! Ahasuerus 18:03, 17 April 2017 (UTC)

Defining "Publisher" in a self-publishing world

I believe there has been some discussion on what constitutes a publisher in the past and I thought I would bring it up in light of self-publishing becoming more common due to the proliferation of on-demand printers, ebook distributors, etc. (e.g., CreateSpace, Kindle Direct Publishing Smashwords/Smashwords, etc.) Many of these seem to masquerader as publishers but what really is a publisher? In my mind, a publisher is just a literary publicist which happens to have a somewhat special place in that they are historically listed on the copyright page and thus can be used to identify publications. Part of this process is the definition of a literary product or edition and allocation of an ISBN (a standardized literary product/edition identifier).

I see CreateSpace (an Amazon company) as an on-demand printer (or on-demand fabricator; remember they started with DVDs as "CustomFlix"). Kindle Direct Publishing (though they have "publishing" in their name) does not directly publish anything (though they do distribute) as they are more of an online publisher's toolkit and actual publication is directed by the publishers that use them (much as a traditional publisher can choose among printers). Smashwords is similar (although they seem to require an ISBN vs. KDP allocating an ASIN).

I would like to suggest updates to Template:PublicationFields:Publisher to clarify self-publishing and exclude known on-demand printers, ebook distributors, etc. such as CreateSpace and Smashwords (and perhaps we can work on emptying CreateSpace and Smashwords). I would also go so far as to say I believe that in general (there may be some exceptional cases) we should not have "CreateSpace" or "Smashwords" in the publisher name at all (though a publisher comment that a specific publisher seems to publish exclusively via their services seems appropriate). Unless these are actually related imprints, this means renaming publishers such as Peculiar Productions / CreateSpace, Pink Fox Publications & CreateSpace and The Chakat's Den / CreateSpace and merging publishers such as Chimericana Books / Smashwords and Triskell Press and Smashwords. We could also consider our publisher records that contain the words "Amazon" or "Kindle" (though a brief look tells me that world might be a bit more complex). Thank you for your feedback, Uzume 18:43, 12 April 2017 (UTC)

Discussion on this started awhile back at Rules_and_standards_discussions#What counts as a publication for an e-book?, but no clear resolution. -- JLaTondre (talk) 19:35, 12 April 2017 (UTC)
I believe that one outcome of that discussion was to agree that CreateSpace should not be used as a publisher name. Converting existing ones will be quite a project though. And it hasn't really been agreed what to convert them to. I've been seeing some cases of the publisher space in such cases being left blank; that may not be a good idea because it doesn't make clear whether the blank means "self-published" or "unknown". As for using the author's name, that's clearly the thing to do if Joe Blow has (like some people do) set up a company named "Joe Blow, LLC". But do we want to be able to distinguish between such cases and people who are just self-publishing without a company? Vasha 20:56, 12 April 2017 (UTC)
Re: it doesn't make clear whether the blank means "self-published" or "unknown", that's what Fixer does at this time. There is just no way of telling without using Look Inside and even that is not always definitive because it only shows some of the pages. Even in the best case scenario, i.e. when all relevant pages are visible, a book can be clearly self-published yet the publisher is unknown. It's been a royal pain... Ahasuerus 23:24, 12 April 2017 (UTC)
Well, I guess the blank does mean "unknown" or "not certain what to say", then. I suppose I would rather have blanks that can maybe be filled in at some point than have the space filled by something wrong or misleading. --Vasha 23:31, 12 April 2017 (UTC)
I agree. I would rather see a field unpopulated than with some special "unknown" value. That said, I agree with "uncredited" (when it is known and it can even be a variant if we later learn who really did it; "unknown" would not make any sense in such a context). Uzume 23:49, 12 April 2017 (UTC)
If we're going to try to "define" a publisher, I'd like to suggest that we try for a definition that applies to how fanzines were published as well. The most common scenario is the common "self-publishing" model, where the editor made up the masters, stuck them in a machine, and turned the crank themselves. Sometimes they gave themselves and their living room a fancy name (which I think we should list). Sometimes they had a friend run off all the copies for them (occasionally, mailing the masters to a friend in another state, who took over the work). Sometimes it was a clubzine, so they took the masters to the club machine, and other club members helped duplicate, collate, staple, etc., and hence the club is often the "publisher". But in many of these cases, the "publisher" of record is often doing nothing more than the self-published "publisher": The fanzine editor has done all the work except for the duplication, and the "publisher" has done nothing but making copies of the thing they were handed. Yet that club, or that friend with a Gestetner, is still listed as the publisher. Chavey 05:17, 13 April 2017 (UTC)

Note templates and abbreviations

It has been suggested that the new note templates be aligned on what is displayed. Currently we display a mix of abbreviations/acronyms and full length name/descriptions. One option that has been discussed is to use abbreviations/acronyms with full length name/descriptions in title attributes to create tooltip popups. Regardless whether we decide to go with the technical solution to implement both or not, we should standardize the naming. Here is a grouped list of current templates:

LinkTemplate Label
Internal A
Publisher
External ASIN ASIN
ASIN-UK ASIN-UK
BL British Library
BNB British National Bibliography
BNF Bibliothèque nationale de France
DNB Deutsche Nationalbibliothek
FantLab-authorFantLab author
FantLab-pubFantLab publication
FantLab-title FantLab title
JNB Japanese National Bibliography
OCLC OCLC
LCCN LCCN
NDL National Diet Library
SFBG-pub SFBG publication
SFBG-publisher SFBG publisher
SFBG-title SFBG title
Specific Internal Bleiler1 Science-Fiction: The Gernsback Years by Everett F. Bleiler and Richard J. Bleiler, 1998
Bleiler78 The Checklist of Science-Fiction and Supernatural Fiction by E. F. Bleiler, 1978
Clute/Grant The Encyclopedia of Fantasy, eds. John Clute and John Grant, 1997
Clute/Nicholls The Encyclopedia of Science Fiction, 2nd edition, eds. John Clute and Peter Nicholls, 1993
Currey Science Fiction and Fantasy Authors: A Bibliography of First Printings of Their Fiction and Selected Nonfiction by L. W. Currey, 1979
Miller/Contento Science Fiction, Fantasy, & Weird Fiction Magazine Index (1890-2007) by Stephen T. Miller and William G. Contento
Reginald1 Science Fiction and Fantasy Literature: A Checklist, 1700-1974 by Robert Reginald, 1979
Reginald3 Science Fiction and Fantasy Literature 1975 - 1991 by Robert Reginald, 1992
Tuck The Encyclopedia of Science Fiction and Fantasy through 1968 by Donald H. Tuck, 1974-1982
Specific External Contento Index to Science Fiction Anthologies and Collections, Combined Edition, William G. Contento
FantLab FantLab
Locus1 The Locus Index to Science Fiction
SFBG SFBG
SFE3 SFE3
None Tr Translated by

Notice things are far from balanced with some being labelled extremely tersely and other extremely verbose.

If we decide to adopt the tooltip approach, we could either add a new field (to support both label and popup title) or instead directly use the template name as the label and expand and use the current label at the popup title. The former requires extra code. The latter is problematic for some existing templates (though we could ignore templates without labels some are still an issue, e.g., "Tr" which is not really useful as a label).

By the way, why do we have an SFE3 and not an SFE3-entry? We could perhaps also do with a corollary FE (and FE-entry?) for linking to the Digital Edition of the Encyclopedia of Fantasy at: http://sf-encyclopedia.uk/fe.php

Thank you, Uzume 21:18, 12 April 2017 (UTC)

I am afraid I was so busy implementing template tool tips that I missed this section when it was posted 1.5 hours ago. I hope the current implementation addresses the issues that you raised in the second to last paragraph. I am not sure what you mean by "not an SFE3-entry". Ahasuerus 23:05, 12 April 2017 (UTC)
I was referring to something like:
  • 'SFE3-entry': ('http://www.sf-encyclopedia.com/entry/%s', 'SFE3'),
  • {{SFE3-entry|norman_john}}
to refer to the SFE3 "Norman, John" entry. I shall have to examine and update table based on the new changes. Uzume 23:42, 12 April 2017 (UTC)
Oh, I see. Yes, I agree that it would be desirable. The only catch is that SFE3's linking values are not the same as their display values, e.g. in this case it's "norman_john" vs. "John Norman". I'll have to revamp the template code to support this type of dichotomy. 00:32, 13 April 2017 (UTC)
Yes, if one wants to have a different display value for the relative URL (or identifier vs. the label and tooltip). I am not saying that is necessary in this case but it could be useful (and adding that kind of support would allow internal linking using record numbers while displaying names or titles, etc. too vs. how we currently link with A and Publisher using string match/searching). Uzume 06:37, 13 April 2017 (UTC)

Template tool tips

Templates have been changed to support tool tips. For example, if you hover over "SFE3" on Chris Foss's Summary page, you will see the full name used by the SFE3 project. If you hover over the templates on this page, you will see what "OCLC" and "LCCN" stand for. Some of the longer displayed template names have been changed to template codes.

It would be nice to make it clear that these tool tips exist, but I am not sure what the best way to do it would be. Ahasuerus 22:59, 12 April 2017 (UTC)

Are you referring to something akin to how "class" "hint" (for cursor selection) and the question mark icon are employed for transliterations? Some style could be applied. Uzume 06:43, 13 April 2017 (UTC)
That's right. I just haven't been able to come up with a unobtrusive yet clear way to indicate that the links are "hoverable". Ahasuerus 15:45, 13 April 2017 (UTC)
Maybe a little CSS like:
abbr a:link, abbr a:visited {text-decoration-line: underline; text-decoration-style: dotted;}
which would mimic most browsers abbr default styling, with a
abbr a:hover, abbr a:active {text-decoration-line: underline; text-decoration-style: solid;}
to indicate it's still a link, although users will still get the cursor change on hover as well. Albinoflea 17:43, 13 April 2017 (UTC)
Thanks, I'll experiment later today. Ahasuerus 18:14, 13 April 2017 (UTC)
I noticed your markup code is a bit strange. Using the examples you mentioned above I see things like
  • <abbr title="Third Edition of the Encyclopedia of Science Fiction"><a href="http://www.sf-encyclopedia.com/">SFE3</a></abbr>
  • <abbr title="Library of Congress Control Number"><a href="https://lccn.loc.gov/2014658071">LCCN 2014658071</a></abbr>
  • <abbr title="WorldCat/Online Computer Library Center"><a href="http://www.worldcat.org/oclc/806016149">OCLC 806016149</a></abbr>
This seems inverted. Should it not be something more like:
  • <a href="http://www.sf-encyclopedia.com/"><abbr title="Third Edition of the Encyclopedia of Science Fiction">SFE3</abbr></a>
  • <a href="https://lccn.loc.gov/2014658071"><abbr title="Library of Congress Control Number">LCCN</abbr> 2014658071</a>
  • <a href="http://www.worldcat.org/oclc/806016149"><abbr title="WorldCat/Online Computer Library Center">OCLC</abbr> 806016149</a>
With the current method, you are putting everything in an "a" tag inside of an "abbr" tag. If you are putting everything in, it almost begs the question why bother with the "abbr" tag (not to mention the identifier is not part of the actual abbreviation is it?)? On the other hand, if you put the "abbr" tag just around the abbreviated label that makes more sense (you could also link only the identifier but I am not sure that is for the best; besides in some cases like "SFE3" there isn't one). Uzume 23:13, 13 April 2017 (UTC)
As for style, why not something like abbr[title] {cursor:help} abbr[title]:after {content: url('/question_mark_icon.gif')}? Uzume 23:13, 13 April 2017 (UTC)
The reason I wrapped the whole link in the "attr" tag is that I wanted the mouse-over functionality to be available regardless of which part of the link the mouse hovered over. However, you are right that it defeats the original purpose of the "attr" tag. It's probably better to put the hover-over/"title" functionality inside of a "span". Ahasuerus 00:22, 14 April 2017 (UTC)
If you want the tooltip on the entire thing why not put it directly on the "a" tag link (no need to add an extra "span" tag)? That said, the tooltip and "title" attribute are really only applicable to the abbreviation (and thus the "abbr" tag). Uzume 00:46, 14 April 2017 (UTC)
Albinoflea's suggestion (dotted underlining for templates) has been implemented. I'll need to take a closer look at "abbr". Ahasuerus 18:17, 16 April 2017 (UTC)

British Library added to the list of "other sites"

British Library has been added to the list of "other sites" that we link to using ISBNs. Ahasuerus 23:19, 12 April 2017 (UTC)

I noticed this seems to link via ISBN-10 and not ISBN-13 (which the URL supports). Thank you, Uzume 23:43, 12 April 2017 (UTC)
FYI, the BnF link is broken and needs to be fixed to http://catalogue.bnf.fr/rechercher.do?motRecherche=%s. Uzume 12:59, 14 April 2017 (UTC)
Thanks, I will take a look once I deploy the "unlimited primary verifications" patch. Ahasuerus 03:57, 15 April 2017 (UTC)
BLIC and BnF have been fixed. Open Library had to be adjusted as well due to certain software changes. If you find anything unusual, please let me know. Ahasuerus 16:55, 16 April 2017 (UTC)

Why is it allowed to vote for variants/translations?

I wonder why it is allowed to vote for variants/translations. Shouldn't a vote be given for the work and not for each variant separately? Example: Cixin Liu's 三体 has gotten one vote and the English translation The Three-Body Problem has gotten two votes. Shouldn't casting votes be either disallowed for variants and centralized in the main parent title, or if allowed be aggregated into one single result for all variants of a title? Jens Hitspacebar 21:09, 14 April 2017 (UTC)

I can see someone reading one variation/translation and voting for that one, especially if they haven't read another one. Perhaps the database could combine all votes for the main work and all variations to create a new user rating? "Combined User Rating", perhaps? It would be interesting to see how different variations rated compared to the original work and the combined user rating. ···日本穣 · 投稿 · Talk to Nihonjoe 22:44, 14 April 2017 (UTC)
That's a good idea. Keep the votes separated per title as they are now but calculate an additional "Combined User Rating" across main parent title and its variant titles. Jens Hitspacebar 08:12, 16 April 2017 (UTC)
It's certainly doable. This may also be an opportunity to take another look at the way we display votes. As of Saturday morning we had:
  • 1,317,500 titles
  • 9,555 titles with votes
  • 382 titles with 5 or more votes
Considering the fact that we don't display the "User Rating" value unless we have 5+ titles on file, our voting data is largely hidden. Perhaps it would be better to always display what we have. The original concern was that it would give undue prominence to unrepresentative votes, but it could be mitigated by displaying the number of people who have voted for the displayed title. Something like:
  • User Rating: 8.65 (17 votes)
  • User Rating: 5.00 (1 vote)
 ? Ahasuerus 17:57, 17 April 2017 (UTC)
I like this idea. ···日本穣 · 投稿 · Talk to Nihonjoe 22:44, 17 April 2017 (UTC)
I like it too. Jens Hitspacebar 12:22, 18 April 2017 (UTC)
OK, FR 1049, "Make votes more visible and informative", has been created. Ahasuerus 16:21, 27 April 2017 (UTC)

Translator of Cornelia Funke's "Dragon Rider"

In verifying my copy of this book, I noted an interesting issue about the translator. In my American first edition, the copyright page says: "Original English translation copyright © by Oliver Georg Latsch", but then says "This translation by Anthea Bell copyright © 2004 by The Chicken House". I wondered if there were an earlier publication translated by Latsch, but WorldCat says no, the only earlier edition they know, the British edition, WorldCat says was also translated by Bell. As it turns out, there's an uncorrected proof copy currently for sale on Abebooks, and the credit says "Translated by Anthea Bell and Oliver Latsch". I'm guessing they didn't like Latsch's translation, hence they hired Bell, and I further suspect that the Latsch translation was never published. Consequently, I've promoted Anthea Bell from being listed as the translator of my publication to being the translator for this title record, where I've documented this issue. I mention this both because it's somewhat interesting, and just in case anyone knows a publication in this title (of which there are a dozen) that was NOT translated by Anthea Bell. Chavey 23:09, 15 April 2017 (UTC)

Server downtime - 2017-04-15 @8pm

The server will be unavailable between 8pm and 8:05pm server time. A new patch will be installed. The patch will change the way primary verifications work and allow an unlimited number of primary verifiers. Help will be updated later tonight. Ahasuerus 23:51, 15 April 2017 (UTC)

Done. If you run into any issues, please post your findings here. Ahasuerus 00:04, 16 April 2017 (UTC)
Great. It should be noted there are many (more than 500 by my quick estimate) pub notes that need to be updated that refer to a primary verification by number (e.g., "PV1", "PV2", "PV3", "PV4", "PV5" and "PVT" or "PV(T)"). Thank you, Uzume 14:34, 16 April 2017 (UTC)

sort by author in search

It would be nice to have the option to sort by author in advanced title & publication search. Ideally sorted by last name, first name. Vasha 10:25, 16 April 2017 (UTC)

I have considered adding it, but how should the sort work for titles/publications with multiple authors? Ahasuerus 12:58, 16 April 2017 (UTC)
First ones by Author A, then ones by Author A + Author B, then A+C, then A+C+D, then A+D.....
I'm not sure how complicated the sorting can get. Is it possible to put the authors within a publication in alphabetical order so as to make sure not to have some works listed as by A+B and some as by B+A? Vasha 14:53, 16 April 2017 (UTC)
I know that the more complicated you make it, the longer the running time. I would actually be happy with something ultra-simple that only approximately groups same authors together. --Vasha 18:57, 16 April 2017 (UTC)
Understood. I'll see what I can do. Ahasuerus 20:49, 16 April 2017 (UTC)

"Last User Edit/Verification" field added

The list of primary verifications displayed on publication pages has been enhanced to display the "Last User Edit/Verification" date for each primary verifier. Hopefully the new field will make it easier to determine which verifiers are currently active.

Ideally, the new field would also include each verifier's "last Wiki activity" date, but I haven't been able to find a quick and reliable way to find it in the Wiki database. Ahasuerus 20:48, 16 April 2017 (UTC)

In theory that should not be that hard (and should be similar to how the new messages on the user talk page works as that just compares the last activity on the talk page to the last user's activity). Uzume 00:53, 17 April 2017 (UTC)
I am certainly open to suggestions! :-) At first I thought that "user_touched" in "mw_user" would work, but no such luck. Ahasuerus 01:26, 17 April 2017 (UTC)
Try mw_revision. Select max(rev_timestamp) from mw_revision where rev_user = xxx. --MartyD 01:44, 17 April 2017 (UTC)
Ah, yes, much better. <installs a fix> Thanks! Ahasuerus 02:28, 17 April 2017 (UTC)

Nightly cleanup reports -- update

Here are the new timings. Only reports that took more than 1 second to complete are included:

Report 1 took 9.488000 seconds to compile
Report 2 took 9.001000 seconds to compile
Report 3 took 6.322000 seconds to compile
Report 8 took 3.619000 seconds to compile
Report 11 took 1.014000 seconds to compile
Report 14 took 2.366000 seconds to compile
Report 15 took 1.446000 seconds to compile
Report 16 took 16.035000 seconds to compile
Report 20 took 3.408000 seconds to compile
Report 21 took 1.481000 seconds to compile
Report 29 took 1.063000 seconds to compile
Report 32 took 4.533000 seconds to compile
Report 33 took 21.543000 seconds to compile
Report 34 took 10.743000 seconds to compile
Report 36 took 1.148000 seconds to compile
Report 38 took 21.580000 seconds to compile
Report 40 took 1.983000 seconds to compile
Report 41 took 1.290000 seconds to compile
Report 42 took 3.037000 seconds to compile
Report 45 took 3.136000 seconds to compile
Report 47 took 26.933000 seconds to compile
Report 48 took 3.195000 seconds to compile
Report 49 took 1.614000 seconds to compile
Report 50 took 1.295000 seconds to compile
Report 52 took 34.622000 seconds to compile
Report 54 took 8.457000 seconds to compile
Report 58 took 1.857000 seconds to compile
Report 59 took 1.828000 seconds to compile
Report 60 took 1.836000 seconds to compile
Report 61 took 1.847000 seconds to compile
Report 63 took 3.329000 seconds to compile
Report 64 took 1.366000 seconds to compile
Report 68 took 4.362000 seconds to compile
Report 69 took 44.199000 seconds to compile
Report 70 took 14.796000 seconds to compile
Report 79 took 3.217000 seconds to compile
Report 80 took 19.965000 seconds to compile
Report 86 took 1.103000 seconds to compile
Report 87 took 5.158000 seconds to compile
Report 88 took 3.617000 seconds to compile
Report 92 took 3.200000 seconds to compile
Report 93 took 11.676000 seconds to compile
Report 94 took 1.106000 seconds to compile
Report 95 took 1.077000 seconds to compile
Report 97 took 4.512000 seconds to compile
Report 99 took 35.509000 seconds to compile
Report 107 took 4.769000 seconds to compile
Report 111 took 14.773000 seconds to compile
Report 127 took 5.989000 seconds to compile
Report 137 took 7.227000 seconds to compile
Report 143 took 1.791000 seconds to compile
Report 144 took 1.058000 seconds to compile
Report 151 took 37.451000 seconds to compile
Report 161 took 5.103000 seconds to compile
Report 167 took 28.999000 seconds to compile
Report 168 took 3.050000 seconds to compile
Report 188 took 1.449000 seconds to compile
Report 191 took 4.427000 seconds to compile
Report 192 took 1.047000 seconds to compile
Report 193 took 23.072000 seconds to compile
Report 196 took 3.369000 seconds to compile
Report 197 took 3.375000 seconds to compile
Report 200 took 3.870000 seconds to compile
Report 204 took 6.525000 seconds to compile

Ahasuerus 13:47, 18 April 2017 (UTC)

It seems like you highlighted all the ones over 20 seconds. If I add all the highlighted ones up we are now down to 269 seconds (under 4.5 minutes). This is down from 721 seconds (12 minutes)—more than a 2.5× speedup. Of course that is a very rough estimate as I did not count the excluded 20 seconds or less of each report not highlighted (which can add up with many reports) but it shows that even with just a little targeted work serious progress can be made (of course as we improve things there are diminishing returns). Uzume 19:45, 18 April 2017 (UTC)
One thing to keep in mind is that the cleanup reports are not the only thing that the nightly process does. It also rotates the banner and rebuilds the "stats" reports. The "stats" reports take a fair amount of time to rebuild; they too are on my hit list. Ahasuerus 19:58, 18 April 2017 (UTC)
Yes, I know. I was just looking at that code. Do we have times on the tasks from nightly_os_files.py (I am guessing no)? Uzume 21:38, 18 April 2017 (UTC)
Not at the moment, but I plan to add some checkpoints. Ahasuerus 21:58, 18 April 2017 (UTC)
In terms of the reports, however, the next biggest issue is Report 69. I looked at this and I imagine it is related to the 250+ "like" matching clauses (joined by "or") in the query caused by "badUnicodePatternMatch" (the dictionary returned by "unicode_translation" has that many entries). Uzume 21:38, 18 April 2017 (UTC)
That's right. When I limit the report to just one "bad" Unicode character, it takes less than a second. The more characters I add, the longer it takes. Ahasuerus 22:02, 18 April 2017 (UTC)
I am wondering if switching to single (long) "regexp" pattern would be better. Uzume 21:38, 18 April 2017 (UTC)
It's certainly something to look into. If you happen to have the proposed regexp handy, I can test it on the development server. Ahasuerus 22:00, 18 April 2017 (UTC)
It would have to be built from the dict like we currently are. I have not tested this code at all but I imagine something like:
def badUnicodePatternMatch(field_name):
        return "{field} regexp binary '{regexp}'".format(field=field_name, regexp="|".join(unicode_translation()))
Uzume 00:09, 19 April 2017 (UTC)
I am afraid "format" was added in Python 2.6 while we use Python 2.5. Ahasuerus 23:41, 18 April 2017 (UTC)
Easily fixed:
def badUnicodePatternMatch(field_name):
        return "%s regexp binary '%s'" % (field_name, "|".join(unicode_translation()))
Thanks. Alas, using regexp instead of LIKE makes performance much worse :-( Ahasuerus 01:13, 19 April 2017 (UTC)
Why don't we upgrade? I believe the latest 2.7.13 is compatible (3.x of course is not). Uzume 00:09, 19 April 2017 (UTC)
There are many components that I would like to upgrade, but we'll need Al for that. He is the only one who has full server access and can roll things back in case an upgrade messes things up. I am hesitant to change Python, MySQL and MediaWiki as long as we don't have that level of access.
As far as compatibility goes, at one point I tried running the ISFDB software under Python 2.6, but it immediately ran into problems. I am sure they are fixable, but it's a lower priority than MySQL and MediaWiki. Ahasuerus 01:21, 19 April 2017 (UTC)
I thought I got that working once. I will have to try that again once I get my development system back up. Uzume 01:52, 19 April 2017 (UTC)
I have an idea for this one. I just haven't gotten to it yet. I think it needs to be split up to do something easy/cheap to whittle down the set, then do the more complex matching against that smaller set. --MartyD 01:50, 19 April 2017 (UTC)
Ah, you mean something like looking for &#768; instead of looking for that combined with letters in 12 different ways (A&#768;, E&#768;, I&#768;, N&#768;, O&#768;, U&#768;, a&#768;, e&#768;, i&#768;, n&#768;, o&#768;, u&#768;). And use that same sort of logic on the whole set and then whittle that down after that for complete matches once the set is already smaller. That makes sense. I might try to process the keys somehow first to get a set of patterns that will reduce the set first before running the entire thing. Uzume 02:06, 19 April 2017 (UTC)
Well, it takes less than a second to retrieve the 12K title records with "#&" in the title (select title_id,title_title from titles where title_title like '%&#%';). We could then use Python to hunt for "bad" Unicode characters, which should take only a few milliseconds. Ahasuerus 02:35, 19 April 2017 (UTC)

(unindent) Perhaps but methinks we can find what you are looking for (plus a few other wrong cases) by just doing this:

def badUnicodePatternMatch(field_name):
        # All of the keys are either a single XML entity mapped character or a single simple character followed by single XML entity mapped character
        # We only care about the single XML entity mapped character so we remove the lead character when it is not an ampersand
        # And then we remove duplicates by pushing the list into a (frozen) set
        # This substantially reduces the number of patterns to search for
        unicode_patterns = frozenset(key if (key[0] == '&') else key[1:] for key in unicode_translation())
        return ' or '.join("%s like binary '%%%s%%'" % (field_name, pattern) for pattern in unicode_patterns)

And this does not require changing the query to search inside of a search and still reduces the number of patterns from 248 to 44 (a reduction of more than 5.5×). It will also find other aberrant combinations like a combining character after a character that has no business being combined with because it just finds all combining characters used in the map (regardless of what it might be combined with). Uzume 03:41, 19 April 2017 (UTC)

The updated query finished in 6.24 sec, which is a very good improvement. I'll have to take a closer look to see what the new logic does with combining characters. Ahasuerus 03:53, 19 April 2017 (UTC)
The new patterns will for example find any occurrence of "&#768;" and not just the combinations in the table (so it will catch "B&#768;", etc.). The set of search patterns are used for reports 65 though 70 so it should reduce some other reports as well. Perhaps we need some reports of bad and suspect Unicode sequences on transliterations too? Another thing to look at is if each of those fields is indexed. For the problem report number 69 this is true (so indexing won't yield anything better), however, according to the schema documentation these could perhaps be improved: 65 by indexing publisher_name in publishers and 66 by indexing pub_series_name in pub_series. However, according to the reported times these are apparently not currently an issue. Uzume 04:32, 19 April 2017 (UTC)
I don't know that an in-depth discussion of coding-related issues is appropriate for the Community Portal, --MartyD 10:58, 19 April 2017 (UTC)
We should probably move technical discussions to another page. Development has been mostly inactive for the last few years, but it (or one of its sub-pages) could be resurrected. Ahasuerus 11:53, 19 April 2017 (UTC)
but one comment on the above: Reducing the number of patterns would help but will not produce the biggest win. The basic performance with those reports and badUnicodePatternMatch is it leads to scanning all of the titles multiple times (once per pattern), and that remains a fixed cost. The original performs 248 x 1.317M = 326.62M comparisons. Reducing to 44 does help, but is still 57.95M comparisons. And the cost grows significantly (+248 or +44 per title) as titles are added. The trick is to minimize the comparisons against that full set when we know the candidate set is very small. Using the titles one as an example, I would try something like:
select title_id from titles where title_title like '%&#%' and (... the "or" comparisons ...)
If MySQL's optimizer does the right thing, it should do one scan for the first pattern (so 1.3M comparisons) and then one comparison for each member of the set and for each specific pattern (so 248 x 11.2K = 2.78M or 44 x 11.2K = 0.49M). As it is functionally equivalent to the existing code, I would try it and see what the results are, then tackle reduction of the patterns if necessary. --MartyD 10:58, 19 April 2017 (UTC)
Yes, I was planning to look at the "and (...)" approach but I noticed we were already using far too many extraneous patterns to begin with. Ahasuerus's comment about doing it partially in Python (looking for "&#" in the title in SQL first) underscored the need for that but databases are designed to handle data efficiently so I figured it would be better to focus upon the SQL and I was already embroiled in writing the Python to boil set of patterns down. Uzume 12:48, 19 April 2017 (UTC)
1.7 sec on the development server and 2.3 sec on the production server! Great job, folks! :-) Ahasuerus 11:04, 19 April 2017 (UTC)
One thing I don't understand about the patterns is whether, say, all occurrences of "&#768;" are problematic or just the enumerated ones (B&#768;, etc). If it's all of them, then the trick of ignoring the letter-modified portion of the character to find the sequence is definitely a good approach. --MartyD 10:58, 19 April 2017 (UTC)
Combining Diacritical Marks are used by Unicode to allow entering accented characters as a combination of two characters. For example, consider this ISFDB title. The characters "è̂" and "à̂" are actually combinations of two separate characters: "è"/"^" and "à"/"^".
That is fine for display purposes, but what happens when Unicode has another character which is identical to the two combined characters? Inevitably, some users will enter it one way and some users will enter it the other way. The two versions will be displayed the same way, but searching will be broken because a search will find just half the matching entries.
That is why we have a translation table (unicode_translation) which normalizes "combining diacritics" at data entry time. The cleanup reports are just another way to make sure than nothing got past the translation table.
Uzume's approach is interesting because it also finds occurrences of combining diacritics which we currently do not trap. Something to consider. Ahasuerus 11:49, 19 April 2017 (UTC)
We can very easily do both with:
def badUnicodePatternMatch(field_name):
        # Reduce the set of bad Unicode to search for by finding all "combining diacritic" combinations, not just the ones we already know how to replace
        # All of the keys are either a single XML entity mapped character or a single simple character followed by single XML entity mapped character
        # We only care about the single XML entity mapped character so we remove the lead character when it is not an ampersand
        # And then we remove duplicates by pushing the list into a (frozen) set to substantially reduce the number of patterns to search for
        bad_unicode_set = frozenset(key if (key[0] == '&') else key[1:] for key in unicode_translation())
        bad_unicode_sql_patterns = " or ".join("%s like binary '%%%s%%'" % (field_name, bad_unicode) for bad_unicode in bad_unicode_set)
        # All the patterns contain XML entity mapped characters so optimize the SQL to find those (and throw away everything else) first
        return "%s like binary '%%&#%%;%%' and ( %s )" % (field_name, bad_unicode_sql_patterns)
Uzume 13:21, 19 April 2017 (UTC)

Banner rotation

I doubt the nightly banner rotate is much of a performance bottleneck for the nightly run, however, I thought I would point out it does impact performance (perhaps slightly) via a change in client bandwidth. It would probably make more sense to have every page with the banner add a nightly_banner.css and just update the CSS nightly instead of updating CurrentBanner and IsfdbBanner.jpg. The client browsers can then cache the images and only has to get tiny CSS file update. Of course we could also use CSS sprites and JavaScript (and then only those users would see the banner rotate) and could rotate it every hour or something then as there is no cost and users would see the rotate even without refreshing any pages. It it not a priority though. Uzume 22:12, 18 April 2017 (UTC)

I like this idea. ···日本穣 · 投稿 · Talk to Nihonjoe 18:39, 27 April 2017 (UTC)
Upon reflection I don't think the current system, which requires the nightly job to rotate banners, is optimal. We could modify the software to display the right banner based on the day of the month. Ahasuerus 19:12, 27 April 2017 (UTC)
How many banners are there? Is there a gallery we could look at? ···日本穣 · 投稿 · Talk to Nihonjoe 20:26, 27 April 2017 (UTC)
There are 10 banners. There is no way to view them as a gallery although it would be easy to implement. You can review them by accessing www.isfdb.org/IsfdbBannerX.jpg , where X is one of the following numbers: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Ahasuerus 20:42, 27 April 2017 (UTC)

Advanced Search changes - 2017-04-18

The software behind the Advanced Search has been revamped. For the most part the functionality remains the same. However, the new code will make it easier to add new ways of sorting the data like the recently requested ability to sort titles and pubs by author.

For now, Advanced Title Search and Advanced Publication Search have been enhanced to sort titles and publications by title and then by date transparently. For example, if you do a search on "title is exactly Heaven" using the default title sort, the results will also be sorted by title date.

Due to the scale of the changes, the likelihood of new bugs is high. If you see anything unusual, please report your findings here. Ahasuerus 21:41, 18 April 2017 (UTC)

ISFDB logo now clickable

Clicking the ISFDB logo in the navigation bar on the left now takes you to the ISFDB home page. Ahasuerus 00:04, 19 April 2017 (UTC)

Similarly, clicking the composite image displayed at the top of ISFDB pages now takes you to the home page. Ahasuerus 00:46, 19 April 2017 (UTC)
Why is the logo different on the home page (where it looks like a card catalog) than elsewhere (where it looks like a catalog card)? Uzume 00:28, 19 April 2017 (UTC)
I don't really know. I think that's how Al did it back in the day. Since he has an arts background and I am colorblind, I rarely question his design choices. Ahasuerus 00:46, 19 April 2017 (UTC)

And I wonder why we are using a GIF for that (I am sure it is historic). Uzume 00:28, 19 April 2017 (UTC)

Most likely! Ahasuerus 00:46, 19 April 2017 (UTC)
There's this one, too. I think it would be useful to have a link on the wiki side that took you to the actual main page of the site (http://www.isfdb.org/) instead of the wiki main page (http://www.isfdb.org/wiki/index.php/Main_Page). Can we add such a link to the Navigation menu? I'm not sure what to call it, unless we have a "Main Page" link and a "Wiki Main Page" link. ···日本穣 · 投稿 · Talk to Nihonjoe 16:43, 19 April 2017 (UTC)
I have added a link to the main ISFDB page, renamed the link which takes you to the main Wiki page and dropped "Random Page". How does it look? Ahasuerus 18:30, 19 April 2017 (UTC)
Awesome. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 04:09, 20 April 2017 (UTC)
Oh no!—the death knell of Random page. I know I won't miss it (you can still pull it out of Special pages if you do). In terms of looks do not forget the various skins, e.g.:
If you like one you can of course permanently select one from your Preferences "Skin" tab (and do not forget each also has a print version too). Uzume 18:52, 20 April 2017 (UTC)
This has been one of my pet peeves forever... Thanks for adding these links! Albinoflea 22:30, 19 April 2017 (UTC)
One pet peeve at a time :-) Ahasuerus 23:03, 19 April 2017 (UTC)

Advanced Author Search - e-mail

Advanced Author Search has been changed to allow searching for e-mail addresses. For example, you can search on e-mail address contains sff.net if you want to find all authors whose e-mail addresses are invalid now that SFF.net has shut down. Ahasuerus 18:58, 19 April 2017 (UTC)

Perhaps it would be good to simply run a job to remove all of those? They are invalid now, and most of them probably don't have a public replacement. Email is not a terribly important entry, anyway (at least IMHO). ···日本穣 · 投稿 · Talk to Nihonjoe 16:38, 20 April 2017 (UTC)
Some of them have public replacements, e.g. as per Lawrence Watt-Evans's Web page, his public e-mail address is misenchantedpress@gmail.com. Ahasuerus 16:46, 20 April 2017 (UTC)
I have replaced several of them already. I do not think a mass deletions is warranted. Uzume 18:04, 20 April 2017 (UTC)

Tangent

Is Tangent left out for some reason or does it qualify by the rules of acquisition? It should be noted there seems to be two such serials: the first was a short lived two issue fanzine in 1965 and the other a regular semipro printed SF/F review magazine (but has published some fiction) from 1975 until 1977. The latter restarted as a print magazine in July 1993 and went to web in 1997. More information is available:

  • the earlier fanzine:
    • GC Tangent 1965 (the earlier fanzine published by British Science Fiction Association and edited by Roger G. Peyton)
  • the later semipro review magazine:

It seems like at least the latter should be "in" to me (but the earlier also published some interesting fiction). Thank you for your feedback, Uzume 19:29, 20 April 2017 (UTC)

"Watching" an author page

One of the annoyances with moving all of our author bios over to the database is the loss, AFAIK, of the ability to "watch" a bio page. I could do that so long as the bios were on the Wiki side, so I could see if someone added some new information about an author, or deleted information I had put there. My understanding is that there is no way to do that with the author pages any more. Too bad. Chavey 03:38, 21 April 2017 (UTC)

That could be a cool feature. Basically, have it allow each user to create a list of authors they want to keep an eye on, including those they haven't verified. If it could also notify once a day of any new items for that author, or any changes to the author page, that would be even more awesome. That would keep it from being too resource intensive. ···日本穣 · 投稿 · Talk to Nihonjoe 19:46, 21 April 2017 (UTC)

Ahasuerus downtime -- 2017-04-21

Ahasuerus says that he may be unavailable until Sunday afternoon "for technical reasons". Humans, what can you expect?... Fixer 20:23, 21 April 2017 (UTC)

In Lands That Never Were

Hi -- I just heard from Gordon Van Gelder that [3] this is misattributed; he says it was actually published by Thunder's Mouth Press, not Four Walls and Eight Windows. I see Amazon has Thunder's Mouth Press. I don't want to submit a correction as it's been primary verified. If anyone has a copy or knows how to find a way to verify it online, would they correct it? I'm hoping to run the F&SF article on the main page of Wikipedia, and would like the bibliographic data to be as accurate as possible. Thanks for any help. Mike Christie (talk) 22:11, 21 April 2017 (UTC)

Did you ask Don Erikson? ···日本穣 · 投稿 · Talk to Nihonjoe 00:03, 22 April 2017 (UTC)
I have now; thanks for the nudge -- I should have done that first. Mike Christie (talk) 00:31, 26 April 2017 (UTC)

Title page enhancements

The "Publications" section of the Title page has been enhanced. If a title has translations or other variants associated with it, the "Publications" section will now display the following line:

  • Displaying all variants and translations • Do not display translations • Do not display variants or translations

Clicking "Do not display translations" or "Do not display variants or translations" will re-display the title page with the requested subset of publications as well as the option to "Display all variants and translations".

The new logic is still experimental. Let's give it a day or two and see if additional tweaks may be desirable. Ahasuerus 23:57, 23 April 2017 (UTC)

Nice enhancement! Chavey 05:35, 25 April 2017 (UTC)

When did transliteration become romanization?

Although transliterations support was added to many record types (I am still looking forward to them for title/work series), apparently all the help specifies this is only for use with romanizations (i.e., Latin transliterations): Template:TitleFields:TransliteratedTitle, Template:AuthorFields:TransLegalName, Template:PublisherFields:TransliteratedPublisherName. When did this become romanization only? Are there plans/interest in supporting other transliterations? Should I take this to rules and standards discussions or make a feature request to support such things? On a related note, if we really only support romanizations why are these fields labelled as transliterations; wouldn't it be better to correctly label them as romanization fields? Thank you, Uzume 15:51, 24 April 2017 (UTC)

In practice, they had been used also for Kana transliterations of Kanji titles/authors (for Japanese)... The help may be lagging behind on this though. Annie 16:15, 24 April 2017 (UTC)
It should be stated that the Japanese use of kana for collating, indexing and searching (used to equate different kanji forms) is not within the realm of transliteration but withing the realm of transcription (Transcription (linguistics) which is a very different thing). For example "山", a character meaning "mountain" (see wiktionary) can be pronounced different ways and cannot be transliterated, however, it can be transcribed in a context were the pronunciation is determined (e.g., to "yama", "san", etc.). Uzume 19:58, 24 April 2017 (UTC)
The use of the recently added "transliterated" fields has been largely limited to romanizations from the beginning. Certain other, less common, cases are also supported as listed on the Template:AuthorFields:TransLegalName page:
  • The author's working language uses multiple scripts. For example, Japanese names are usually written using kanji, but in certain cases hiragana or katakana are used. In Serbian, both Latin and Cyrillic are used, although the Cyrillic alphabet is considered primary. In Azerbaijani, the Perso-Arabic script, Cyrillic and Latin have all been used at different points.
  • The author has lived in different countries which use different alphabets/scripts. For example, Alexander Lomm lived in Russia as well as Czechoslovakia and wrote SF in Russian and Czech. Since we list Russian as his working language, the Russian form of his legal name, "Кличка, Вацлав", is currently entered in the Legal Name field. The Czech version of the name, "Klička, Václav", and the fully Romanized version, "Klichka, Vatslav", appear in the Transliterated Legal Name field.
Ahasuerus 16:20, 24 April 2017 (UTC)
Well I ask this because I recently had a moderator remove Japanese kana and Cyrillic transliterations (I had added) of an English working language author record (both canonical and legal names) and I was quoted the help that these are romanizations. This author does have translated publications in languages supporting those scripts. I think that is wrong and want to know how I can change that so we do support such transliterations (either rule-wise or technically). Should we be removing valid transliterations in certain scripts and why? This is not really clear based on the help vs. the above discussion so I wanted to clarify this and ask for getting support of having other transliterations that apparently are not mentioned in the current help. "In practice" and "use of the recently added 'transliterated' fields has been largely limited to" do not really spell "policy". Uzume 16:37, 24 April 2017 (UTC)
Cyrillic transliterations are tricky because of the different alphabets that had been created through the years. Which one would you use - the Russian, the Bulgarian, the Serbian, another one? A non-used by anyone subset that contains only letters that all of them share? The name as used in Russia/Bulgaria/Serbia/elsewhere (as opposed to an actual transliteration)? So if we open the door for these, we run the risk of making this a lot bigger project that it needs to be - if a Russian edition even shows up, there will be a variant. And no - I am not mistaking translation and transliteration - it is just that a "Cyrillic" alphabet/script actually do not exist on its own - it is a collection of scripts based on a common ancestor (pretty much like the Latin ones - but I do not see anyone arguing that an English name needs to be transliterated into the Polish variant for example as well)- and the transliteration by definition is conversion between scripts :) Annie 16:58, 24 April 2017 (UTC)
This is not much different than the case for Chinese characters used by various Chinese languages/dialects, Japanese, and (traditional) Korean and is one of the reasons Unicode differentiates between scripts and writing systems despite things like Han unification (there are some Japanese created kanji called kokuji). Also similarly there are numerous romanization systems for Japanese (not to mention other historic kana systems). Some of the more modern and common ones include: Hepburn romanization, Nihon-shiki romanization, and Kunrei-shiki romanization. I believe the case is similar for Korean (last I checked). So the "door" is already open with regards to romanizations because even from a single language there are many. Leaving the door open for romanizations but not other transliterations seems unbalanced and ill thought out. Uzume 17:51, 24 April 2017 (UTC)
The issue was considered back when transliteration support was first implemented. The decision to use romanization was a conscious one. Ahasuerus 18:28, 24 April 2017 (UTC)
If so I propose we correctly label the fields as romanizations and not transliterations since apparently that is consciously decided upon and correctly labeling them would be clearer. Uzume 19:06, 24 April 2017 (UTC)
As indicated in the previously mentioned Template:AuthorFields:TransLegalName, the field is used for more than romanizations. Ahasuerus 19:38, 24 April 2017 (UTC)
Template:AuthorFields:TransLegalName mentions Chinese romanizations and kana for Japanese but I would submit one cannot transliterate such scripts, however, they can be transcribed within a context and a proscribed pronunciation. It is easy to get transliteration and transcription confused. One can have romanization within transcription too, e.g. Yale romanization of Mandarin vs. Yale romanization of Cantonese. Clearly these use mostly the same Chinese script but different pronunciations. Uzume 19:58, 24 April 2017 (UTC)
I don't know enough about this issue to tell whether it affects what we are doing here. I'll ask Linguist to stop by. Ahasuerus 21:17, 24 April 2017 (UTC)
(resolving an edit conflict) The different romanizations still use Latin that people recognize. The Macedonian/Serbian letter Џ will be less readable for a Bulgarian/Russian user than the Latin variant. Same with the Bulgarian/Russian Я or Щ or the Russian/Belorussian Э - a Macedonian/Serbian will be better served by the Latin than having a "Cyrillic" transliteration with letters they do not know. There isn't a common way to move to Cyrillic. The Chinese example above is not relevant here - it is the reverse of what we are talking about. The difference between your examples and the Cyrillic one is that you are talking about multiple scripts in one language; the Cyrillic family is the opposite case. Unless if you are proposing that we need to add a transliteration in ALL Cyrillic languages/alphabets, it is just not practical. Or do you propose we transliterate into each Chinese dialect? Annie 18:43, 24 April 2017 (UTC)
I was never suggesting a move to Cyrillic just supporting Cyrillic transliterations where appropriate (and "where appropriate" is a different language exercise as you point out). I already suggested we should have some rules about which transliterations we support but I would like it to include more than just a blanket "romanization" (for romanization, e.g, see ALA-LC romanization). Uzume 19:06, 24 April 2017 (UTC)
As indicated in Template:AuthorFields:TransLegalName, we don't have a preferred romanization system. The field can be used for any and all romanizations:
  • [enter additional values in the new fields ... if...] There are multiple competing Romanization schemes for the author's working language. For example, there are multiple Romanization conventions for the Chinese language, including Gwoyeu Romatzyh, Wade-Giles, and Pinyin.
This was done by design. We had a number of discussions of romanization systems in 2012-2014 and decided against picking just one. Hence the ability to add multiple romanizations. Ahasuerus 19:41, 24 April 2017 (UTC)
Agreed, however, can I create my own home grown romanization scheme tomorrow and submit data for it? I was not proposing standardizing on just one but standardizing nonetheless. Uzume 20:00, 24 April 2017 (UTC)
I don't think this has been a problem so far, but it may be useful to expand Help to say something like "If you are not sure how to transliterate a letter or a word, you can consult the list of common transliteration schemes [somewhere]". Ahasuerus 21:05, 24 April 2017 (UTC)

(after edit conflict)

Let's consider the case of Lois McMaster Bujold, a widely translated author. Her works have appeared as by:
  • ロイス・マクマスター・ビジョルド
  • 洛伊斯.比约德
  • 路易丝·麦克马斯特·比约德
  • Loiz Mekmaster Bižol
  • לויס מקמסטר בוז`ולד
  • 洛伊絲.莫瑪絲特.布喬
  • Лоис Макмастер Буджолд
  • Lois M Bujoldová
  • Лоис Макмастър Бюджолд
  • Λόις Μακμάστερ Μπυζόλντ
and that's just the names that we have on file. We list these name as pseudonyms, but we don't enter them in the Transliterated Name field of the main author record. I don't see the value in adding them there: we would be looking at 156,000 author names times a few dozen alphabets, syllabaries, etc times multiple transliteration systems per alphabet. That's millions upon millions of values.
The reason why the "transliterated" fields were added in the first place was to help our users read/pronounce names and titles written in languages/alphabets/etc that they are not familiar with. Latin is the universally recognized alphabet and is sufficient for our purposes.
We have also overloaded this field for certain corner cases -- see above -- since we didn't have another place to put this information.
As far as the issue of clarity goes, I think that the current policy is explained clearly in the Help section that I quoted above. If there are any ambiguities, we can clarify them. Ahasuerus 17:03, 24 April 2017 (UTC)
I would suggestion "our purposes" could be better for helping our users out if we supported users of other languages. If we had Japanese kana transliterations that would help (and perhaps get more of them here) them read and pronounce Latin, Cyrillic and other text we use here. And the case could be made for Cyrillic language users, etc. Uzume 17:55, 24 April 2017 (UTC)
Japanese, Chinese, Russian etc children learn Latin characters when they study math, physics, chemistry and other subjects. It's a universally recognized alphabet at this point. Ahasuerus 18:26, 24 April 2017 (UTC)
Would the addition of other transliterations help some Japanese/Chinese/etc users use the site? Probably, although we would first need to add user preferences for transliterations and add "alphabet" labels to transliterations. Would the benefit justify the amount of time that it would take to change the software and to enter the requisite data? I don't think it would, at least not given our current resources and other priorities. Ahasuerus 18:32, 24 April 2017 (UTC)
Many from other countries do learn Latin scripts but certainly not all. I would say our user base is still highly European and North American concentric (where Latin-scripted languages are extremely predominate), however, I am not sure that we should consider that a feature and future goal of our project. I am not opposed to "eventually not now" answers. I realize we have priorities and limited resources (we have many FRs that are not being addressed yet too). The point is, I believe we should consider going that way (even if it is currently only an idealized future). Uzume 18:54, 24 April 2017 (UTC)
Well, there are thousands of things that would be "nice to have". Some of them have been proposed at various points in time, e. g. making the user interface and Help available in multiple languages. They may or may not be difficult to implement depending on the preferred design. I don't think repeated discussions of these issues (this one was debated extensively in 2012-2016) are worth our bandwidth. There may come a time when our resources and our list of priorities change, at which point we will revisit them. Ahasuerus 19:52, 24 April 2017 (UTC)

Let's not forget that this field is not only used for romanizations of non-latin alphabets, but also for providing alternate diacriticless versions of latin-alphabet titles that contain non-Latin-1 characters. That is useful, not for readability purposes (titles with the diacritics stripped off are the opposite of readable), but rather so that they can be found if people type them into the search box without diacritics. So that's already a deviation from the stated purpose of having the romanizations.

I think, maybe, it would be good to have a separate field for alternate titles -- this would not be displayed in mouseover, or visible to users, but it would be used for variants that might be searched for. This would be useful not only for allowing people to pull up Vonarburg's L'œil de la nuit whether they search for "oeil" or "œil", but could also be used for titles that have ampersands in them (who can remember whether to search for "and" or "&"?) and so much more. --Vasha 03:32, 25 April 2017 (UTC)

Filing collections and anthologies as non-genre

I do think that a title should be marked non-genre if the majorities of the titles collected in it are non-genre; is that assumption right? I'm asking because it seems that some titles we file as genre will likely turn out as non-genre if we apply this rule. In particular, I'm thinking of Vonnegut's Welcome to the Monkey House, which I reread right now, having made about the half of it (the first reading was long ago). I have the impression that it is regarded as a genre collection by many, but most items seem not to be in that category. Stonecreek 19:14, 24 April 2017 (UTC)

I am on board with labeling container titles as non-genre when the majority of the published contained titles are non-genre. Uzume 19:26, 24 April 2017 (UTC)

Maurice Level

A lot of Maurice Level's stories need the English translations connected to their French titles. I've done a bit of checking online but the information available is sketchy. What I did discover, though, is that it sounds as if most of the stories are non-speculative. Is there anyone here interested in working on that, tidying up the page and figuring out which are the speculative stories? --Vasha 19:04, 26 April 2017 (UTC)

Christopher Nuttall vs. Christopher G. Nuttall

If memory serves, a few weeks ago someone volunteered to work on reversing suboptimal pseudonyms and asked for more targets. I knew we had some, but I couldn't think of anything at the time. Well, here is one: Christopher Nuttall vs. Christopher G. Nuttall. Ahasuerus 19:03, 28 April 2017 (UTC)

So basically, Christopher G. Nuttall should be the main name? ···日本穣 · 投稿 · Talk to Nihonjoe 19:31, 28 April 2017 (UTC)
That's right. At this point most of his books have been published as by "Christopher G. Nuttall". Ahasuerus 19:42, 28 April 2017 (UTC)
I submitted a removal of the pseudonym relation. What do you think the quickest order of the next steps would be? I don't want to make more work than necessary. ···日本穣 · 投稿 · Talk to Nihonjoe 19:39, 28 April 2017 (UTC)
Unfortunately, there is no magic bullet at this time. Every existing VT needs to be broken and new ones need to be set up. Ahasuerus 19:43, 28 April 2017 (UTC)
There are three stages, and you have to wait for approval between each one:
1A. Break old pseudonym relationship; create the new one; transfer author info from the old canonical name to the new one.
1B. Go into the pseudonym page (Christopher G. Nuttall), click "view all titles by this pseudonym", open all those records, and break the variant relationship.
2. After approval of those changes, open all the records that remain on the Christopher Nuttall page. If a record has a publication, make it a variant of a new CGN title. If it has no publications, change the name on it to CGN.
3. Once no records remain on the CN page, go to the CGN page and use "Check for Duplicate Titles" to merge all duplicates.
This is not the method that necessitates the fewest approvals, but it is the easiest, most streamlined and automatic procedure. Additional steps would have to be added if there were translations, for example, but Nuttall is a simple case. --Vasha 19:54, 28 April 2017 (UTC)
Yes, that's pretty much it. I have approved the pseudonym removal submission, so we can proceed with the next step. Ahasuerus 23:44, 28 April 2017 (UTC)
Okay, I've removed all the variants and moved the info from one name to the other. Have to wait for those to be approved before I can revariant. ···日本穣 · 投稿 · Talk to Nihonjoe 02:28, 29 April 2017 (UTC)
I've now varianted all the titles which were credited only to Christopher Nuttall. Once those are done, all the ones left on Christopher Nuttall should be deleted since they will be titles without any pubs under them (since the titles have never been published under just "Christopher Nuttall" (without the "G."). There's no need to change the author and then merge them as that would be an additional step. I can do that once the currently-waiting items are approved. ···日本穣 · 投稿 · Talk to Nihonjoe 17:55, 29 April 2017 (UTC)
Actually, those titles that were formerly at the top level on the CN page may contain information that the CGN titles you unvarianted from them don't-- series, length, links to reviews, notes, etc. So don't delete them. You could check for differences and transfer the info over, but the change-name-and-merge procedure is simpler. --Vasha 18:51, 29 April 2017 (UTC)
The series information has already been transferred. I've submitted all the author changes. ···日本穣 · 投稿 · Talk to Nihonjoe 19:15, 29 April 2017 (UTC)
Okay, I think all of the final submissions are done. Just waiting for them to be approved so I can verify. ···日本穣 · 投稿 · Talk to Nihonjoe 19:34, 29 April 2017 (UTC)

Alex Irvine

Also, we may want to reverse Alex Irvine and Alexander C. Irvine. Ahasuerus 23:44, 28 April 2017 (UTC)

I've done step one for Alex Irvine. --Vasha 08:02, 29 April 2017 (UTC)
And it's all finished. --Vasha 19:53, 29 April 2017 (UTC)

Title Merge tweaks

The Title Merge page has been tweaked. If you are merging 2+ titles and at least one of them has a "Yes/No" flag (juvenile, non-genre, etc) set to "Yes", the page that lets you select the desired value will default to "Yes" instead of the first displayed value. Ahasuerus 19:32, 29 April 2017 (UTC)

The Geek's Guide to the Galaxy

Lightspeed often publishes interviews conducted as part of the podcast The Geek's Guide to the Galaxy. We have some of these in the database credited to The Geek's Guide to the Galaxy and some to David Barr Kirtley or to Kirtley and John Joseph Adams. Should these be linked as pseudonyms, and if so, how? --Vasha 00:02, 4 May 2017 (UTC)

Too bad it's not specfic...

...because it would be totally awesome to have this on the site. ···日本穣 · 投稿 · Talk to Nihonjoe 23:03, 5 May 2017 (UTC)

The cover makes it look like it could be a time slip romance, but apparently not... Ahasuerus 00:29, 6 May 2017 (UTC)
Yup. It's finger-lickin' good! ···日本穣 · 投稿 · Talk to Nihonjoe 19:08, 8 May 2017 (UTC)

New cleanup reports: invalid/unsupported HTML in notes and synopsis

We have 10 new cleanup reports which look for invalid and/or unsupported HTML in record notes and title synopses. The data will become available on Tuesday morning (server time.) I expect that the reports will find 1,500+ records with problematic HTML.

The vast majority of the records that the new reports will find contain malformed HTML: misspelled tags, improperly closed angle brackets, non-existent HTML tags, etc. In addition, they look for other types of HTML issues. The fact that we have been allowing arbitrary HTML in notes is a serious security issue which could potentially cause massive problems. For this reason the new reports also identify records with HTML tags and attributes which will be disallowed in the near future, including embedded style sheets and divs.

Here are the HTML tags that the reports currently allow. Tags without attributes:

  • abbr
  • b
  • blockquote
  • caption
  • center
  • cite
  • del
  • em
  • h1
  • h2
  • h3
  • i
  • li
  • ol
  • p
  • pre
  • q
  • span
  • strong
  • sub
  • sup
  • table
  • tbody
  • td
  • th
  • tr
  • u
  • ul

Self-closing tags:

  • br
  • hr
  • p

Tags with attributes:

  • a
  • table
  • td
  • th
  • tr

These lists will be tweaked and further restricted as we get ready to implement automatic HTML sanitizing at submission creation time. Ahasuerus 17:57, 8 May 2017 (UTC)

I'm making progress on them but sometimes the correction of errors is above my knowledge of HTML and its subtleties (after all I'm a COBOL guy). My feeling, apart that there are some frequent offenders with the same mistakes (probably a cut & paste effect) is that too much sophistication in the layout of the notes is never a good idea. Hauck 08:42, 9 May 2017 (UTC)
I went through 281 of these last night -- all but the publication records. There are 31 records remaining in the "naughty" lists, most of which are due to tags that I think you should accept (the comment tag and the font tag) or else tags that are acceptable but which you're rejecting because it's using attributes of that tag (lists with "type=" and "start=" to be precise). There's another batch of title recs that have embedded images, which seems like they should be admitted, possibly with limitations. And then there's one "meta" tagged record ("The Kiss"), whose purpose you may recognize when you see it. I don't know if that tag should be on the "good" list, or if there should be an "exceptions" list, or if there's some other solution. Chavey 13:00, 9 May 2017 (UTC)
I agree with Hauck on the issue of "too much sophistication" in the html being a problem, not a strength. Some of the records we had would have been impossible for a non-html expert to be able to edit or update, or even to read in the code format. Several of those in the title records I had to simplify dramatically, e.g. some (div with inset margins) tags being replaced with blockquotes. Chavey 13:04, 9 May 2017 (UTC)
Thanks for working on these! I have further tweaked the Title list, which is now down to 18 records. The first 6 are OL's with various "start"/"type" values, which can be easily replaced with HTML blockquotes. The 11 that follow are all img's, which can't be allowed because they leave us wide open to XSS attacks -- see XSS Filter Evasion Cheat Sheet for a very long list of possible attacks. We'll have to replace them with regular HTML links. The last one is the "meta" tag that you mentioned.
As a general rule, tags without attributes ("b", "i", etc) are easy to sanitize. As long as we have a white list of allowed tags, we can sanitize the input in three simple steps:
  • Convert all tags on the white list from <tag> to [[tag]]
  • Convert all special HTML characters like < and > to their HTML versions
  • Convert the white listed tags back to the HTML format
On the other hand, attributes are much more difficult to deal with and require complicated handling, which is time-consuming and error-prone. We will have to support valid href's for "a" tags, but otherwise attributes will not be supported. It will affect a couple dozen notes which use fancy tables, but it's a small price to pay.
HTML comments are another huge security hole and need to be disallowed.
As far as the "meta" tag goes, the easiest way to implement this functionality is to add a new template. We can call it something like "norobots" and it will be converted to HTML at display time. Ahasuerus 15:50, 9 May 2017 (UTC)
This looks like the right approach to me. I had changed a bunch of > and < characters to their HTML versions in the cleanup I did. I had done a couple of blockquote and "by hand" start/type attributes, but I left some of the easier ones in just in case those were options that could be more generally allowed. (I replaced all the related attributes for the "li" tag.) Doesn't surprise me that they can be abused, or that images could be, but I'm a little surprised to hear that comment tags can be as well. (I was thinking about using them to put in sources in some of the author notes. It would be a nice way to make them available without cluttering up the 'standard' presentation.) I like your idea about the "norobots" tag. I don't know whether the approach I used will actually work. When I've used norobots before, it's always been in the header, where meta tags normally live. Google hasn't re-indexed isfdb since I added that tag, so I don't know yet if it will work. But if it does, a template would be a natural way to implement it. Chavey 23:03, 10 May 2017 (UTC)
How about putting them after the {{BREAK}} template? All notes support the break syntax that sends whatever is after it to a new page. That way they are easily visible if you click on the link but do not clutter the main page. Annie 23:07, 10 May 2017 (UTC)
Excellent solution! I still don't think about the BREAK command enough.
I just completed all of the "titles with unsupported HTML", except for the image tag. I'll leave that for Ahasuerus to think about :-) . The most complicated of the lists was Asimov's The Road to Infinity, which I converted into a table. I think it managed to capture the original fairly well, without all of the extra blank lines before and after each sub-heading. I'll try to begin work on "Publications" tomorrow. Chavey 00:35, 11 May 2017 (UTC)
That's right, separating core information from our sources was one of the intended uses of the BREAK template.
As far as "fancy tables" go, it may be possible to preserve some of the existing functionality by creating one or more templates which will be converted to HTML at display time. Granted, as of Saturday morning, we had relatively few table-related tags with attributes: 16 "tables", 2 "trs", and 24 "tds". Still, if the cleanup effort finds that it's something that we can benefit from in the future, we can look for commonalities which could be used to create templates. Ahasuerus 01:17, 11 May 2017 (UTC)

(unindent) Re: the proposed {{NOROBOTS}} template. I have poked around and there are some issues associated with putting "meta" tags into the body of an HTML page. The core problem is that they are invalid under our (4.01) version of HTML, although they are apparently valid under HTML5. The resulting HTML may be displayed differently by different browsers, although user testing suggests that major browsers shouldn't be confused by it.

I am more concerned by the fact that Google looks for HEAD-only HTML tags in the BODY section and assumes that their presence may be indicative of the site having been spammed/hijacked -- see this post in which Matt Cutts, the head of Google's Webspam team, talks about some unexpected "meta" tags in the BODY section. Ahasuerus 18:00, 11 May 2017 (UTC)

It looks to me like they're only looking flagging certain specific meta flags as indicative of a hacked website, so I suspect we'll be okay. Since the "big" search engines will all be interpreting HTML5, I'm guessing there won't be any real problems with that, but I'm not sure how to be confident about that.
I finished off most of the publication fixes, leaving only the lists with attribute tags (which I can finish off tomorrow). There's one more 'font=red' tag, and I don't remember how we dealt with that in the title recs. Altogether, there are now 42 complaints. There also seem to be two spurious complaints (mismatched angle brackets and double quotes) that have cropped up on both reports without linking to any records. It might be that someone has beat me to correcting them on both days or, as I know has happened before, something odd is getting reported that isn't really there. Chavey 09:49, 12 May 2017 (UTC)
All done! The links in this cleanup report now show no complaints. The two editors whose work was most affected by these constraints have been informed about the changes made. (And one has already responded 'That's fine".) One other editor very regularly used a non-standard style of tag that's not in the html documentation, but was accepted by most browsers (putting a "/" *after* tags like ul, i, a, etc. to start the tag). There were literally *hundreds* of those to be changed. But I only looked at the dating of a couple of them, all of which were from a few years ago, so I didn't bother to tell that editor about the problem. We can wait to see if more such records crop up later before we mention it to him. Chavey 20:07, 13 May 2017 (UTC)
Many thanks! In the meantime, I have tweaked the logic some more and will deploy the new version in a day or two. It should find a few dozen bad "href"s. Ahasuerus 20:36, 13 May 2017 (UTC)

"Universe" and routine sequels by other hands

Is there a criterion for the creation of a superseries, commonly named "Universe", such as Neverland Universe and Oz Universe?

Those two are sprawling. Within the Neverland Universe subseries Peter Pan, and the Oz Universe subseries Oz, we do include some Sequels by Other Hands (Encyclopedia of Fantasy entry by John Clute), both authorized and unauthorized.

Those examples suggest to me that series Five Children Universe is a mistake, for what seem to be rather straightforward latterday sequels to three E. Nesbit novels.

  1. Cresswell 1994: "A sequel to E. Nesbit's children's classic, "Five Children and It". After the original five children went home, the Psammead ("It") was feeling lonely, but then a new family of children was sent to the White House."
  2. Saunders 2014: "A sequel to E. Nesbit "Five Children" books set during World War I."

--Pwendt|talk 22:59, 10 May 2017 (UTC)

I'd say that if there is more than one series in the same universe/story-universe (or a series and individual books), it is a Universe and we need a series to show the connection. How would you call the "Five Children Universe" one? The sequels are not properly part of the series (so adding them to the series itself is a bad idea) and not connecting them makes no sense. Annie 23:03, 10 May 2017 (UTC)
I think having an overarching series is good in this case. Since they are unofficial sequels, especially. ···日本穣 · 投稿 · Talk to Nihonjoe 23:08, 10 May 2017 (UTC)
I am not sure what the consensus is as of 2017, but in the past a number of editors felt strongly that "sequels by other hands" should not be put in the same series as the original books. Not all of our series are handled that way, e.g. consider The Swiss Family Robinson or Pym, but many are. Ahasuerus 01:23, 11 May 2017 (UTC)

Shakespeare and his co-authors

Is there a criterion for booking William Shakespeare as a formal co-author of some latterday work. Do we, should we, indeed rely on the title page alone, as the manual suggests, or sometimes book a Shakespeare as a co-author "regardless"? (to be continued)

Recently I broke some relationships as variants --that is, freed some children-- of our 5 Shakespeare dramas (as SHORTFICTION, perhaps an exception to the general scope of the database). And added 3 such works to the existing series A Midsummer Night's Dream, which previously contained the Shakespeare drama and Carter short story alone.

Is there consensus that we should use series in this way? Certainly it will make the Shakespeare summary bibliography easier for visitors to read, and easier for editors to improve. If generally approved in advance, I will create shortfiction series led by the 4 other Shakespeare dramas (Hamlet, Macbeth, The Tempest, The Winter's Tale).

--Pwendt|talk 23:32, 10 May 2017 (UTC)

There's no exception or special criterion that I'm aware of. However, I'm not sure what you are asking about and so can't say if there is consensus or not.
Did you mean Shakespeare's Stories for Young Readers which is a publication of Shakespeare stories as told by E. Nesbit? The titles of the Shakespeare works have "(abridged)" appended to them as a way of denoting that these are not the same text as the Shakespeare versions of the works. The stories are co-authored as being by William Shakespeare and E. Nesbit. I wished that publication record had included a note about what's on the title page. There's an Amazon Look Inside available but it does not include the title page. I suspect the title page only has "E. Nesbit" as that's what's on the front cover. The publication record should also have had a note about that within the publication the stories are only preceded by it's title and do not credit the authors. However, it's clear from the author's introduction that she revised the Shakespeare stories so that they could be enjoyed by children.
Another way that publication could have been handled is to only credit William Shakespeare and to variant title the stories from E. Nesbit to William Shakespeare. That has the benefit of "Used These Alternate Names: William Shakespeare" appearing at the top of E. Nesbit's bibliography and "Used As Alternate Name By: E. Nesbit" appearing at the top of William Shakespeare's bibliography.
Still another way that publication could have been handled is to not credit Shakespeare at all on the theory that they are E. Nesbit stories patterned after Shakespeare works. The downside is it means someone looking at Shakespeare's bibliography would not know that the Nesbit version was available.
Had I entered this publication I likely would have used the variant title route as Nesbit and Shakespeare were not co-authors. The author's introduction makes it clear she intended to re-tell the stories for a young audience. The publication only credits Nesbit as the author on the cover. I'll assume the title page credit is the same. The individual stories do not have author credits which gives us some leeway when it comes to what we enter the publication record. --Marc Kupper|talk 18:30, 11 May 2017 (UTC)

Primary Verifications - new section is hard to spot

It used to be that Verifications was a single section with the primary verifications color coded in green. I suspect the color coding did not work well for color-blind people but it did make it easy to tell at a glance if a publication was primary-verified.

We now have two sections, "Primary Verifications", and "Secondary Verifications." Numerous times recently I've glanced at a page and thought "there are no primary verifications" as visually it's very similar to the Contents section that's above it. The PVs are now colored gray. Can we add some color coding to the Primary Verifications section? I'd be happy with the green that you used to use but am posting the request here to see if others have ideas. --Marc Kupper|talk 07:18, 11 May 2017 (UTC)

Sorry, my bad. I didn't think of the impact on the color scheme when redoing verifications because... well, I guess because I generally don't think in terms of colors. I'll happily change all primary verifications to green or anything else that our color-enabled editors may like! Ahasuerus 14:12, 11 May 2017 (UTC)
I think that green should be fine - and consistent with the table below it. Annie 16:19, 11 May 2017 (UTC)
Done! Ahasuerus 17:12, 11 May 2017 (UTC)
Did you loose the white background on purpose? See here for example. Annie 17:28, 11 May 2017 (UTC)
It seems to look OK under Firefox and Chrome. The HTML also validates when run against the W3 validator. Are you using a different browser? What kind of problem are seeing? Ahasuerus 17:47, 11 May 2017 (UTC)
Browser issue apparently - now it looks fine. The two verifications sections were missing their white boxes around them on Firefox 53.0.2 for some reason and even Ctrl+F5 could not get them to show... Sorry for bothering you :) Annie 17:53, 11 May 2017 (UTC)
No worries. The patch also fixed a pre-existing CSS bug; perhaps Firefox was confused until something forced it to flush one of its caches. Ahasuerus 18:03, 11 May 2017 (UTC)
Thank you. At first I was confused as the sections did not have a white background but think you made a change in the last few minutes and a sample of test cases looks good.
  • [4] Standard pub with no primary verifications.
  • [5] Standard pub with primary verifications.
  • [6] Chapbook pub with small Contents section and no primary verifications.
  • [7] Chapbook pub with small Contents section and with primary verifications.
--Marc Kupper|talk 19:44, 11 May 2017 (UTC)

ASIN-UK

I am currently working on adding support for "third party identifiers" like ASINs and LCCNs. It would appear that different Amazon stores (UK, FR, JP, etc) can use different ASINs for most of their products. However, e-books appear to be an exception: the same ASINs are apparently used by all of their stores. If true, it means that we probably won't need separate country-specific ASIN IDs (or Notes templates) once we add support for third party identifiers. The display will look something like:

ASIN B06Y1L3VRG: CA, DE, FR, JP, UK, US
COPAC 11111: Web view, XML view
LCCN: 98765 [note the different display format since there is only 1 link]
OCLC: 12345

Does this look about right? Or are there e-books that different Amazon stores list under different ASINs? Ahasuerus 18:41, 11 May 2017 (UTC)

There had been historically e-books available only in some of the regions and in this case the links won't work for the places where it is not valid. I am not sure if these days there is a case where the same ebook has different ASINs but I will check some of my older books tonight and see if I can discover something... Annie 19:02, 11 May 2017 (UTC)
I am sure some ASIN links won't work in some countries. On the other hand, the only alternative that I can think of would be to enter one ASIN per country. That would result in an awful lot of duplicate ASINs. It would also require a great deal of manual work in order to check what's available at each country-specific store. And, of course, Amazon can make a subset of the "missing" ASINs available at any time. Ahasuerus 19:25, 11 May 2017 (UTC)
True. Quick check on my earliest e-books show that the ASINs are the same on both UK and US... so I suspect we can assume so unless if someone knows something else. Annie 19:45, 11 May 2017 (UTC)
It would make things a little more complicated, but the duplicate ASINs could be addressed by having the database compare submissions of new ASINs against those already in use for the publication. If they are the same, it could auto-merge the duplicate ASINs (or create a cleanup report that would allow mods or other users to verify they are indeed the same product). This would help prevent the duplicate ASIN bloat you describe. ···日本穣 · 投稿 · Talk to Nihonjoe 19:47, 11 May 2017 (UTC)
I'm not sure what the issue is. We have long allowed for duplicate ISBNs. More recently logic was added to the moderator-approval page to highlight that an ISBN or ASIN was duplicated. Most of the time the warning is spurious as the record being added is for a different printing than existing records. People are encouraged to specify the sources for information that they add to a publication record. Ideally, they are already saying if the source was amazon.com, amazon.ca, amazon.co.uk, etc. rather than just "Amazon". If the notes say the source was amazon.com and someone later discovers that ASIN or ISBN is a 404 on another site then it's not on that site. Do we really need to add logic to note that at this date and time from this source IP address a particular URL did not work? Not all ISBNs are on all the Amazon sites. ASINs are no different.
BTW, is isfdb's logic aware of the ISBNs that only work on www.bn.com? Amazon uses ASINs for e-books. Barnes and Noble uses what look like ISBNs for e-books but they only work on www.bn.com and are a 404 on Amazon's sites. I can't recall the prefix though. --Marc Kupper|talk 20:38, 11 May 2017 (UTC)
The folks at B&N use what they call "BN IDs", which are 13 digits long and kind of look like ISBN-13s. They even used to call them "ISBN-13"s on their product pages. However, their IDs start with 294, which would make them invalid if they were ISBNs. B&N no longer calls them "ISBN-13s", e.g. see the "BN ID" field on this page. Ahasuerus 20:54, 11 May 2017 (UTC)
I guess the next question is whether we want to add "BN IDs" to the list of supported third party identifiers. At the moment, it includes the following types of identifiers:
  • ASIN
  • BL
  • BNB
  • BNF
  • COPAC
  • DNB
  • FantLab
  • Goodreads
  • JNB
  • LCCN
  • NDL
  • OCLC
  • Open Library
  • SFBG
but we can certainly add more. Ahasuerus 21:00, 11 May 2017 (UTC)
Thank you for the BN ID explanation and link to an example. I'm confused by some of those as they are web sites that index what's available elsewhere but do not directly provide copies of a story for reading. While I can see citing one of them as a source of information about a story I would never expect to enter their ID number in ISFDB's ISBN/Catalog # field for a publication record. What field is this thread about? --Marc Kupper|talk 21:28, 11 May 2017 (UTC)
It's about the new group of fields that I am currently working on, "external identifiers" -- see this FR. Adding software support for them will enable robots like Fixer to submit ISBN-less e-books. In addition, moving these identifiers from notes to a separate field will enable users to search the database by OCLC/LCCN/etc number as well as other Good Things :-) Ahasuerus 21:49, 11 May 2017 (UTC)

Does Wattpad count as being published?

I know we've decided that a story appearing on the author's own website does not mean it has been published and is not listed as a first appearance date in the database. Should appearing on Wattpad be treated the same? It seems similar to me. --Vasha 20:25, 11 May 2017 (UTC)

I see several troublesome aspects:
  • Many but not all authors use handles rather than names.
  • I could not see publication dates for the stories.
  • It's not clear if authors are allowed to revise or delete a story. Once we link to something would that content be stable?
Offhand though - there's not much to distinguish a wattpad.com from an amazon.com from someone putting something on their own web site. I believe Amazon.com now allows people to set a price of $0.00. It used to be the minimum was $0.99 but recently I was adding an author to isfdb and saw that some of her ebook editions were priced at $0.00. --Marc Kupper|talk 21:06, 11 May 2017 (UTC)
Sometimes $0.00 prices are promotional, but there are tricks that can force Amazon to set them to $0.00 permanently -- see this post. Ahasuerus 21:23, 11 May 2017 (UTC)
If you can buy it/get it, it is published in my book. I still think that we need to revisit the rules around what is allowed in terms of web content considering the changing realities of books publishing and maybe instead of checking sites one by one, we need to reopen that discussion... :) Annie 21:31, 11 May 2017 (UTC)

Sluggish times?

Hello, it seems that the db is particularly slow this morning. A general problem? Hauck 07:30, 12 May 2017 (UTC)

Sluggish for me as well for the last at least 30 minutes. Annie 07:41, 12 May 2017 (UTC)
We haven't had significant performance problems in a while, so I don't have any of my performance profiling tools enabled. If the issue persists, I can enable them and see when the spikes occur. Ahasuerus 14:05, 12 May 2017 (UTC)
It went back to normal in about a hour. Hauck 15:11, 12 May 2017 (UTC)

External identifiers - preliminary cleanup

As indicated earlier, I am currently working on adding support for external identifiers (OCLCs, LCCNs, ASINs, etc.) I need to clean up certain things behind the scenes before I can add the new functionality.

I plan to deploy a few minor patches over the next day or two. They won't change anything that editors normally see, but it's possible that they may introduce new bugs. If you see anything unusual, please post your findings here. Ahasuerus 22:55, 12 May 2017 (UTC)

Seiun Awards for foreign works

For Best Translated Long Story and Best Translated Short Story, the entries were put in before we had Japanese support for titles. Should I go through and change the awards to point to the Japanese title since it was the Japanese translation that got the award? This may involve me creating a new variant entry in some cases, but that's fine. ···日本穣 · 投稿 · Talk to Nihonjoe 05:50, 13 May 2017 (UTC)

The software fully supports awards given to variant titles/translations, so I don't think there should be a problem. Ahasuerus 16:54, 15 May 2017 (UTC)
Not really a problem, but: this can result in a mix of original and translated titles on the author's award page. Example: Wolfgang Jeschke's awards, see The Cusanus Game and Das Cusanus-Spiel there. That was one of the reasons why I linked the award records for Best Foreign Work of the Kurd-Laßwitz-Preis to the original title and not to the translation. Another reason was, that the award is given for the work, not the translation. However, a disadvantage of the way I did it is that the titles on the award year page become a mixture of languages, for example German, Chinese and English on the 2017 Kurd-Laßwitz-Preis page (see Cixin Liu's entry there), which deviates from the way award nominations and results are usually published (using the titles of the translations) by the award committees. Maybe the software could add a "translation of TITLE X" for award records where applicable. Jens Hitspacebar 17:42, 15 May 2017 (UTC)
In this case, the award is specifically for the translation. Hence, "Best Translated X". ···日本穣 · 投稿 · Talk to Nihonjoe 17:55, 17 May 2017 (UTC)
Not sure what you want to say. That's what I was talking about as well above, in the sense of "award given to an author for his work which got translated into the award's language" (and not in the sense of "award given to the translator for the quality of the translation"). Or do you mean something different? :) Jens Hitspacebar 18:53, 17 May 2017 (UTC)
I'm saying the award is given to the translated work, not to the original work. As I've never attended one of their ceremonies, I don't know if they specifically call out the translator for doing a good job. ···日本穣 · 投稿 · Talk to Nihonjoe 19:27, 17 May 2017 (UTC)
Interesting points. Let me take a closer look at the software... Ahasuerus 18:32, 15 May 2017 (UTC)
(after sleeping on it) Since title awards can be linked to VTs, I think the parent data should be displayed on all award pages, including the author-specific Award Bibliography page and the Award Year page. For translation, it will say "translation of ..." while other types of variants will say "variant of ...", which is how the Content section of the Publication page handles it. Ahasuerus 17:37, 16 May 2017 (UTC)
Sounds good. Jens Hitspacebar 08:47, 17 May 2017 (UTC)
OK, FR 1058 has been created. Once I finish implementing "external identifiers", I will take care of the minor FRs and bugs that have been piling up for the last few weeks. Ahasuerus 15:47, 17 May 2017 (UTC)
Sounds good. I will work on switching the awards over to the correct titles. ···日本穣 · 投稿 · Talk to Nihonjoe 17:55, 17 May 2017 (UTC)

Unsupported HTML in notes/synospes - Bad HREFs

The recently implemented "invalid/unsupported HTML" reports have been updated to look for malformed HREFs. The data (25 pubs and titles) will become available tomorrow morning. Ahasuerus 16:35, 15 May 2017 (UTC)

List "Variant Title" and "Translation" on author's page separately?

Just a minor thing, but with the title pages now having variant titles split up nicely into the "Variant Titles", "Translations", "Serializations" and "Serialized Translations" sections, shouldn't the same distinction be made on the author's page to make it all more consistent? It'd first list all "Variant Title" records of the parent title as bullet points, then all "Translation" title records etc. Jens Hitspacebar 21:11, 15 May 2017 (UTC)

Well, the Title page is a little different. With a table, we can have up to 4 columns and then we sort individual VTs chronologically within each column. I suspect that a similar layout would look very busy on the Summary Bibliography page, so we are stuck with the current list layout.
We could split VT lists into 4 sections (VTs, translations, etc), but then we would lose the chronological ordering. If we decided to do that, I think the best section order would be:
  • Variant Titles
  • Translations
  • Serializations
  • Serialized Translations
Ahasuerus 18:22, 16 May 2017 (UTC)
Yes, that was what I had in mind: keep the bullet lists but organize them into the sections you mentioned above, one below the other. Changing the whole layout to a table-based would be over the top. Another, even simpler solution which keeps chronological ordering would be to just replace the "Variant Title" label with "Translation" where applicable and only split "Serializations" into "Serializations" and "Serialized Translations" ("Serializations" is already a separate section with its own chronological ordering anyway, therefore splitting it into two sections shouldn't be such a big change for users). I think the latter solution is sufficient because it'd implement what my main concern was: label it as a translation and not as a variant title if it is a translation. Jens Hitspacebar 08:45, 17 May 2017 (UTC)
Sounds good. FR 1059 has been created. Ahasuerus 18:14, 17 May 2017 (UTC)

Publication summary display issue

The standard practice is to have the date of a variant title be the first date that title+author name combination appeared. That's fine, except it leads to the following issue: On publication summary pages, contents items that differ from the canonical title in title and date have (variant of CanonicalTitle Year) appended to them; contents items that differ in only author and date have [as by Pseudonym] appended to them. So the original publication year is not displayed in the latter case. It would be nice to come up with a way of having the original year there. I'll admit I can't think of one at the moment. --Vasha 22:15, 19 May 2017 (UTC)

Further thought: We are already treating translations and variant titles differently for many purposes. So maybe in a table of contents things should be displayed like this: VariantTitle • (OriginalYear) • length by CanonicalAuthor (variant of CanonicalTitle) [as by Pseudonym] OR in the case of translations, TransTitle • (TransYear) • length by CanonicalAuthor (translation of CanonicalTitle OriginalYear) [as by Pseudonym]. So for variant titles in the original language, the date of the variant wouldn't be displayed here, but I generally find it confusing having it in tables of contents anyway. --Vasha 22:27, 19 May 2017 (UTC)

Doctor Who Audio plays/novels/audiobooks

Are the Big Finish Doctor Who audio plays/novels/audiobooks (call them whatever you want - different places call them differently) eligible for inclusion? We already have a few in Original Audiobooks (Big Finish). I do not think that they are any different from their BBC counterparts (which are an extension of the printed series and as such eligible I think -- we have a few of them over here and the only difference between these and the printed series is the formats... I think that they should be eligible but wanted a confirmation before I get this series sorted out and get a lot more of these added. Thanks! Annie 20:14, 22 May 2017 (UTC)

And if the decision is that they are not eligible, some of those above will need to be zapped :) Annie 21:13, 22 May 2017 (UTC)
That Original Audiobooks (Big Finish) series looks like a mixture of some eligible and some non-eligible. The title of the series is misleading because some of those included are full cast audio dramas, not books, such as The Land of the Dead, The Apocalypse Element and the 'Gallifrey' titles. These don't qualify for inclusion and I remember removing about 50 of the audio dramas a few years ago after some discussion (including the first two above titles that have since reappeared). As for the rest, I expect titles that are indicated as "read by..." on their covers are eligible for inclusion as audiobooks, but it will also be worth checking if there is an actual book that serves as the source for the audio. I'll happily remove the dramas. PeteYoung 13:25, 24 May 2017 (UTC)
Let's remove the dramas then. :) There are some of the audio books that were never published as printed books but are novels actually and are part of official novels lists (don't get me started - confusing does not start covering it). I guess I will need to go and look through all those disks and figure out which ones are what. Single reader - eligible; multiple readers - ineligible (except when it is a multi-cast reading of an existing text). Is this what we are converging to? I am happy with this distinction. Annie 16:50, 24 May 2017 (UTC)

If Magazine now online

Of interest to many editors here, the complete run of If is now available for free in multiple formats at the Internet Archive. You're welcome. PeteYoung 12:39, 24 May 2017 (UTC)]

Oooooh. ···日本穣 · 投稿 · Talk to Nihonjoe 19:30, 24 May 2017 (UTC)
Hey, I was planning to sleep some time this decade... :) Annie 19:51, 24 May 2017 (UTC)

Server downtime at 5pm - External Identifiers are finally here

The server will be down for patching between 5:00pm and 5:10pm server time. The patch will add support for publication-specific external identifiers. Patch notes to follow. Ahasuerus 20:37, 24 May 2017 (UTC)

The server is back up. Patch notes:

The following types of identifiers will be supported out of the box (more can be added if needed):

+----------------------+----------------------------------------+
| ASIN                 | Amazon Standard Identification Number  |
| BL                   | The British Library                    |
| BN                   | Barnes and Noble                       |
| BNB                  | The British National Bibliography      |
| BNF                  | Bibliothèque nationale de France       |
| COPAC                | UK/Irish union catalog                 |
| DNB                  | Deutsche Nationalbibliothek            |
| FantLab              | Laboratoria Fantastiki                 |
| Goodreads            | Goodreads social cataloging site       |
| JNB                  | The Japanese National Bibliography     |
| LCCN                 | Library of Congress Control Number     |
| NDL                  | National Diet Library                  |
| OCLC/WorldCat        | Online Computer Library Center         |
| Open Library         | Open Library                           |
| SFBG                 | Catalog of books published in Bulgaria |
+----------------------+----------------------------------------+

Any number of identifiers can be entered when adding, cloning or editing a publication. Once associated with a publication, an external identifier appears on the Publication page and is automatically linked to the associated third party Web page. A publication can have more than one identifier of the same type, e.g. it can link to 2+ OCLC records. The functionality is similar to templates, but it’s more advanced because a single identifier can link to multiple Web pages. For example, an ASIN identifier links to the 6 supported Amazon stores while COPAC identifiers link to two different versions of the COPAC data: human readable and XML-based. You can still use linking templates in notes, but it’s best to limit their use to third party records which are not directly related to the publication, e.g. "The skeleton OCLC record {{OCLC|123456}} may be related."

The vast majority of ASINs (over 46,000) have been automatically migrated from Notes to the new field. A few thousand will have to be migrated manually. There is a new cleanup report to facilitate the process. The first iteration of the report is limited to the most straightforward cases; the data (146 pubs) will become available tomorrow morning. OCLC, LCCN, BL and other record ID haven’t been touched yet, but I’ll try to automate the conversion process to the extent possible.

A new Advanced Search, “Publication Search by External Identifier”, is now available. It does exactly what you would expect it to do. We may want to discuss making external identifiers available on certain other pages, e.g. in the standard “publication table” used by the Title page, Advanced Publication search, etc.

As always, a major patch may contain bugs or break something unrelated. If you notice anything unusual, please post your findings here.

Once the dust settles, I will create a cleanup report to look for e-books without ISBNs, ASINs or BN IDs. I will also enhance Fixer’s logic to check external identifiers when creating new submissions. Once everything is in place, Fixer will be able to create submissions for ISBN-less publications, something that he has been unable to do up until now.

Here is an example of what a publication record with multiple external IDs looks like. Ahasuerus 21:07, 24 May 2017 (UTC)

Compatibility Issues

The "add external identifier button" is not working. --Vasha 00:48, 25 May 2017 (UTC)

Have you tried a full page reload (Control-F5 in most browsers)? Ahasuerus 01:04, 25 May 2017 (UTC)
Fully quit and restarted the browser, still nothing. I am using Firefox. --Vasha 01:15, 25 May 2017 (UTC)
And... it does work in Safari. So why not Firefox? I have rather a lot of plugins... can list them for you if you need --Vasha 01:21, 25 May 2017 (UTC)
It's working under Firefox on my side, so I suspect that you are right and that there is something about your installation that causes a conflict. What's odd is that the added JavaScript code is similar to the code responsible for all the other "Add ..." buttons. Hm. Could you check the console? It's under Tools, Web Developer, Web Console. Once the console area appears at the bottom of the page, please click on the "JS" mini-tab and check the "Error" and "Warning" boxes, then reload (Control-F5) the page to see if anything shows up in the console area. TIA! Ahasuerus 01:53, 25 May 2017 (UTC)
I removed an extension called Awesome Timestamp and it's working now. Whew. --Vasha 14:24, 25 May 2017 (UTC)
Glad the mystery has been solved! Sorry about the conflict, but, unfortunately, there is not much we can do about add-on conflicts on the software side... Ahasuerus 20:50, 25 May 2017 (UTC)

Lowercase 'x' in ISBNs

Another odd thing (in Safari, this time). I entered the ISBN "014008374x" with a lowercase x and got the message "Catalog number is missing etc." but when I entered it uppercase "014008374X" it was accepted. I'm pretty sure that didn't use to happen. --Vasha 01:47, 25 May 2017 (UTC)

I think I encountered this scenario a couple of months ago. It will go away when we add a separate field for catalog IDs. Ahasuerus 01:56, 25 May 2017 (UTC)

Tweaking the Format and Location of External Identifiers

What about pushing the external identifiers below the notes field? With some exceptions (ex. LCCN), they normally aren't information from the pub itself & they seem pretty prominent in the example. Maybe it just because it's a change to what I'm used to, but I find them distracting in the current location. -- JLaTondre (talk) 22:05, 24 May 2017 (UTC)

Sure, we can do that. Let's wait 24 hours and see if there are any other suggestions. Ahasuerus 23:05, 24 May 2017 (UTC)
I love these. But I agree - they should move somewhere else - they clutter the page a bit. I wonder if a separate container (like the ones for the two types of notifications) won't make sense here... Annie 23:09, 24 May 2017 (UTC)
I also find this "mass" of (for me) uninteresting and likely-to-grow-bigger (with all the public libraries of the world) data particularly annoying. IMHO this should go either on a separate page or way out of sight ;-). Just think of users (like me) that sometimes consult the db on a smartphone with a quite small screen. Hauck 06:51, 25 May 2017 (UTC)

(unindent)I agree with moving the External IDs further down -- I'd rather have more important information near the top. I'm also concerned about the appearance for records that have, say 8 OCLC numbers. With books from before ISBNs, it's not uncommon for the WorldCat system to be unable to link related OCLC records (via the "View all formats" option). Hence there are times when I've included as many as 8 OCLC records for one book. When they're just in a comma separated list, that's not ugly; but when each one gets their own line, it just looks ugly. Chavey 14:11, 25 May 2017 (UTC)

It sounds like we have consensus re: moving external identifiers to the bottom of the section. Let me see if I can do it real quick. I will also try to change the logic to display multiple OCLC/etc records on the same line. Ahasuerus 15:43, 25 May 2017 (UTC)
OK, External IDs have been moved below the Note field. Ahasuerus 20:51, 25 May 2017 (UTC)
The display software has been changed to show all IDs of the same type on one line. For example, this pub has 4 OCLC records associated with it and they all appear on the same line. Ahasuerus 01:56, 26 May 2017 (UTC)
Excellent! Chavey 05:15, 26 May 2017 (UTC)

Catalog IDs

Is there a plan for getting publisher catalog codes out of the ISBN field (the one field=one data principle of database design)? --Vasha 00:39, 25 May 2017 (UTC)

Yes, indeed! As the Roadmap says:
  • Move catalog IDs to a separate field, which will help with pubs that have both an ISBN and a catalog ID, e.g. book club publications.
Ahasuerus 01:05, 25 May 2017 (UTC)
There isn't a 'BCE' option in the drop-down menu. --~ Bill, Bluesman 18:10, 25 May 2017 (UTC)
'BCE' as in "Book Club edition"? Ahasuerus 19:44, 25 May 2017 (UTC)
Yes, though if there's going to be a new field for Catalog #s it doesn't really matter. Saw that note above after posting. Hopefully that new field will be directly below the ISBN field as pre such #s it's the identifier of the record in most cases. --~ Bill, Bluesman 22:12, 25 May 2017 (UTC)
That's the plan! Ahasuerus 22:17, 25 May 2017 (UTC)

Help Updates

Can we add the table from above to here? Annie 23:52, 24 May 2017 (UTC)

Done. Ahasuerus 00:36, 25 May 2017 (UTC)
In addition, the mouseover Help has been updated to display full identifier type names when hovering over drop-down lists. Ahasuerus 18:06, 26 May 2017 (UTC)

BNB links

Can you check your code for the link for BNB? If you click on the link for it from the new fields (use your own example above), it leads to an empty result page in BNB. The code is correct - once you do a search for it, it find it - but that initial search seems to be sending some additional details wrong. Annie 20:01, 25 May 2017 (UTC)

The link was working yesterday, but today all BNB IDs and templates are broken. Let's wait a day or two to see if it's a temporary glitch. If not, we'll switch to the full "search" URL.
That's the nice things about IDs and templates -- when a site changes its URL schema, all we need to do on our side is change the ID/template definition instead of updating hundreds or thousands of Note fields. Ahasuerus 22:21, 25 May 2017 (UTC)
Ah, so the first time I decided to check a BNB link and their server is broken. I guess I should stay away from any other server I had not tried yet :) Thanks for checking :) Annie 22:40, 25 May 2017 (UTC)

LCCN link bug

I've also entered a bug about a problem in interpreting certain uncommon LCCN numbers. Chavey 14:11, 25 May 2017 (UTC)

Fixed. The LCCN associated with the affected pub has been changed to "a 48004732". Ahasuerus 17:01, 26 May 2017 (UTC)

Barnes & Noble IDs

Here is a project for anyone interested. We have 170 publications with "294" ISBNs. Almost all of them were published by "Barnes & Noble Books / Google Books".

294 "ISBNs" are not real ISBNs, they are Barnes & Noble's proprietary IDs. They look like they may be EANs, but they are definitely not ISBNs. We'll need to move them from the ISBN field to the "External IDs" field. Ahasuerus 23:10, 24 May 2017 (UTC)

I tried one and I like it. Awesome addition! ···日本穣 · 投稿 · Talk to Nihonjoe 23:22, 24 May 2017 (UTC)
I'll get them cleared this weekend - had been slowly chipping at them when I have a minute. Annie 18:45, 26 May 2017 (UTC)
Thanks! Once they are gone, we'll be able to close FR 1055. Ahasuerus 21:55, 26 May 2017 (UTC)

"ASIN-UK" removed

Now that we know that different Amazon Web sites use the same ASINs (books only), I have removed "ASIN-UK" from the list of supported templates. All pre-existing occurrences of the template have been replaced with external IDs. Ahasuerus 23:11, 25 May 2017 (UTC)

Functionality to add to "Make Variant"

Hello, is it possible to add the "Check for duplicate title" button in the result screen of "Create New Parent Title" (Option 2) of "Make This Title a Variant Title or Pseudonymous Work". This function would check duplicate for the new Title/Author couple and will avoid the task of manually checking if this new combination is already present in the db (if it's not already been done by the "Check for Duplicate Titles" option, a phase that I usually forget). Thanks. Hauck 08:36, 26 May 2017 (UTC)

Hey, great idea. --Vasha 12:26, 26 May 2017 (UTC)
Done. "Check Parent Title for Duplicates" is now available on the two Make Variant post-approval pages. I am not sure how useful it will be when processing Make Variant submissions which link to existing titles, but it's harmless and there is something to be said for consistency. Ahasuerus 15:30, 26 May 2017 (UTC)
Thanks. Hauck 16:49, 26 May 2017 (UTC)

ASIN Cleanup Part 2

A batch of 900 pubs with ASINs in notes has been converted to use external identifiers. The ASIN cleanup report has been modified and will find 193 additional pubs tomorrow morning. Ahasuerus 03:13, 28 May 2017 (UTC)

The ASIN cleanup report has been tweaked to find additional (ca. 200) ASINs.
Also, a new cleanup report has been created and deployed. It looks for publications with explicit BL/BLIC URL in notes. Please note that BL URLs have changed in the last 12 months, so some links may not work when you click them. Ahasuerus 04:14, 30 May 2017 (UTC)
Personal tools