Thursday, August 21. 2014
If, as rumoured, google builds a platform for depositing unrefereed research papers for “peer-reviewing” via crowd-sourcing, can this create a substitute for classical peer-review or will it merely supplement classical peer review with crowd-sourcing?
In classical peer review, an expert (presumably qualified, and definitely answerable), an "action editor," chooses experts (presumably qualified, and definitely answerable), "referees," to evaluate a submitted research paper in terms of correctness, quality, reliability, validity, originality, importance and relevance in order to determine whether it meets the standards of a journal with an established track-record for correctness, reliability, originality, quality, novelty, importance and relevance in a certain field.
In each field there is usually a well-known hierarchy of journals, hence a hierarchy of peer-review standards, from the most rigorous and selective journals at the top all the way down to what is sometimes close to a vanity press at the bottom. Researchers use the journals' public track-records for quality standards as a hierarchical filter for deciding in what papers to invest their limited reading time to read, and in what findings to risk investing their even more limited and precious research time to try to use and build upon.
Authors' papers are (privately) answerable to the peer-reviewers, the peer-reviewers are (privately) answerable to the editor, and the editor is publicly answerable to users and authors via the journal's name and track-record.
Both private and public answerability are fundamental to classical peer review. So is their timing. For the sake of their reputations, many (though not all) authors don't want to make their papers public before they have been vetted and certified for quality by qualified experts. And many (though not all) users do not have the time to read unvetted, uncertified papers, let alone to risk trying to build on unvalidated findings. Nor are researchers eager to self-appoint themselves to peer-review arbitrary papers in their fields, especially when the author is not answerable to anyone for following the freely given crowd-sourced advice (and there is no more assurance that the advice is expert advice rather than idle or ignorant advice than there is any assurance that a paper is worth taking the time to read and review).
The problem with classical peer review today is that there is so much research being produced that there are not enough experts with enough time to peer-review it all. So there are huge publication lags (because of delays in finding qualified, willing referees, and getting them to submit their reviews in time) and the quality of peer-review is uneven at the top of the journal hierarchy and minimal lower down, because referees do not take the time to review rigorously.
The solution would be obvious if each unrefereed, submitted paper had a reliable tag marking its quality level: Then the scarce expertise and time for rigorous peer review could be reserved for, say, the top 10% or 30% and the rest of the vetting could be left to crowd-sourcing. But the trouble is that papers do not come with a-priori quality tags: Peer review determines the tag.
The benchmark today is hence the quality hierarchy of the current, classically peer-reviewed research literature. And the question is whether crowd-sourced peer review could match, exceed, or even come close enough to this benchmark to continue to guide researchers on what is worth reading and safe to trust and use at least as well as they are being guided by classical peer review today.
And of course no one knows whether crowd-sourced peer-review, even if it could work, would be scaleable or sustainable.
The key questions are hence:
1. Would all (most? many?) authors be willing to post their unrefereed papers publicly (and in place of submitting them to journals!)?My own prediction (based on nearly a quarter century of umpiring both classical peer review and open peer commentary) is that crowdsourcing will provide an excellent supplement to classical peer review but not a substitute for it. Radical implementations will simply end up re-inventing classical peer review, but on a much faster and more efficient PostGutenberg platform. We will not realize this, however, until all of the peer-reviewed literature has first been made open access. And for that it is not sufficient for Google merely to provide a platform for authors to post their unrefereed papers, because most authors don’t even post their refereed papers in their institutional repositories until it is mandated by their institutions and funders.
Harnad, S. (1998/2000/2004) The invisible hand of peer review. Nature [online] (1998), Exploit Interactive 5 (2000): and in Shatz, B. (2004) (ed.) Peer Review: A Critical Inquiry. Rowland & Littlefield. Pp. 235-242.
Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne 35.
Harnad, S. (2010) No-Fault Peer Review Charges: The Price of Selectivity Need Not Be Access Denied or Delayed. D-Lib Magazine 16 (7/8).
Harnad, S. (2011) Open Access to Research: Changing Researcher Behavior Through University and Funder Mandates. JEDEM Journal of Democracy and Open Government 3 (1): 33-41.
Harnad, Stevan (2013) The Postgutenberg Open Access Journal. In, Cope, B and Phillips, A (eds.) The Future of the Academic Journal (2nd edition). Chandos.
Tuesday, August 5. 2014
DOE: The Importance of Requiring Institutional Repository Deposit Immediately Upon Acceptance for Publication
A peer-reviewed journal article is either accessible to all of its potential users or it is not accessible to all of its potential users (but only to those at subscribing institutions).
Open Access (OA) is intended to make articles accessible (online) to all their potential users, not just to subscribers, sothat all potential users can read, use, apply and build upon the findings, not just subscribers.
OA comes in two forms:
Gratis OA means an article is accessible online to all its potential users.
Libre OA means an article is accessible online to all its potential users and all users also have certain re-use rights, such as text-mining by machine, and re-publication.
For individual researchers and for the general public the most important and urgent form of OA is Gratis OA.
The reason Gratis OA is so important is that otherwise the research is inaccessible except to subscribers: OA maximizes research uptake, usage, applications, impact and progress.
The reason Gratis OA is so urgent is that lost research access means lost research impact and progress. The downloads and citations of papers made OA later never catch up with those of papers made OA immediately:
Gentil-Beccot, A., Mele, S., & Brooks, T. C. (2010). Citing and reading behaviours in high-energy physics: Scientometrics, 84(2), 345-355.
The date when a peer-reviewed paper is ready to be made OA is the date when the final, peer-reviewed draft is accepted for pubication.
Sometimes there can be delays of months between the date of acceptance and the date of publication of the pubisher’s version of record (VOR).
And some (a minority) of publishers have imposed embargoes of up to 12 months from the date of publication before authors can make their articles OA.
The delay from acceptance to publication, and the delay from publication till the end of any OA embargo all add up tp lost research access, uptake, usage, applications and progress.
DOE and OSTI have been directed by OSTP to adopt a policy that ensures that OA is provided to federally funded research — by 12 months after the date of publication at the very latest.
This is not a mandate to adopt a policy that ensures that OA is provided "at the very latest possible date."
Yet that is what DOE has done — no doubt under the influence of the publishing industry lobby.
The interests of research and researchers -- and hence of the public that funds the research -- are that the research should be made OA as soon as possible.
The interests of (some of) the publishing industry are that it should be made OA as late as possible.
The DOE has adopted a policy that serves the interests of the publishing industry rather than those of research, researchers and the tax-paying public.
This is why DOE policy has been so warmly welcomed by the Association of American Publishers (AAP) as well as CHORUS (a publisher consortium expressly created to try to keep access-provision and the timing of compliance with open access mandates under the control of publishers rather than fundees and their institutions).
The simplest remedy for this is not necessarily that the permissible OA embargo length needs to be reduced (though that would be extremely welcome and beneficial too!).
Even within the constraints of a permissible OA embargo of 12 months at the very latest, there is a simple way to make the DOE policy much more powerful and effective, guaranteeing much more and earlier access.
All that has to be done is to make immediate deposit of the author’s final, peer-reviewed draft, in the author’s institutional repository, mandatory immediately upon acceptance.
Not just the metadata: the full final draft.
If the author wishes to comply with a publisher OA embargo, the deposit need not be made OA immediately.
Institutional repositories have an automated copy-request Button with which a user can request a single copy for research purposes, and the author can comply with the request, with just one click each.
This is not OA, but it is almost-OA, and it is all that is needed to maximize research access, usage and progress during any permissible OA embargo.
And besides maximizing access during any permissible OA embargo, requiring immediate institutional deposit also mobilizes institutions to monitor and ensure timely compliance with the funding agency’s requirement.
The metadata for the deposit can be exported from each institutional repository to the DOE PAGES portal immediately, and then the portal, too (like google and google scholar), can immediately begin referring users back to the Button at the institution so the author can provide almost-OA with a single click until the end of any embargo.
There is no need whatsoever to wait either for the publisher’s VOR, or for the end of the publisher’s embargo, or for Libre OA re-use rights: those can come when they come.
But immediate institutional deposit needs to be mandated immediately.
Otherwise the DOE is needlessly squandering months and months of potential research uptake, usage and progress for federally funded research.
Please harmonize the DOE OA policy with the corresponding EU OA policy, as well as the HEFCE OA policy in the UK, the FRS OA policy in Belgium, and a growing number of institutional OA policies the world over.
Monday, June 30. 2014
Richard Poynder: "If you were composing the Subversive Proposal today how different would it be? Would it be different? If so, would you care to rephrase it to fit today’s environment? In other words, how would the Subversive Proposal look if written for a 2014 audience (in less than 500 words)?"SH: Knowing now, in 2014, that researchers won’t do it of their own accord, I would have addressed the proposal instead to their institutions and funders, and in less than 200 words:And this is how I should have written the original Proposal in 1994:"To maximize the access, uptake, usage, progress, productivity, applications and impact of your publicly funded research output, mandate (require) that the refereed, revised, accepted final draft of all articles must be deposited in the author’s institutional repository immediately upon acceptance for publication as a condition for research evaluation and funding. If you allow a publisher embargo on making the deposit OA (freely accessible to all online), implement the automated almost-OA Button (and don’t let the embargo exceed 6-12 months at most). This is called “Gratis Green OA.” Do not pay for Gold OA journal publication fees (“Fool’s Gold”) until global Green OA has made subscriptions unsustainable; then you can pay for Fair-Gold out of your subscription cancellation savings. Fair-Gold will also be Libre OA (with re-use rights such as data-mining, re-mixing and re-publishing). Ignore publishers’ lobbying to the effect that Green OA will destroy peer-reviewed journal publishing: it will re-vitalize it and save the research community a lot of money while maximizing the access, uptake, usage, progress, productivity, applications and impact of their research."FREE ONLINE ACCESS TO REFEREED RESEARCH: A SUBVERSIVE PROPOSAL
Wednesday, June 18. 2014
Important Addendum and Correction from Eloy Rodrigues (June 19):
The two Chinese OA Mandates (NSFC and CAS) came fast (2014), but the possibility of complying with them is coming slowly (no repository till 2016).
In addition, articles need not be deposited until 12 months after publication.
In most fields, especially the fast-moving sciences, the benefits of Open Access (maximised uptake, usage, impact and progress) are biggest and most important within the first year of publication. That is the growth tip of research. Access losses in the first year are never fully caught up in later years. The iron needs to be struck when it is hot.
There are two very simple steps that China can take to minimise the needless loss of research uptake, usage and impact because of lost time:
(1) China should set up the repositories immediately, using the available free softwares such as EPrints and DSpace. It requires only a server and a few hours worth of set-up time and the repository is ready for deposits. There is no reason whatsoever to wait two years. It would also be sensible to have distributed local repositories — at universities and research institutions — rather than just one central one. Each institution can easily set up its own repository. All repositories are interoperable and if and when desired, their contents can be automatically exported to or harvested by central repositories.
(2) Although an OA embargo of 12 months is allowed, China should mandate that deposit itself must be immediate (immediately upon acceptance for publication). Access to the deposit can be set as closed access instead of OA during the embargo if desired, but EPrints and DSpace repositories have the “Request-Copy” Button for closed-access deposits so that individual users can request and authors can provide an individual copy for research purposes with one click each. The repository automatically emails the copy if the author clicks Yes.
Monday, June 9. 2014
On Tue, May 27, 2014, Alicia Wise (ELS-OXF)
Alicia Wise (Elsevier):Stevan Harnad:
I agree completely that Elsevier's Green OA No-Embargo Policy has not changed at all from the way Karen formulated it 10 years ago:
"An author may post his version of the final paper on his personal web site and on his institution's web site (including its institutional respository). Each posting should include the article's citation and a link to the journal's home page (or the article's DOI). The author does not need our permission to do this, but any other posting (e.g. to a repository elsewhere) would require our permission. By "his version" we are referring to his Word or Tex file, not a PDF or HTML downloaded from ScienceDirect - but the author can update his version to reflect changes made during the refereeing and editing process. Elsevier will continue to be the single, definitive archive for the formal published version."But SHERPA Romeo is classifying the Policy correctly as Green (and for some Elsevier journals "Blue," which actually also means Green! But because of Romeo's absurd colour scheme, "Romeo Blue" means that the refereed final draft can be immediately self-archived without embargo, whereas "Romeo Green" is reserved for when both the refereed final draft and the pre-refereeing draft can be immediately self-archived -- which is utterly irrelevant for OA, and causes needless and endless confusion, being at odds with the way "Green OA" is now universally used!)
But I also have to add that some of the confusion is caused by Elsevier's more recent attempts to add some pseudo-legal hedges to its Green OA policy, to the effect that Elsevier's authors retain the right to do everything Karen specified in 2004 except if they are required to exercise that right (by their institutions), in which case they may not do it.
That is every bit as absurd as SHERPA's green/blue distinction, and can and should also be ignored by all authors. But you wanted to learn more...
I think that today, the 10th anniversary of the Elsevier Green OA Policy, would be an excellent day to publicly scrap the empty hedges and re-assert the very progressive and constructive Elsevier Policy as it was and is. The hedges just cause gratuitous confusion and are very bad for Elsevier's image...
With best wishes,
How Elsevier Can Improve Its Public Image
Elsevier's Public Image Problem
Institutions & Funders: Ignore Elsevier Take-Down Notices (and Mandate Immediate-Deposit)
Some Quaint Elsevier Tergiversation on Rights Retention
Publisher Double Dealing on OA
Free Will and Systematicity
Elsevier requires institutions to seek Elsevier's agreement to require their authors to exercise their rights?
Date: Thu, 27 May 2004 23:51:58
From: Stevan Harnad
Elsevier Gives Authors Green Light for Open Access Self-Archiving
Elsevier has just gone from being a Romeo "Pale-Green" publisher to a full Romeo Green publisher: Authors have the publisher's official green light to self-archive both their pre-refereeing preprints and their refereed postprints.
Elsevier has thereby demonstrated that -- whatever its pricing policy may be -- it is a publisher that has heeded the need and the expressed desire of the research community for Open Access (OA) and its benefits to research productivity and progress.
There will be the predictable cavils from the pedants and those who have never understood the real meaning and nature of OA: "It's only the final refereed draft, not the publisher's PDF," "It does not include republishing rights," "Elsevier is still not an OA publisher."
I, for one, am prepared to stoutly defend Elsevier on all these counts, and to say that one could not have asked for more, and that the full benefits of OA require not one bit more -- from the publisher.
For now it's down to you, Dear Researchers! Elsevier (and History) is hereafter fully within its rights to say:
"If Open Access is truly as important to researchers as they claim it is -- indeed as 30,000+ signatories to the PLoS Open Letter attested that it was http://www.publiclibraryofscience.org/cgi-bin/plosSign.pl -- then if researchers are not now ready to provide that Open Access, even when given the publisher's official green light to do so, then there is every reason to doubt that they mean (or even know) what they are saying when they clamour for Open Access."Elsevier publishes 1,700+ journals. That means at least 200,000 articles a year. Eprints.org will be carefully quantifying and tracking what proportion of those 200,000 articles is made OA by their authors through self-archiving across the next few months and years. Indeed we will be monitoring all of the over 80% of journals sampled by Romeo that are already green.
(The following Romeo summary stats are already out of date, because 1700 pale-green journals have now become bright green! http://www.ecs.soton.ac.uk/~harnad/Temp/Romeo/romeosum.html but we will soon catch up at: http://romeo.eprints.org/ [which is under construction, waiting for full journal lists from each of the 93 publishers sampled so far].)
The OA ball is now clearly in the research community's court (not the publishing community's, not the library community's). Let researchers and their employers and funders now all rise to the occasion by adopting and implementing institutional OA provision policies. Don't just sign petitions for publishers to provide OA, but commit your own institution to providing it:
Saturday, May 31. 2014
DSpace follows (not quite "leads"!) EPrints in providing access (not quite "Open Access"!) to research during publisher OA embargo periods (via the facilitated Request-Copy Button):
Just in time, I hope, to help shape the implementation of the US Public Access Policy by ensuring that mandatory deposit is (1) immediate (not waiting to deposit only after the allowable OA embargoes of publishers have elapsed) and (2) institutional (not institution-external).
Institutions can then (a) monitor and ensure compliance with the US Public Access Policy and (b) implement the institutional repository's facilitated Request-Copy Button which allows the author to provide an individual copy to an individual requestor with a single click on a case by case basis during the publisher's OA embargo period.
(Both metadata and full-texts of institutional deposits can then be automatically exported to or harvested by any central repositories desired: disciplinary, national, or even funder-based.)
Wednesday, May 28. 2014
There are many reasons why grumbling about attempts to replicate are unlikely in the physical or even the biological sciences, but the main reason is that in most other sciences research is cumulative:
Experimental and observational findings that are worth knowing are those on which further experiments and observations can be built, for an ever fuller and deeper causal understanding of the system under study, whether the solar system or the digestive system. If the finding is erroneous, the attempts to build on it collapse. Cumulative replication is built into the trajectory of research itself — for those findings that are worth knowing.
In contrast, if no one bothers to build anything on it, chances are that a finding was not worth knowing (and so it matters little whether it would replicate or not, if tested again).
Why is it otherwise in many areas of Psychology? Why do the outcomes of so many one-shot, hit-and-run studies keep being reported in textbooks?
Because so much of Psychology is not cumulative explanatory research at all. It is helter-skelter statistical outcomes that manage to do two things: (1) meet a crierion for statistical significance (i.e., a low probability that they occurred by chance) and (2) are amenable to an attention-catching interpretation.
No wonder that their authors grumble when replicators spoil the illusion.
Yes, open access, open commentary and crowd-sourcing are needed in all fields, for many reasons, but for one reason more in hit-and-run fields.
Video interview of Stevan Harnad by Maciej Chojnowski (CeON, University of Warsaw) prior to Invited Keynote on "How to Formulate Effective Policies to Open Access to Research Worldwide". Conference on Opening Science to Meet Future Challenges. Centre for Open Science, part of the Interdisciplinary Centre for Mathematical and Computational Modelling at the University of Warsaw, 11 March 2014
Saturday, May 24. 2014
To derive the full benefit of the Frontiers in Innovation, Research, Science, and Technology (FIRST) Act, it is now extremely important that all universities and research institutions implement the faciltated copy-request Button in their institutional repositories. (The repository software as well as the Button are free for all.)
The Button ensures that even during a publisher embargo on Open Access (OA) any researcher worldwide can immediately request and any author can immediately provide a single copy of any embargoed deposit for research purposes with just one extra click each — just as long as the author’s final, peer-reviewed draft has been deposited in the repository immediately upon acceptance for publication, rather than only after a publisher OA embargo period has elapsed.
This is not OA. It is only “Almost-OA.” But the copy-request Button ensures that the immediate-deposit does not lie fallow during any allowable OA embargo period. And that’s what research and researchers need most.
For DSpace Repositories: https://wiki.duraspace.org/display/DSPACE/RequestCopy
For EPrints Repositories : http://wiki.eprints.org/w/RequestEprint
Sale, A., Couture, M., Rodrigues, E., Carr, L. and Harnad, S. (2014) Open Access Mandates and the "Fair Dealing" Button. In: Coombe, RJ Wershler, D & Zellinger, M (Eds) Dynamic Fair Dealing. U Toronto Press
Wednesday, May 21. 2014
Sponsored by Microsoft Research Connections
UQÀM Cognitive Sciences Summer School 2014
WEB SCIENCE AND THE MIND
JULY 7 - 18 2014 Universite du Québec à Montreal, Montreal, Canada
Full Programme: http://users.ecs.soton.ac.uk/harnad/Temp/AbsPrelimProg3.htm
Cognitive Science and Web Science have been converging in the study of cognition:
(i) distributed within the brain
(ii) distributed between multiple minds
(iii) distributed between minds and media
SPEAKERS AND TOPICS
Katy BORNER Indiana U Humanexus: Envisioning Communication and Collaboration
Les CARR U Southampton Web Impact on Society
Simon DeDEO Indiana U Collective Memory in Wikipedia
Sergey DOROGOVTSEV U Aveiro Explosive Percolation
Alan EVANS Montreal Neurological Institute Mapping the Brain Connectome
Jean-Daniel FEKETE INRIA Visualizing Dynamic Interactions
Benjamin FUNG McGill U Applying Data Mining to Real-Life Crime Investigation
Fabien GANDON INRIA Social and Semantic Web: Adding the Missing Links
Lee GILES Pennsylvania State U Scholarly Big Data: Information Extraction and Data Mining
Peter GLOOR MIT Center for Collective Intelligence Collaborative Innovation Networks
Jennifer GOLBECK U Maryland You Can't Hide: Predicting Personal Traits in Social Media
Robert GOLDSTONE Indiana U Learning Along with Others
Stephen GRIFFIN U Pittsburgh New Models of Scholarly Communication for Digital Scholarship
Wendy HALL U Southampton It's All In the Mind
Harry HALPIN U Edinburgh Does the Web Extend the Mind - and Semantics?
Jiawei HAN U Illinois/Urbana Knowledge Mining in Heterogeneous Information Networks
Stevan HARNAD UQAM Memetrics: Monitoring Measuring and Mapping Memes
Jim HENDLER Rensselaer Polytechnic Institute The Data Web
Tony HEY Microsoft Research Connections Open Science and the Web
Francis HEYLIGHEN Vrije U Brussel Global Brain: Web as Self-organizing Distributed Intelligence
Bryce HUEBNER Georgetown U Macrocognition: Situated versus Distributed
Charles-Antoine JULIEN Mcgill U Visual Tools for Interacting with Large Networks
Kayvan KOUSHA U Wolverhampton Web Impact Metrics for Research Assessment
Guy LAPALME U Montreal Natural Language Processing on the Web
Vincent LARIVIERE U Montreal Scientific Interaction Before and Since the Web
Yang-Yu LIU Northeastern U Controllability and Observability of Complex Systems
Richard MENARY U Macquarie Enculturated Cognition
Thomas MALONE MIT Collective Intelligence: What is it? How to measure it? Increase it?
Adilson MOTTER Northwestern U Bursts, Cascades and Time Allocation
Cameron NEYLON PLOS Network Ready Research: The Role of Open Source and Open Thinking
Takashi NISHIKAWA Northwestern U Visual Analytics: Network Structure Beyond Communities
Filippo RADICCHI Indiana U Analogies between Interconnected and Clustered Networks
Mark ROWLANDS Miami U Extended Mentality: What It Is and Why It Matters
Robert RUPERT U Colorado What is Cognition and How Could it be Extended?
Derek RUTHS McGill U Social Informatics
Judith SIMON ITAS Socio-Technical Epistemology
John SUTTON Macquarie U Transactive Memory and Distributed Cognitive Ecologies
Georg THEINER Villanova U Domains and Dimensions of Group Cognition
Peter TODD Indiana U Foraging in the World Mind and Online
Syndicate This Blog
Materials You Are Invited To Use To Promote OA Self-Archiving:
The American Scientist Open Access Forum has been chronicling and often directing the course of progress in providing Open Access to Universities' Peer-Reviewed Research Articles since its inception in the US in 1998 by the American Scientist, published by the Sigma Xi Society.
The Forum is largely for policy-makers at universities, research institutions and research funding agencies worldwide who are interested in institutional Open Acess Provision policy. (It is not a general discussion group for serials, pricing or publishing issues: it is specifically focussed on institutional Open Acess policy.)
You can sign on to the Forum here.