Скачать 81.38 Kb.
What publication trends follow from an increasing bias to rigor? While many academics toil selflessly in the cause of research, rigor alone may deny that goal.
IS publication trends
We consider IS publication trends by editors, authors, reviewers and readers.
If editors want a journal of the highest quality, Type I errors of publishing affect journal reputation more than Type II errors. If a journal publishes wrongly readers will know, but a useful paper falsely rejected does not see the publication light of day, and later publication elsewhere can be attributed to a standards difference. Since scarcity increases demand, a high journal rejection rate can be seen as an indicator of quality. Exclusive journals that attract more submissions can reject more to become even more exclusive.
A trend for journals to succeed by rigor means fewer top IS journals over time, and indeed while our field has expanded, overall IS journal rankings have remained remarkably consistent over time (Rainer & Miller, 2005). Already we seem at the point where only MISQ and ISR count and it is impossible to create more “A” journals. Yet two journals generating say 50 papers a year poorly represents a field with 5,000 in its professional society and perhaps 10,000 in all, especially given repeat authors, often senior professors, who have figured out how to publish in them. Such exclusiveness might seem good, but can have harmful effects on promotion prospects in the field as a whole (Kozar, Larsen, & Straub, 2006).
Rigor is easier to maintain for familiar and restricted content. In the 1990’s IS researchers focused on issues critical to IS practitioners a decade earlier (Szajna, 1994), and the situation is probably the same today. If rigor makes journals traditional, expanding multi-disciplinary fields will be poorly represented at the top levels. The rigor trend predicts successful IS journals will be exclusive in participation, restricted in scope, outdated in content and innovation averse. This does not seem a good knowledge exchange outcome.
Authors need publication “notches” on their curriculum vitae belt to survive. University tenure committee members who rate candidates outside their specialty tend to count paper numbers not paper quality. This is more “objective” (in being reliable not in being valid). The effect for authors is that a ground-breaking paper and a trivial one both count as “one”. Despite the huge spin-off benefits of innovation, even valid new papers are risky, as they may not flourish until after the tenure decision, e.g. Einstein’s papers took over a decade to be accepted. Publish or perish means it pays authors to increase paper numbers rather than paper content, e.g. to publish overlapping variants, to publish in least-publishable-units, and to publish in groups. Authors are being pressured to manage, market and network, rather than to think.
Some successful IS authors resubmit rejections directly to other journals, barely reading the reviews, until they get a positive response. The logic is that rejection is as much about attitude as content, and this saves time given long review cycles. In sum, the trend is for authors to flood conferences and journals with more junk than goods, to recycle old ideas with catchy new labels but little new substance, and to make minor incremental “advances” to the gatekeepers favorite theories. This does not seem a good knowledge exchange outcome.
Suppose reviewers, who often labor unpaid and unknown, and are usually over-worked, wish to finish reviews quickly. The easiest option is to simply accept, but if another reviewer finds serious faults, this could be professionally embarrassing. The next easiest reviewer response is to find enough faults to reject the paper. When enough faults are found, the review is over. While to praise when others condemn implies naiveté, a scathing review within a set of praises is commendable rigor. The third alternative, to spend time growing the paper, is more risky. If the authors ignore good advice, the reviewer has wasted their time, while if they take it, the authors get credit for the reviewer’s ideas. In summary, the expected trend is for reviewers to increasingly deny rather than grow value, which does not seem a good knowledge exchange outcome.
Readers read to get meaning for their effort, to get cognitive value without breaking their cognitive bank. More rigor means more complex papers that take more effort to read, but contain less risky innovation. If journals feel obliged to publish the n+1th rigorous paper on a topic, whether it adds anything or not, authors will repeat the same ideas in sophisticated ways. The chance of reading an apparently new paper only to find that it really says little new, goes up. If the number of new ideas per paper is going down, readers getting less value for effort will redress the imbalance. One way is to read more superficially, e.g. skim for keywords rather than read for meaning, skip from beginning to end, or just read the abstract and section headings. Another is to assume groups know more and read what everyone else is reading. The predicted end point is an academic herd grazing on trendy words in a sort of intellectual “groupthink” (Janis, 1972), not an ideal outcome for a system traditionally based on individuals sharing independent thought.
The expected result of a rigor bias is journals more rigorous than relevant, authors more prolific than productive, reviewers denying not inspiring, and readers group grazing but not digesting. These trends may self-reinforce, e.g. few high rigor journals means authors over-submit which increases reviewer denying which reduces reader value. The final vision of journals as exclusive castles of knowledge, manned by editor-sovereigns and reviewer-barons, who raise a barricade of rigor against an unwashed mass of peasant-authors, is not inspiring. The alternative vision, of journals as gardens of knowledge is more akin to the spirit of the Internet itself, where knowledge flows freely in unexpected ways.
The lesson of physics
Physicists learned last century that rigor alone is insufficient for research progress. The established Newtonian framework was rent apart by new theories of relativity and quantum mechanics, which then ushered in the modern world. Physicists learned that theories may be valid but incomplete, so science does not only advance by collecting data like a stone accumulates moss. Knowledge is a process not a thing, so as well as specification, knowledge progress also involves reframing, as the term paradigm shift implies (Kuhn, 1970). If knowledge specification is like an observer seeing more details, a paradigm shift is like the observer seeing from a new vantage point. To see from a new physical point of view the observer must move physically. Likewise to see from a new conceptual point of view the conceiver must move mentally. Both cases require effort and change. Without a paradigm shift, physics would still be specifying the details of Newtonian mechanics, rather than developing into new cosmic, nuclear and quantum realms.
Given this history, physicists value innovative theoretical thinking. Lee Smolin, who developed quantum gravity theory, writes: “… there is now the problem of making sure that young people have the freedom to wander across boundaries established by their elders without fear of jeopardizing their careers. It would be naïve to say this is not a significant issue. In many areas of science we are paying the consequences of an academic system that rewards narrowness of focus over exploration of new areas” (Smolin, 2001, p183).
Murray Gell-Mann, who invented the quark, similarly points out the need to: “… get away from the idea that serious work is restricted to beating to death a well defined problem in a narrow discipline…” (Gell-mann, 1994, p346).
Both suggest academia should invite the young to cross-topic innovation, not make them choose between their career and their creativity. Nobel physicist Robert Laughlin even suggests: “… all real theoretical physicists are anarchists.” (Laughlin, 2005, p179), able to bravely go intellectually where others dare not.
In this spirit of innovation, physicist Paul Ginsparg created the Los Alamos bulletin board, the first truly successful electronic journal of science. Every morning theoretical physicists download new papers in their field and discuss them over morning coffee. Laughlin asked Ginsparg why this academic advance had not spread to other fields: “Paul suggested that physicists are self-selected to value eccentricity and novelty of ideas above all else, even at considerable professional risk to themselves. ” (Laughlin, 2005, p179).
While in Physics academics may value (and publish) well put eccentric thought, in IS academia seems to be going the opposite way. Rather than welcoming anarchist innovators, we seem to be driving them out into IS practice, to make breakthroughs there. Eric Raymond, a software practitioner, summed up the value/rigor contrast by comparing a vibrant open bazaar with a static closed cathedral (Raymond, 1997), the former impure but growing, and the latter pure but not. It seems time to open the doors of our ivory towers to more people, more ideas and more exchanges, but how?
|Modeling & Tools: Information Systems Using the Knowledge Pyramid to Characterize Systems J. N. Martin, The Aerospace Corporation Modeling & Tools: Multiple Sectors||Main Categories of Information Technologies Systems Regarding Process Orientation and Knowledge Orientation|
|The move to "open systems" requires re-thinking interaction with knowledge representation systems||Knowledge is a power. That is truth. But the greatest power is the knowledge about knowledge, I e. how to learn|
|The 21st century belongs to the knowledge age, where acquisition, possession and application of knowledge are the most important resources||Computer Science, Knowledge & systems|
|Computer Science, Knowledge & Systems||Eglash, R. “Nanotechnology and Traditional Knowledge Systems.” In Donnie Maclurcan (ed)|
|Representing and using legal knowledge in integrated decision support systems: DataLex WorkStations||Avoiding Information Overload: Knowledge Management on the Internet|