ARCHIVE: Publish and Perish: An open letter regarding academic peer reviews and evaluations


In connection with our feature length documentary Do Not Bend: The Photographic Life of Bill Jay the Bill Jay Estate has given us permission to curate a series of his articles from the over 400 he wrote to post on our site as they were originally written. We hope that you enjoy reading them and that despite when they were written that you find them still relevant to the photographic discussion of 2018.

[ I was hesitant to include this item on my web site because it is personal, of arcane interest and it does not deal directly with photography. But a recent conversation with a teacher, at a different institution, convinces me that my problem was, and is, more common than I had imagined. With the faintest of hope that my experience will be of some comfort to him and to others I offer it for what it is worth. A few introductory remarks will suffice. At the time I was a Professor in the School of Art at Arizona State University. I taught the History of Photography and was a member of the Art History faculty. This open letter was addressed to Leonard Lehrer, Director of the School, whose commitment and vision had given the School, especially its photography and printmaking programs, national prominence. After he left, these programs withered. Yes, one person can make all the difference. The letter was distributed to all my colleagues. I think the rest is self-explanatory…]
Dear Leonard,

In order to address the broader issues of academic publishing and peer evaluations, I must first “set the scene” by focusing on a specific case. That means I must, regretfully, become personal, because the case with which I am most familiar is my own. But I will try to be as succinct as possible not only because you are familiar with the facts but also because I have an embarrassed reluctance to dwell on the notion that I have been forced (I feel) to play the role of onlooker in the academic game.

The facts are these:

When I became eligible to apply for tenure at the University, and when I was later eligible for promotion, on both occasions I did not receive a single vote of support from the dozen faculty members in my area of art history. Yet the primary criterion for both tenure and promotion, according to the faculty guide, is the candidate’s publication record. Even a cursory glance at the records would indicate that my publication record was several times greater than that of all my colleagues’ publication lists combined. Ah, well. As one great philosopher remarked: there’s something fishy going on here.

The fishiness began to wriggle and become more tangible and real after my last annual review (May 1988) by a committee comprising individual faculty members drawn from each area of the School of Art. I received an abysmal rating by my colleagues, down near the bottom of the faculty standings. Were there really so many faculty members more productive than myself? If so, I not only accept the poor rating but can honestly say I would be proud to associate with such hardworking colleagues. Anyway, the idea was intriguing enough to warrant a little digging into the files.

It is true that this was a lean period for me. I was on sabbatical leave for eight months of the year, most of the time isolated in a remote cabin, without telephone, mail or any other means of contacting, or being contacted by, the field. Still, for this year I was able to list: four published articles (with five additional articles accepted for publication); two photographs acquired by the permanent collection of a major museum; five guest lectures at various colleges and institutions; two radio interviews; judge of a national fellowship in photography; guest curator (with a prominent museum director) of a major exhibition; completion of a book manuscript (accepted for publication); a week- long workshop at a college renowned for its teaching program; and several other ancillary activities. I also wrote 20 articles on a wide range of topics, from primary 19th century research to critical essays about current issues in the medium.

Fair enough – I admit it – it was not one of my most productive years. I was curious, therefore, to see the annual reports submitted by my colleagues, the faculty members who gave me such a disastrous evaluation, all of whom rated each other far more highly than they rated me. Perhaps I would get an inkling of the exceptional standards which I could not meet.

The evaluation committee comprised eight faculty members, not one of whom was in my field. However four of them were in “scholarly” areas in which publication, as opposed to exhibition of creative work, is the criterion of merit. It is reasonable that these four would set the standards and underline my meager offerings and I was therefore most interested in their “qualifications” to make assessments on my work. Unfortunately, although my report was open to them, their reports were not open to me.

But as far as I can ascertain, these committee members produced three or maybe four articles (apart from local newspaper reviews) and about three outside lectures between all four of them – and that three of the articles and two of the lectures were by one individual. Therefore, three of the committee members produced one article and one lecture in total. Because of the confidentiality of their records I may have misrepresented the facts – slightly. (Indeed, I may have been generous). But the fact remains that the list is hardly an indication of high productivity or stature in their areas. (Just for the record, the artists on the committee did not demonstrate amazing creativity and listed only a few group exhibitions and a couple of lectures between them). Again, as I think you will agree, the reek of fishiness is in the air.

Fortunately, faculty members are, or become, incessant gossips. So, in coffee shops and corridors, I could quickly ascertain the inside scoop on what transpired behind closed doors. It turns out that there were two main factors which led to my evaluation as one of the leading klutzes of the department.

First, lecturing to other colleges and universities, curating of exhibitions, producing images which are requested by museum collections, and so on, are all subsumed under the minor category of “service to the field” by University-wide as well as by School of Art policy and therefore carry little professional weight. It matters not a jot that they are evidence of productivity and that they indicate a faculty member’s standing in the field. They are merely evidence of career “wheel-spinning.”

Second, and more relevant to what comes later, is that my articles were deemed “insubstantial,” “lacking in seriousness,” “trivial,” and “light weight.” Also and most significantly, they were not published in approved refereed journals. I was called a “mere journalist” and not a “true scholar.” It was suggested by one committee member (who was sincerely trying to be helpful but sounded condescending and patronizing) that I should “try something more serious and substantial, like a book, because we do think you are capable of it…”

Anyway, the upshot is that all my articles were summarily dismissed. Fair enough. I can see their point, although I would lend a little more credence to their opinions if they practiced what they preached. And a glaring inconsistency is in the recommendation that I attempt a book. I have authored 10 books to date, with several more in production. Six of these books comprise original research on 19th century individuals; five of which remain the standard texts for biographical information. The committee sitting in judgment on my endeavors has produced one (15 year old) book between the lot of them (although one member has produced several handsome exhibition catalogues).

But there are other inconsistencies (notice that I am refraining from calling them hypocrisies). I am a reader (referee) for one of the best academic/scholarly journals in the area of Victorian history, for which I also write occasionally.

Also one of my articles deemed “inconsequential” has already been republished three times in the past six months. An editor wrote that it was “the best article ever published” in the magazine and I have received several complimentary letters from prominent individuals in the field, who are intending to make it required reading in their classes.

If I needed outside validation for my efforts, whom should I believe? Colleagues of minimal accomplishment outside my field, or professionals of stature within my area of expertise?

Another article which was earlier considered “insubstantial and unscholarly” was reprinted in a textbook for writers which included essays by “major scholars and thinkers in their field.” The editors stated that “we have confined ourselves only to material that is genuinely disciplined – that is well informed, well developed, and well written.”

Obviously my article cannot be “shallow” and “disciplined” simultaneously. I take heart that it was included with essays by E. B. White, James Agee, Margaret Mead, W.H. Auden, Virginia Woolf, John Berger, Bertram Russell, Joan Didion, Roland Barthes, et al. Whose opinion of the article am I most likely to respect? The editors (all academics) of a book on the best examples of writing across the disciplines, or colleagues who have demonstrated meager talents and productivity in their own writings? The delicious irony of this situation is that the article first appeared in an art history journal published by the School of Art and for which I received no academic “credit” from my colleagues.

It is all very curious, if not surprising. But the implications are quite devastating. On my office desk are five letters from different editors in Britain and the USA requesting (in one case, begging) me to write for their journals. It is a bit disconcerting to know that complying with the requests will not only confirm my colleague’s assessment of the “unscholarly” nature of my work but also will actively drop my standing in their evaluations because I have ignored their advice. The conclusion is this: I would gain a better approval rating if I did not publish at all. The ideal, of course, would be to melt into the academic pot of pseudo-scholars by writing one article, heavily footnoted, no more frequently than at three-year intervals, and have it published (as long as it is so obtuse as to be unintelligible) in an academic journal read by 40 people.

These remarks are not meant to be facetious but an objective assessment of a bizarre situation.

To the relief of both of us, that is the end of my personal case history. I will make only brief references to these points from now on, I promise. But this minor saga of one man’s sojourn in scholar-land was necessary to relate in order to move on to more general themes.

Musing on my lowly status in academia (according to my peers) has led to a few observations about academia and the notion of what constitutes “value” to a field. There is certainly a pressure in academia to publish (or exhibit, in the case of a creative artist). And rightly so. As far as I can determine there are four important reasons why the publish-or-perish syndrome is worth preserving and, I would assert, further emphasizing. In order of merit:

1. Publishing demonstrates productivity. This is important because continually active minds contribute to better teaching in the classroom. Only inquiring, curious, shifting, acquisitive intellects can encourage an enthusiasm for a questioning alertness in young adults. Teachers must “profess” the value of learning and demonstrate it in enriched lives. The benefit of research/ publication is to the student.

2. Publishing disseminates information/ ideas to the community of professionals. Articles in print are the most efficient vehicles (as yet, although this will change) for the sharing of discoveries and the airing of issues. The benefit is to the field.

3. Publishing promotes the institution. Whenever an individual is linked to a particular university, credit occurs to the institution as well as to the individual. This is important not only because a patron should be publicly thanked but also because enhanced prestige attracts a higher caliber of student who, in turn, will bring credit to the institution, which in turn… and so on. The benefit is to the university.

4. Publishing justifies education. Public demonstrations of the value of knowledge and literacy bring credit to the academic life in general. The benefit is to the cultural life of the age.

I would like you to pause at this point and reexamine these four assertions. Please do not read on if they are not acceptable or if I have grossly misrepresented the purposes of academic publishing.

(Have a cup of tea)

Now I can move on with the presumption that, with minor adjustments, there is general agreement on the value of publishing and of publicly demonstrating a high level of productivity.

Let me draw some conclusions from our agreement.

Not one of these benefits of productivity/publication demands that the article appears in an approved, small-circulation academic journal, written for a handful of scholars in an esoteric area. Indeed, the opposite is true. The more widely circulated the periodical and the broader its constituency the more benefit to the author, student, institution and society. Certainly, the need and value of refereed journals is vastly overrated by the academic system.

There is some merit to the idea of referees for scientific literature (from where the idea was adopted in the arts), but that is another issue. Suffice to say that none of the legitimate reasons for introducing the referee system to scientific literature has any validity in the arts.

In the areas of art and art history refereed journals have had, in my opinion, totally negative effects: the isolation of the scholar from the culture; the promotion of a particular form of bad writing, the creation of narrower and narrower areas of specialization; the dulling of intellectual inquiry; and the insidious, mind-deadening use of pseudo-intellectual and unintelligible jargon. (An important book that addresses this theme is The Last Intellectuals, Russell Jacoby, Basic Books, Inc., N.Y., 1987). But that, as I said, is another issue.

Also, I have a personal problem here, so I had better declare it. There are no refereed journals in photography. Hence, most of my articles (all 300 plus of them) are not worthy of consideration when evaluated by my colleagues.

In case this seems as though I have a personal gripe against the system (which is true) I would like to add that I understand, if not condone, my colleagues’ position. Academics who tend to encourage committee activity tend to be the least productive members of the faculty. This is reasonable. The highly active individuals avoid time-consuming university politics as much as possible. But out of the committees come the definition of The Game and the rules for the successful playing of academic snakes and ladders. And here I come, cap in hand, saying, I want to play The Game but I will use my own rules. No wonder they cry “cheat” and throw me out. I can’t blame them.

What I am trying to accomplish in this letter is a rewriting of the rules. To my rules, of course, because I think they are better rules, apart from the fact that they will benefit me. I do not hold any hope that this goal will be achieved, but some things are worth doing if only to discharge crud from the author’s head.

So back to the nitty-gritty. What do I want, but not expect, to change? Well, if the above four principles are accepted, it must follow that…
…evidence of productivity has value in and of itself, even if publication does not result.

…publication in wider-circulation periodicals is more meritorious and useful than in narrow academic journals.
…articles should be judged (if judged at all by colleagues ignorant of the field) by their content not by the name of the publication. By any standards of logic it is absurd that a single article can be highly meritorious if published in one journal of dubious distinction yet completely worthless if published in a leading journal of the field.

…reading a paper/delivering a lecture at the invitation of another university has all the benefits to students, the field and the institution which we have agreed are the purposes of productivity. It is beyond my comprehension that another university can consider the individual to hold such stature in the field, that it is willing to pay honorarium, airfare, expenses to him/her for the edification of their own faculty and students but that the lecturer’s own faculty consider the invitation to be without merit.

I could continue to point out the silliness of it all. I am well aware that the fun and games are rampant throughout academia and are not isolated to the School of Art. But enough. This is becoming tedious for both of us I am sure. Suffice to say that the present system of evaluating peers is not only illogical it is downright destructive. It is ludicrous when, in order to play The Game, a faculty member must become less productive, limit visibility, and seek smaller audiences.

I intend to make this letter “open”, not because I feel my situation is unique (in the sense that I have been singled out for special abuse by my peers) but because I am not unique. All around me in academia I see committed, enthusiastic, productive colleagues who are being penalized by knee-jerk committee hacks and lackeys who are less productive than the individuals over whom they sit in judgment.

A final tip of my hat to you, Leonard. Without your acknowledgment of the situation, I would not have my job. You have reversed every evaluation of me by my peers, and I know this has “cost” you. You were the lone voice of support in my tenure and promotion applications and, by extension, saw more clearly than most that there should be flexible strategies within the fixed rules of academia if productivity was to be encouraged. Without you, I would be steamrollered by my colleagues at every turn.

Respectfully, therefore, I place this promise on record. Never, ever again will I submit myself, my work or my profession to the indignity of peer review. I will not allow second-rate, minimally accomplished colleagues of low visibility in their own areas (and totally nonexistent in my field) to sit in judgment on my activities, and pontificate on my professional life and accomplishments.

That is not only undignified (sure, I do have some pride) but insulting – and totally meaningless. (It is meaningless because no one is entitled to examine me unless he/she knows more about the subject than I do. And that is self-evidently true for everyone in every situation.)

Yes, I do understand the price-tag; especially in forfeiting any chance of merit pay which might be available.

So be it.
Yours sincerely,
Bill Jay

P.S. This piece is not research and will not be published in any approved journal and it is, I hope, perfectly intelligible and therefore has no university validity.


Header Image of Bill Jay: David Hurn/Magnum

The UNP co-produced documentary film, Do Not Bend: The Photographic Life of Bill Jay will be screened across the UK and the US in 2018.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: