It’s ok to fail

In his book A Short History of Nearly Everything, Bill Bryson tells the harrowing story of Guillaume le Gentil:

Le Gentil set off from France a year ahead of time to observe the transit [of Venus] from India, but various setbacks left him still at sea on the day of the transit – just about the worst place to be since steady measurements were impossible on pitching ships.

Undaunted, le Gentil continued on to India to await the next transit in 1769. With eight years to prepare, he erected a first-rate viewing station, tested and retested his instruments, and had everything in a state of perfect readiness. On the morning of the second transit, June 4, 1769, he awoke to a fine day, but just as Venus began its pass, a cloud slid in front of the Sun and remained there for almost exactly the duration of the transit: three hours, fourteen minutes, and seven seconds.

Stoically, le Gentil packed up his instruments and set off for the nearest port, but en route he contracted dysentery and was laid up for nearly a year. Still weakened, he finally made it onto a ship. It was nearly wrecked in a hurricane off the African coast. When at last he reached home, eleven and a half years after setting off, and having achieved nothing, he discovered that his relatives had had him declared dead in his absence and had enthusiastically plundered his estate.

This is what we’ve chosen as researchers: a life of failure and rejection. In all seriousness, though, research is a process of trial and error almost by definition. It is hard. We do not always succeed. And that’s ok.

If our hypotheses were always spot on, if our procedures always worked exactly as expected – if life was really that predictable – there wouldn’t be much point in conducting research at all. Thankfully for those of us who love research, there are still plenty of things that we don’t know and don’t understand that require investigation. That said, the arduous process of developing new knowledge is replete with surprises and setbacks.

Not knowing any more about le Gentil or his story than what’s written above, I would still question the assertion that he “achieved nothing.” He may not have achieved what he set out to, but that should not by default mean that the entire adventure was without merit. I suspect that the experience of spending eight years in a foreign culture mastering his instruments must have had some unanticipated (and perhaps undocumented) benefits. In my own case, arriving in the field only to find my methods unsuitable was a fortuitous fork in the road. A nightmare at the time, this wholly unexpected scenario presented an opportunity to change tack and experiment with visual methods. Five years later, visual methodology is at the core of my research agenda. It hasn’t been an easy journey, but it has certainly been an interesting one.

In some ways, I have also been incredibly lucky. Although my PhD fieldwork did not go at all according to plan, my essentially made-up method worked well enough that I was able to return home with sufficient data to successfully complete my thesis on schedule. Not everyone is so lucky. And I’m not so lucky all of the time. Sometimes despite doing everything right, our research still goes awry. A cloud passes in front of the sun. What then?

I don’t know when or how it started, but a culture has developed in academia that rewards ‘success!!’ at the expense of knowledge and understanding. We are under enormous pressure to get it right. Some, though not all, of this pressure comes from the need to publish (‘as much as possible!!’). Journals accept papers that present significant (i.e. positive) findings. Professor Keith Laws recently observed that:

This publication bias* is pervasive and systemic, afflicting researchers, reviewers and editors – all of whom seem symbiotically wed to journals pursuing greater impact from ever more glamorous or curious findings.

He goes on to say that the solution is not the creation of special journals that publish negative or null findings. (An idea I’ve personally heard discussed on more than one occasion.) Instead, Laws argues that we need to make room for these “unloved” findings in mainstream journals. True, this depends in part on the cooperation of reviewers and editors. It also depends on us; we supply the content.

About a year ago, I submitted a manuscript to a top methodology journal. The article details three attempts a photographic data collection, two of which were only moderately successful. The third attempt was undertaken in conditions that were far from ideal and was largely unsuccessful as a result. One reviewer picked up on this, questioning why I chose to proceed with the research. I responded truthfully that that’s the nature of my work. The article is now in press.

In a previous post, Monica voiced concern that university metrics encourage the mass-production of ‘plywood’ rather than oak- or mahogany-quality research. The expectation that our research will churn out positive results (within a 2-3 year timeframe) compounds the problem and changes the very nature of the endeavor. Sometimes your procedure won’t go to plan. Sometimes your results won’t be what you expected. Sometimes a cloud passes in front of the sun at exactly the wrong moment. That’s the harsh reality of research. And, it’s ok.

(And if we’re bold, we can even get it published: warts, failings and all.)

*I don’t think that psychology is so different from other social science disciplines.

Rethinking Research Methods

It seems to me that research methods are the ‘ugly duckling’ of research practice. Just think about it for a second:

• Research methods courses (in my experience anyway) are generally a dull prerequisite that must be endured before you’re allowed to get on with the ‘real’ research. This is probably attributable, at least in part, to the fact that the people who teach research methods courses seem only marginally more inclined to be there than the students who are required to attend.
• Many (but not all!) books on research methods are similarly dry and prescriptive. Once again, I suspect that a paycheck may have something to do with the author’s motivation.

The same principle seems to hold true, in my admittedly limited exposure, to PhD methodology chapters. Explaining what you did and why you did it is a tedious formality. The theory and data analysis chapters are where you have a chance to shine and, consequently, are where you invest the most effort.

But it really doesn’t have to be this way. If you stick it out long enough – and perhaps do a bit of reading beyond the usual textbooks – research methods might just surprise you.

Much like most of you, I imagine, I muddled grudging through research methods training, the accompanying reading, and eventually the first several drafts of my PhD chapter. To my surprise, however, a serendipitous sequence of events led to a veritable wonderland of visual research techniques, my methodology chapter became the standout feature of my thesis, and researching and developing innovative research methods is now at the core of my postdoc.

So, to those of you trying to decide which research methods to use, I suggest diverting, if only briefly, from the well-trodden path to explore what else might be possible. You can always go back to surveys or interviews, so why not indulge in a little procrastination and take a look at what other options might be available? One way of doing this could be to browse around the SAGE Methodspace (http://www.methodspace.com/) to see what sorts of things other researchers in your area are getting up to.

To those of you trying to write up your methods chapter, I suggest thinking of it less as a technical document and more as a personal account. This is the one place in your thesis where you can tell your story, misadventures and all. Instead of giving your examiners yet another dry, dull, lifeless methods chapter (that they’ll be just as loath to read as you were to write), why not surprise them with the engaging tale of how the research project you ended up with wasn’t quite the one you had in mind when you started out. I may be wrong, but I suspect they’ll thank you for it.

Pressed for Time? How to survive your PhD in 1 page

 

There simply aren’t enough hours in the day.

Period.

And somehow blogging keeps slipping lower down the list of priorities. Still, we do have the very best of intentions of keeping this blog going – we still have plenty to say! – and with that in mind, I thought that now might be an appropriate time to share a little treasure that I stumbled across some months back.

A few years ago David Gauntlett scribbled down some notes in preparation for a workshop titled ‘Surviving your PhD’. Yes, believe it or not, he has really managed to fit everything you need to know (or nearly everything, at least) onto a single side of a single piece of paper. His reflections provide some quick tips that are easy enough to implement immediately  as well as some broader suggestions that may require more time to digest.

There are two points that I’ll just quickly pick up on, and I’ll let you discover the rest for yourselves:

1) “Don’t spend forever reading without writing”

It is incredibly easy for reading to become a welcome distraction, or even a procrastination technique. Following up an interesting citation in the journal article you read yesterday soon leads to the discovery of an expert you’ve never heard of before, two weeks later you still haven’t written anything and the deadline for finishing your next chapter was three days ago. It happens.

What’s more, sometimes these serendipitous forays into new bodies of literature prove incredibly fruitful. The point is that they shouldn’t keep you from writing because, in my experience at least, the longer the writing hiatus, the harder it is to get going again. Which leads on to the next point:

2) “Publish stuff”

Easier said than done, I know, but you won’t regret it. Let’s be honest: in this economic climate, jobs – academic or otherwise – are hard to come by. Having a couple of publications in hand won’t guarantee you your first job, but they will make you a much stronger candidate. Not only does publishing show that your research and writing skills are of a high standard, it shows that you’re taking initiative to actively engage in your chosen field.

The peer-review process that accompanies publishing can also provide a useful source of critical feedback. The reviewers will likely pick up on weaknesses in your argument that you (and perhaps even your supervisor) weren’t aware of. Additionally, replying to reviewers gives you a chance to practice responding to criticism – an essential skill for surviving your viva.

And with that, it’s time that I got back to writing my long delayed (by reading) journal article.

 So, without further delay, here’s the promised link:

 http://theory.org.uk/david/phdtips.htm 

 

To Our Loyal Readers

It has recently come to our attention that the spam filter may have been blocking legitimate comments.

Please accept our sincerest apologies if you have tried to comment on one of our posts and it never appeared. Now that we are aware of this problem, we will monitor incoming comments much more closely.

We hope that 100thousandwords can become a forum for discussion and welcome your thoughts (questions, too!) as we continue to develop this blog.

Beware of Moving Goalposts

For most people, minor corrections are relatively straightforward. If you’re lucky, your examiners will give you a list of the specific things they want changed and the pages you’ll find them on. This list won’t contain any surprises, as you will have thoroughly discussed its content already during the viva.

Minor corrections should be like this. That doesn’t mean they will be.

It is impossible to relay to you the frustration, anger, and disappointment I have experienced in the past week and a half.

What I can do, though, is warn you about a worst-case scenario. Hopefully you won’t have a similar experience to the one I’ve had. (Most people don’t.) But if you do, at least you’ll be better prepared than I was.

Although my corrections seemed relatively straightforward the first time I read through them, as I began to implement them – and then began to ask questions about them – the plot unravelled with astonishing speed.

Example 1: I was asked to provide information in the Preface to my analysis chapters about a minor event that took place two years after my data was corrected and a month after my thesis had been submitted. During my viva we did discuss the possible implications of this for future electoral process in the country in question, but I was never under the impression that my examiners saw this particular incident as integral to the thesis, my argument, or the case study itself. It was a surprise to find in my list of corrections.

• Q&A – I pointed out to my internal examiner that the event, though interesting anecdotally, had occurred after my thesis had been submitted and therefore had in no way influenced my data or analysis. Question: Do I still need to address it? Answer: Yes.
• Moving Goalpost – Accountability for events occurring after submission.

Example 2: I was asked to provide more background detail about the country and region where I conducted fieldwork, including demographic, ethnic, religious, and colonial heritage. This seems a fair enough request. We had discussed the need for more general background information in the viva so this particular correction (at first anyway) was not a surprise either.

When I began to work on this particular correction, however, I soon realised that I had already written over a page about ethnicity that had apparently gone unnoticed. I had also stipulated in the original thesis that I was not going to go into the thorny issue of the colonial legacy because its relevance is hotly contested in the academic literature and none of my research participants ever brought it up. It seemed, therefore, to add immense complexity to the political back-story without adding any clarity.

• Q&A – I told my internal examiner that I agreed that more background was needed, but wondered whether a discussion of the colonial period was justified, as it didn’t directly impact on my research. Question: Do I need to discuss the colonial period even though it was not a relevant issue raised by my research participants? Answer: You need to better argue your position. At the minute you just say that you are not going to discuss it. Question: Really? I didn’t realise that is how it came across. Ok. So I can just substantiate the argument that it isn’t particularly relevant to this research? Answer: It is relevant to this research because the people speak French and language is part of culture.
• Moving Goalposts – Expectation of specific corrections (i.e. a discussion of language) from vague instruction (i.e. more background on the colonial period)* and redefinition of the scope of the thesis. (In developing the political culture concept used in my thesis, I specifically argue that language and linguistics are beyond its remit.)

Example 3: I was asked to make three changes in terminology. Two of the three we discussed in the viva and I was happy to accommodate for grammatical and technical reasons. The third change in terminology concerned an analytical label I had developed. We never discussed this in the viva and on the short, informal list of typos handed to me by the external examiner this particular term just had a question mark by it. It was, therefore, a surprise to find it on the list of required corrections.

Had we discussed it in the viva, I could have pointed out that the term in question had already been deliberated over and accepted at a number of conferences. It had also been deemed appropriate by two anonymous referees for a top journal in my field and was due to appear in a forthcoming article that had completed the peer-review process.

• Q&A – Question: Does this term still need to be changed despite a) having survived a vigorous peer-review process and b) never having been discussed in the viva? Answer: According to my supervisor, no. According to the internal examiner: tbc.
• Moving Goalpost – Requirement of a change that was not discussed during the viva.

 

Can examiners hold you accountable for events that occur after submission? Can they require specific changes that are not explicitly stated in the formal list of corrections? Can they demand changes that were never discussed at the viva?

I am still trying to find the answers to these questions. Apparently the University has published guidance notes on minor corrections, but so far I have been unable to gain access to them. With only 15 days remaining before the final submission deadline for January graduation, this is an incredibly stressful and frustrating struggle. And one that I can’t help but feel could have been at least partially avoided had I tracked down this guidance earlier.

As things stand, I don’t know where the line is drawn between legitimate corrections and unreasonable demands. I would highly recommend that you track down your University’s formal guidance on minor corrections before your viva. Hopefully you won’t ever need to look at it. But at least it will be there in case you do.

*The formal list of minor corrections does not provide clear instruction regarding the specific changes required by the examiners. I was likewise told that “some statements in the final chapter are too bold” and they need toning down. Exactly which statements this correction refers to are never pointed out; nor for that matter is which chapter they are in (i.e. Chapter Seven or the Conclusion).