Guide to Writing more Scientifically

As a researcher in machine learning and computer vision, now in pursue of my PhD, I have often had to turn a bunch of experiments into a paper. The struggle comes when all of the sudden, everyone expects that I just know how to write it. After all, I did all the experiments! I realized that learning how to write a paper for a conference, and more recently, my dissertation, is the result of an organic, albeit not entirely smooth process.

Here are a few useful aspects that everyone trying to write a scientific report has probably encountered (I know I did). These are not exhaustive and could use a few more examples; something I hope I can add in the near future.

Note that this post is largely based on an excellent book I found called "Helping Doctoral Students Write" by Barbara Kamler and Pat Thomson. I have included a few thoughts I've gathered as a supervisor of several master and bachelor theses, as well as those that I've learned the hard way, submitting to conferences and writing articles myself. As the book mostly focuses on social sciences, I have tried to come up with alternative examples that are related to my work in machine learning. I strongly recommend that you have a look at the book if you find the post useful.

Setting up your mindset for writing:

Contrary to what many could think at the start, writing is not the one thing "left to do" after the real research has finished. Writing is vital because it allows you to engage with your community, engage in discussion and tell your scientific story. Writing is the vehicle to become "an expert" on something. Writing helps organize the ideas and arguments you'll later have to defend in a conference or at a colloquium. At the end of the day, the experiments you conducted or the results you got, will have to fit in a broader context. Writing lets you define that context, highlighting the importance of the results beyond the bare numbers, beyond beating SOTA.

Related work (Literature Review):

The goal of the related work is:

• Identify the different fields that are relevant to the question you're looking into.
• Sketch out the historical development of precursors of your idea.
• Identify current debates wrt. the idea you're developing.
• Point out gaps in the current understanding of the field you're contributing to.
• Localize your contribution within the gaps, the debate, the history, etc.

Make sure you don't write only "who-said-what-about-what" statements. A long list of 'related work' without an evaluative statement leaves the reader wondering about the actual relevance of the work that is being cited, and which ones are more (or less) important. Not having a stance wrt. the related work gives the impression that the author lacks authority or that they are overwhelmed by the contributions of others.

A metaphor that helps when writing the related work is imagining "sitting for dinner with the authors in the related work": first, identify who is sitting at the table with you. Think of your scientific contribution as part of a whole, and the authors sitting next to you are those who have contributed the stuff you are relying on. Imagine that the dinner party is organized by you, so you are in full control, and in a familiar environment. Obviously, you can't invite everyone, so just the most relevant ones are there. As a host, you let the other authors talk about their work but in relation to your own. At the same time, you are not only a passive listener but an active participant in that conversation.

Adopting a critical stance:

Critical doesn't mean "find what's wrong". It sometimes may feel intimidating to critique more senior and acknowledged scholars. Instead of attacking, think "what does this work contribute?"

You can critically assess individual texts of other scholars by asking questions, such as:

• what is the main argument?
• what kind/aspect of X is spoken about in this article?
• from what position?
• using what evidence?
• what claims are made?
• how adequate are these (blank spots and blind spots)?

(Blank spots: What we know enough to question but not enough to answer. Blind spots: What we don’t know well enough to even ask about or care about.)

Incorporate evaluative comments regarding the adequacy (positive or negative) of the arguments in the literature ("There has been little attention to...", "There has been a lack of focus on...", "The over-reliance on X is...").

Referring to myself when writing? (Using "I" or "we")

There are four factors to consider when deciding whether to refer to you (or your team) while writing your paper/thesis:

1. Standards within your community: what practices are being used in some (good) papers you've read? Note that the style may vary for different communities. For example, papers in neuroscience may present the related work in a very different style compared to those published in machine learning venues. Figure out what community are you targeting and prefer the style you see more often. Keep an eye for papers that have won "best paper awards" at highly ranked conferences, as they are also often praised for being well-written.
   
   
2. Agency without personal recounts: when the use of "I" is accepted, be careful not to use it to express personal opinions, without relating it to other works or to the community. That way, you avoid sounding naive by expressing individual experiences. In science, personal stances (opinions, beliefs) are mostly absent anyway.
   
   
3. Personal vs. discursive "I": use "I" as a differentiator for contributions you are doing (in contrast to contributions made by others). For example, when comparing your results to a baseline: "Our proposed approach requires half the training time compared to ResNet50."
   
   
4. Keeping arguments in focus: as an alternative, try omitting the discursive "I" by using the object of the sentence (i.e., the thing "I" is referring to) as the subject. For example, instead of saying "We conduct all experiments using VGG as the main feature extractor", you can rephrase by making the "experiments" the subject of the sentence: "All conducted experiments use VGG as the feature extractor". Note that passive voice (a common way of eliminating a personal or discursive "I") was not necessary.

A functional toolkit for language:

• Be aware of the differences between written and spoken discourse: a prominent property of written language is that most of it happens as 'nouns' (or nominalized). In contrast, speech tends to use verbs as the center of ideas. Moreover, speech tends to be more context-dependent. We can often say "compare it" or "that complies" because the context of the conversation provides enough cues as to what "it" or "that" refers to. This is less so with written text. If your writing is sounding somehow "immature" it can be probably traced to the use of patterns that are usually found in speech, rather than on written text.
  
  
• Nominalization: is the process of changing verbs into nouns. It is very useful for written text as it allows more information to be packed into less text. For example, the two following phrases: "We compare the previous experiment with three baselines yielding the following results" and "A comparison with three baselines yields the following results" convey the same information but the second compresses more information through the nominalization of the word "compare" into "comparison". Nominalization helps in making ideas more concise, especially when conveying abstract ideas, rather than actions or people. Be careful not to overuse this tool, as it will quickly lead to phrases that are too dense and hard to understand (for the previous example: "A results comparison with baseline experiments yields:"). A useful dynamic to find the right balance (especially if you think your text sounds too naive and not "scholarly" enough) is to start with a small segment of text and identify verbs. Use those verbs to break down the text into clauses (a group of words containing a subject and predicate and functioning as a member of a complex sentence). For each clause, try to nominalize the verbs you have e.g., compare > comparison; add > addition; revise > revision. Do this several times as some clauses will collapse into fewer nominalized phrases. Now, in case you realize that your text is too dense, try the opposite: take all nominalizations on your text and expand them with verbs and actors, making clear who made what and when. While finding the right balance, it is important that you're aware of the side effects of nominalization. By removing verbs, agency is reduced i.e., information about who or when is made implicit. In the following two sentences "When the discriminator stops generating gradients, the generator collapses to a single representation" and "An absence of gradients causes a collapse to a single representation" the latter doesn't need to mention what parts of the model are affected by whom. Choosing which sentence is more appropriate will depend on the stage of the text (e.g., is it in the introduction or the methodology?) or the need for granularity (e.g., is it important to differentiate "discriminator" from "generator" or is it enough to talk about GANs in general to convey the message).
  
  
• Passive vs active voice: they're both important and even required when writing. As with nominalizations, passive voice can compress meaning and, in doing so, make some actors and verbs implicit. Avoid combinations of both (nominalization and passive voice) as you run the risk of making your text increasingly dull and vague. Use the same recommendations for nominalization and also recall the strategy for using (or avoiding the overuse of) the discursive "I": play with direct and indirect objects to construct active phrases. For example the phrase "We show that our model can solve the task successfully" could be easily turned into a passive version as "The task is successfully solved by the model", which gets rid of the personal pronoun. Alternatively, you can also rephrase it as "The model successfully solves the task".
  
  
• The Goldilocks dilemma of assertiveness: when writing about your findings, you may be inclined to be cautious about the interpretation of your results. Does your model work better than everything else out there or just for THE dataset and only UNDER the conditions of your setup? When not entirely sure, you can be tempted to resort to modality i.e., modifiers of likelihood or frequency (e.g., might, should, perhaps, possibly). On the other side of the spectrum, modality can convey a high degree of certainty (e.g., must, definitely, always). Ideally, your text has to reflect enough assertiveness to convince the reader about the value of your contributions, but be tentative enough to state exactly what your evidence supports, and not more. For scientific contributions, you must have at least one strong contribution that can be asserted without the need for timid modifiers. Beyond that statement, aspects that are not fully supported by evidence (e.g., arguing why some experiments perform worse than expected) or that are perhaps part of the future work, should also come with the corresponding modal modifiers that establish your level of (evidence-based) certainty about them.
  
  
• Coherent writing: writing allows you to choose the focus of your argument. What are the most relevant aspects of your argument, and which are just tangentially related? This decision is conveyed through the way your text is constructed. A text that appears somewhat incoherent is often due to how the flow of ideas unfolds. To compensate for this, you can resort to something called "Theme analysis". In this context, "Theme" just refers to the "starting point of the sentence" and it defines what the clause is going to be about. Ideally, a sentence starts with the Theme, followed by some complementary information (also called "the Rheme"). This pattern for thinking in terms of "Themes" and "Rhemes" extends to the paragraph, and to the "sections" level, and it establishes expectations about how the text will unfold. By separating your text into Theme and Rhemes (per clause), you can check if there is something that is either too repetitive (e.g., starting all sentences of the related work with "Some studies", "There are studies", "Other work") or all over the place. The magic word here is "cohesion". There are two main strategies to make your text more cohesive using the idea of Themes:
  
  
• Repetition: using a variation of the same idea (with different words). For example, in the related work, building a strong motivation for your research can be to highlight "the lack of consensus", "the little available research", "not enough attention". Note that these clauses look quite similar to those of the last example that cautions about overusing the same Theme. This is an exercise of balance: an argument can be made, that the first example is relying on the same structure and the same language, while the one in favor of repetition is more coherent because the ideas are the ones being repeated, but the language varies slightly. The main takeaway is that repetition, when used in right amounts, can enhance cohesion; too much of it makes passages seem boring, disconnected, and going nowhere.
  
  
• Zig-Zag: expands on newly introduced information as the text progresses. In terms of Themes, zig-zag takes the Rheme of the previous sentence and turns it into the Theme of the next sentence. This gives the text a sense of cumulative development. Take the following two sentences: "This approach introduces a regularization term for the loss function. The resulting cost used for training depends on two factors that keep the model from overfitting". While the first one talks about the approach and the second, about the loss, both are thematically connected. The last sentence is expanding on information provided in the Rheme of the first one.

Structuring the thesis (it's not always IMRAD):

The default structure is known as IMRAD (Introduction, Methods, Report, and Discussion). The main benefit of sticking to this structure is two-fold. For once, it is well known and therefore, the reader knows what to expect. Additionally, IMRAD provides a sound structure for your argument to follow: why is the research important (Introduction), how is it done (Methods), what are the results (Results) and what happens in light of these results (Discussion). The IMRAD structure is pretty well-suited for "simple" ideas (i.e., when the line of argumentation is straightforward; the elements of the argument follow effortlessly one after the other). This is usually the case when writing conference papers. However, for ideas with more complex structures, when you're drawing conclusions from multiple fields or making comparisons across several problems, you probably need to deviate from the strict boundaries of IMRAD. Starting with a joint introduction and related work can still work, but then, the text can spawn into the details of each individual study before eventually finishing with an overall section with conclusions.

Sometimes, when the literature for each topic is well separated (e.g., writing about VQA and NAS. Sure, there are papers mixing both, but the main argument of the thesis you're writing does not revolve around that particular intersection), it makes sense to divide them up throughout your thesis. Think of the reader in such cases: they will first have to chug all references to the literature for VQA followed by the literature for NAS. Only then they will start reading about the contributions of VQA. This kind of text, even though it is correctly structured according to IMRAD, can often feel clunky and tedious to read. More importantly, there is no flow in the argument that should be present throughout the entire thesis.

A big issue with IMRAD is that the reader is not given an argument to follow. Instead, there are monolithic clumps of information that the reader has to patch together in their head in order to make sense of the big picture. For large texts, these gaps in the argument can impatient the reader rather quickly. Ideally, your text should provide an argument to follow along the entire time.

If you think you need to break out from IMRAD, you can think of your thesis (or paper) in terms of blocks that adhere to one of the following writing "types": "recount", "summary", and "argument".

• Recount: talking about what happened or how someone (you or others) did it. This type occurs mostly in the introduction and while describing the methods. This is particularly useful to show how the research was constructed (We start with a pre-trained model and attach a linear layer with 10 output neurons.), or where do the ideas come from (As with the baseline model, we train using an L1 regularization that depends on the current training epoch.). This way, a reader can follow what the train of thought of the researcher (you) is. However, be careful not to add too much detail that ends up wasting too much space and time. Some of the finer details can easily go in the appendix if necessary.
  
  
• Summary: brief account of a larger body of information. A good summary should show a knowledgeable reader that you understand the context and scope of your ideas. A summary, contrary to what automatic tools in Microsoft Word make you think, involves doing new work: you'll have to decide what are the most important ideas, what can be excluded, and what's the minimum amount of information that already provides 'proof' about your findings? Typical places where you need to write summaries are the abstract, introduction, conclusions, and, for long sections, at the beginning and end of said sections (e.g., a chapter on your dissertation).
  
  
• Argument: when you take a justified stance with respect to a particular issue. The goal is to persuade the reader of that which you are arguing and it should be backed up by evidence. The simplest form of an argument consists of three parts:
  
  
1. Your statement.
   
   
2. Points that support the statement, examples, and connections to other arguments.
   
   
3. A summary to reaffirm the original statement followed by possible recommendations.

Now, how can we re-arrange IMRAD in terms of recounts, summaries, and arguments? An Introduction is made of a factual recount (context) and an argument (motivation). The Related Work is essentially a summary and possibly an argument (highlighting the gap that your work is contributing to). The Methodology and Results are largely a series of summaries and factual recounts. The Results section may include small pieces of an argument, leading into the Discussion, which is itself an argument. The Conclusion consists of a summary (what was presented in the work) and an argument (impact, future work, etc). Be careful not to write too many factual recounts, as the text may end up feeling tedious and purposeless (think of a lengthy diary of a researcher). Instead, try including some "arguments" along the way to add purpose as the reader progresses.

Remember, use an argument as the main reference to organize your text. The essence of a paper or thesis is not to recount or summarize, but to argue. Recounts tell what you did but you also need to argue why that stuff actually matters. All in all, the goal of your paper/thesis is to contribute to the advancement of a scientific field!

Next: strategies to build the structure of your dissertation (coming up soon).