How to design a scientific research project

Introduction:

How does a scientific project go from start to finish? As a new researcher, you're NOT expected to know this already, but it is useful knowledge when trying to contextualize your role in a larger project. On this page, I will outline how most scientific projects progress. Also, check out the specific wiki pages about many aspects that will be addressed here (i.e. "How to keep a lab notebook", etc.)

A note for more advanced scientists:

This is a very helpful blog post about advanced experimental design for ecologists by Mattias Rillig (2022). Also check out his posts about proper controls and pseudoreplication!

Parts of a Scientific Project:

1. The Idea:

The first part of any scientific project is to come up with an interesting question you want your project to answer. What is an "interesting question"? You want to choose a question that the answer to will advance the scientific communities' knowledge about something interesting. This might be something with real-world applications (such as an advance in technology or medicine) or a fundamental fact about the world that will help us interpret other data or observations. How do you find this out? A good first start is to review other scientific papers that have been written about a general topic you are interested (such as rainforest conservation, gut microbiome, or biotechnology) and see if there is a large and important question or research area that has not been addressed. This might make a good area or question to develop a project to address.

One fundamental question that I think is currently understudied is how individual genes or groups of genes influence interactions between species and the formations of communities and ecosystems in nature. I think this is important because with the rise of biotechnology, humans will be able to modify the genes of potentially all life forms. However, if we don't understand how genes shape our environment, we won't be able to understand how these technologies can fundamentally alter our planet (and ourselves!).

I thought that studying microbes (bacteria and yeast) that inhabit floral nectar might be a good way to study this "big question" about how genes influence species interactions, communities, and ecosystems. Nectar microbe communities are simple in 2 very important ways. First, the genomes of microbes are generally might simpler and therefore easier to study that more "complex" organisms such as animals or plants. Second, they are easier and faster to grow in the lab. Third, communities in nectar are much simpler than other ecological communities, which makes it easier to understand the contributions of each component. Fourth, understanding nectar microbes in and of themselves can have important real-world value because they might be important for the health of plants (directly by living in them, as well as indirectly by influencing pollination) and they likely interact with relevant pollinators such as birds and bees.

One big thing we don't know about nectar microbes, which might have important consequences for plant health and pollination, is the interactions (and implications of interactions) between yeast and bacteria, which frequently inhabit nectar, but rarely together in the same flower. One specific question might be, "What are the important interactions between yeast and bacteria that live in floral nectar? What is the genetic basis for these interactions? What implication do these interactions have for the individual plant and plant community?"

To figure this out, you need to have a good feel for the literature, or what has already been published on a given topic. You can do this by:

Reading papers on a topic, especially review papers
Talking to experts in the field (like graduate students, postdocs, and PIs in your lab or other labs)
Debating topics with your peers and getting feedback on your ideas
Attending scientific conferences or listening to talks
Reading textbooks written on the general topic

Check out this article about 100 fundamental research questions in Ecology to get a feel for what sort of things the field is interested in: https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/1365-2745.12025

2. Develop a specific question:

The next step is to develop a specific question related to your "big idea" or question. At this stage, the main considerations might include "In what system can I easily answer this question?" "What technology exists that can help me answer this question?" "Are there people around me that can help me answer this question by lending resources or expertise?" "Do others find this question interesting or worth pursuing?".

I think that an interesting question involving nectar microbes is how bacteria and yeast prevent competitors from growing in the same flower. As far as we know, nectar microbes can't control very strongly which flowers they get introduced to (typically on the tongues of bees or hummingbirds in California). In ecology, the idea that the order of species' arrival to a new habitat (such as a flower) influences the final state of a community is called "historical contingency." Historical contingency has been studied in the assembly of ecological communities for a long time and in many contexts, such as the order of new plants colonizing the early Hawaiian islands comparing to the current communities on the island, or which bacteria first colonize a vagina and the vaginal microbiome of an adult woman! This is a really fundamental question in ecology, but it's hard to understand the genetic factors that might be important for these interactions because, at least in the examples I just listed, the communities are COMPLEX! How about, instead of studying this in a very complex system with lots of players, such as an entire island or in a human vagina with lots of different microbes, we started in a simple system (such as a flower) with only a few microbes we can study in-depth? Seems much more reasonable! Fundamentally, how nectar microbes inhibit or support growth of subsequently arriving microbes is a way to study the impact of arrival order on the final diversity of such a community, and since we're working with microbes, it's possible to study these interactions on a chemical and molecular level. Plus, it's not just a proof-of-concept, understanding how microbes interact in flowers might uncover a new antibiotic or strategy humans to combat (or support) microbial growth. It might tell us how we can use nectar microbes to grow stronger plants or plants that are more productive!

When you're first getting started, a helpful starting point when trying to think of a question is to ask what has already been done in the lab and what outstanding questions there are that you can build on. For example, if you are currently working on a project with another student, you can ask how their question can be extended or advanced by adding another factor that hasn't previously been considered. Now, it can be very easy to just do the "next logical experiment" without asking the broader impact of that question. That is when you want to ask yourself, how will this advance the field or address a fundamental question?

An editor of a prominent ecology journal once told me that, when they review manuscripts for the journal, they asks themself:

Did this research address a novel and important question that will alter how most ecologists address a particular question/think about a particular topic?
Did this research combine two previously disparate ideas in an illuminating/novel way?
Did this research overturn or call into question a dogma or intuition in the field?
Did this research address a fundamental question in ecology that previously was largely untested?

Examples of research questions

What is the role of rapid evolution in the interactions between yeast and bacteria in a nectar microbe metacommunity?
Do nectar microbes differ in their rate of dispersal, and how does this affect metacommunity assembly?
How do nectar microbe communities differ in their species' composition and function between plant species?
How important is bird pollination in the assembly of nectar microbe communities?

Again, you should review the literature, especially by reading original research articles on a topic, to define your question and determine whether it is novel or important to pursue.

This is really hard! Where should I start?

Don’t worry if you are struggling with this! It’s easy for me to say, but everyone who has ever designed a research project can tell you that getting started is the hardest part. Luckily, you are surrounded by lots of great resources! If you are having trouble developing a research question, you can and should use all of the resources available to you, including:

Ask a postdoc or graduate student in your lab or who are working on a similar question what they think about a particular idea. You don’t need to come to them with a fully-formed question, but you should try to come to a meeting with some prior reading on the topic and have thought about what they can help you. And don’t expect them to give you a research question! Although scary, the exciting part of leading your own research project is having the ownership over your research question. If you don’t know who to ask, start with people you currently work with and ask who they think could give you some good advice. I cannot stress the importance of discussing and mulling over your ideas with colleagues!
Read a review paper on the topic and look at the “outstanding questions” or “future directions” they propose. Often, at the end of a review paper, the authors will pose several questions that they think are the future of the field. If one of those questions interests you, read other papers that are on that topic and see if you could address it in your research project.
Start with creative constraints. It can be overwhelming and intimidating to try to develop a research project from scratch. I’ve found that it’s often easier to take stock of your “creative constraints” and try to design the most interesting question given that. For example, in our lab, we have lots of different species of nectar microbes (including evolved ones) and different types of synthetic nectar. We can also easily quantify the density of bacteria vs. yeast and different types of yeast. Previously, the lab has worked on competing yeast and bacteria in the lab, as well as observing yeast and bacteria in the field. What is the most interesting question given what materials we currently have the in the lab and what future directions best build off of what the lab has previously done.
Think about pressing questions in another field, then try to apply that question to your field. This was advice coming from my amazing undergraduate advisor, Mark Hunter. Mark encouraged me to read widely in different fields and see what questions interested others and see how we could use this interdisciplinary perspective to advance questions in our own subfield. For example, I studied molecular microbiology and realized that many microbiologists were studying something called “polymicrobial communities”—pretty much, the molecular mechanisms of microbes interacting with each other. I realized that this fits nicely within the frameworks in community ecology (species interactions) and wondered how we could leverage the molecular work done by microbiologists studying “polymicrobial communities” to better understand the molecular mechanisms underpinning species interactions in an ecological sense.

3. Define a specific, testable hypothesis:

After you have a broad question, you'll need to narrow it down to a few specific, testable hypotheses.

Now that you have a general research question, you need to develop specific, testable hypotheses. This might be a cyclical process with the next step, designing an experiment. You want to write a specific hypothesis, then think of how you might go about testing it. You might find that the experiment you designed isn’t feasible or possible for some reason. Then, you’ll want to revisit your hypothesis and see whether you can revise it so you can address using the tools around you.

There are many different ways that people recommend you write hypotheses, so this is by no way the only or best way to do it, but here are a few of my suggestions:

Phrase it as an if, then statement. If X is true, then I expect Y.
Include a hypothesized direction of change
Be able to your hypothesis as a graph (what you expect the result to be)

Example:

If yeast that were serially transferred in bacteria-conditioned nectar evolved to be less affected by priority effects exerted by bacteria, then these evolved yeast should grow better than ancestral yeast when inoculated before bacteria.

Additional resources for developing project questions and hypotheses:

4. Designing the experiment:

While developing your hypothesis, you want to be concurrently (or in parallel) asking whether you can design a feasible experiment to actually test the hypothesis you wrote down. When designing the experiment, make sure you consider:

What treatment groups do I need to address the question?
What controls do I need? Positive controls? Negative controls?

Think about comparing the treatments and controls.

What combinations will yield an answer for your hypothesis? Which ones won’t?
What additional questions could you simultaneously address using your experimental design?

Another important aspect to consider is your statistical analysis.

What statistical tests will I use to see whether I am observing a real (read: statistically significant) effect? What are my sources of variation (technical variation, biological variation) and how much of each type of variation do I expect (may need to run a trial experiment or preliminary tests)?
How many replicates do I need to detect real differences, rather than results due to random chance?

Then, consider logistical constraints:

What materials do you need? How expensive are they? Do we already have them, or do we need to order them? How long do they take to arrive? Does that conflict with the timeframe the experiment needs completed?
How long will it take to conduct the experiment? What is my timeframe?
How many people do I need to conduct the experiment?
What expertise do I need to conduct the experiment? Do I or the lab already have it, or would I need to learn new things first?

Discuss your ideas with colleagues, collaborators, advisors, mentors, and your lab! Many minds are always better than one.

Additional resources for experimental design:

10 simple rules for best experimental design in ecology Components of design in ecological field experiments Designing an ecological study

5. Conducting the experiment:

By this point, you should have a plan about conducting the experiment. I’d highly suggest reading my post about keeping a lab notebook and organizing yourself before starting this phase of the project. I always write out a protocol for my experiment and then copy it into my lab notebook, where I add notes.

A few notes about conducting experiments:

Everything always takes longer than you think it will
If you can do the experiment multiple times, you can expect to throw out the first round’s worth of data. Tons of things will come up the first time that you never expected.
Always build in breaks for yourself – no one is a robot. Take care of yourself and don’t conduct experiments when you’re tired or hungry. You’ll make a mistake and waste more time than if you had taken a break.
Build in fail-safes for mistakes. You will make mistakes, but you can design experiments so that if you catch them, you can fix them or at least partially recover.
Take copious notes and write down everything that could be a source of variation in the experiment. This can save you later, if you notice something confusing or unexpected in your data.
Label everything and keep your labeling consistent. Your future self will thank you.
Be honest and never, ever falsify data! The entire scientific process depends on each person’s integrity as a scientist. Never compromise that.

Also, read my post about best data practices. Consider setting up your data entry sheet before collecting any data! This is a good exercise to check whether you have everything you need.

6. Analyzing the results:

Before sitting down for formal analysis, it’s always a good idea to get a sense of your data by preliminarily plotting it. This isn’t to address any of your hypotheses, but to get a feel for your data. How is it distributed? How much variance is in your data? Does your data have any trends in it or anything unusual that you didn’t realize about your experimental design that is influencing your results. You may notice an error in your data collection method or something else unusual by doing a quick exploration of the data.

After you have a sense of your data, then work on the analysis you planned to test your hypothesis. I always suggest plotting the data first (that’s also why it’s helpful to have your hypothesis written out as a plot) and then write your statistical test.

After you have addressed your main question, you may have other ideas of additional questions you can address with your data. Explore them! Is your data actually telling a different story that what you expected?

Often, your results will push you in a direction to address further questions. Experiments are rarely “once and done.” Frequently, your results will prompt other questions, which you will prompt follow-up experiments.

Here is an interesting note on teaching critical thinking: apo-nid244676-1369771.pdf

7. Developing a story:

After you have collected enough data and analyzed it, you may start to think about the collection of pieces to come together into a “story.” As you watch scientific talks, you’ll notice that research is often presented a series of vignettes centered around a central topic. You’ll have a big problem or question that the research addresses in some meaningful way. Papers are written like this too: an introduction presenting the “big picture question” and the specific question of the study, then the methods, results, discussion, and finally, conclusion. One challenge of developing a project is deciding when enough data has been collected to make a complete story that can be presented (such as through a poster or a talk) or written up as a paper.

Keep in mind that the final story of your data may end up being unexpected, or very different from what you set out to test.

Putting together a cohesive and compelling story is very challenging for a new scientist to realize, so it’s important to involve mentors in this process! Your story may also be a part of a larger project that you are working on in a collaboration—they are not always stand-alone papers.

8. Presentations and writing a manuscript:

Once you feel like you have a story, you can start putting it together for presentations and/or a manuscript. At the beginning stages, you will likely present your results in a poster or talk before writing up a manuscript. At later stages (once you have a few/many publications), you may primarily present published research, or a combination of published and unpublished work.

The details of preparing a presentation and writing up a manuscript will be discussed in detail in another post.

Undergraduate TrainingCallie ChappellMarch 19, 2020undergraduates, scientific project, scientific method2 Comments