Chapter 8: Writing Methods, Results, and Discussion

Writing the Methods Section

This section is pretty straightforward and well-set for all secondary studies since we can use the original study’s methodology. By using an existing study methodology of the dataset that we are using for our graduate research project, we need to read, understand, and provide a summary of each part of the original study’s methodology. Particularly, it is necessary to focus on the original study’s research design, types of sample (probablity vs. non-probability sampling), sampling methods, and data collection process for our secondary research project. Depending on the original study, we need to review its data dictionary, codebook, user guide, or user manual to find information for each component of the original research methodology. The following are the key points to be included in the graduate research paper/thesis.  

Research Design. Research design indicates the type of the original research and how the data are collected. The type of research can be identified depending on the types of data such as quantitative or qualitative research. Quantitative research collects numeric data based on types of quantitative research such as surveys, experimental, quasi-experimental, and single system designs while qualitative research collects data comprising words, photos, and images through verbal interviews, focus groups, case study, life history, participant observation, and ethnography. When we identify the original research utilized either quantitative or qualitative data, then we need to identify the specific type of research under quantitative or qualitative research. The secondary study this book addresses focuses on numeric data based on a survey questionnaire in quantitative research.

Another type of research is related to time-dimension: cross-sectional vs. longitudinal research. Cross-sectional research collects data just one time at a given point in time while longitudinal research collects data multiple times over extended periods of time to examine developmental trends or change over time among the population of interest. Longitudinal research further has three different types of research, including trend, cohort, and panel studies based on whether the multiple data collection follows the same study participants or not. Longitudinal research that doesn’t follow the same study participants and collect data from different participants every time of data collection, can be a trend or cohort study. A trend study collects data every year, every other year, every 5 years, every 10 years and so on. The Centers for Disease Control and Prevention implements the Youth Risk Behavior Surveillance System biannually and the Behavioral Risk Factor Surveillance every year, while the U.S. Census data are collected every ten years. The purpose of those studies is to understand the trend of the population over time. Cohort research follows and collects data from the same cohort, which is defined as a group of people who share a certain defining characteristics in common, but not necessarily the same people to understand trends or outcomes of a certain cohort group. Research examines a cohort study with baby boomers and MZ generation every year, every 5 or ten years without following the same people. Longitudinal research that follows the same study participants over time is called a panel study.

In many original studies, their research design may not be clearly stated in user manuals or data dictionaries. As such, it is critical to clarify and understand the study’s data format and data collection methods to identify the types of research design.

It is noteworthy that IRB (Institutional Review Board) approval status should be indicated in this section. When we access the dataset and other materials without any credentials or restrictions, IRB approval is not required. However, we need to provide the web link to access the dataset and other materials in this section. When we access the dataset with a credential such as creating an account or requesting a dataset by certain means, then we are required to fill out and submit an IRB form to get approval from the Institutional Review Board. A secondary study can be exempt or expedited since the original research has been reviewed and approved by their IRB.

Sampling. Sampling indicates sampling methods describing how the study participants were recruited in the original research. To understand the original research’s sampling methods, we briefly revisit basic information. In sampling methods, there exist two key sampling methods: Probability (called random sampling) and non-probability sampling methods. Probability sampling methods include the following four types: 1) simple random, 2) systemic random, 3) stratified random, and 4) multi-stage cluster random sampling methods. Simple random sampling means simple random picks from the total possible subjects from the list of the total possible subjects which is called a sampling frame using a random table. System random sampling selects research participants with a certain interval (every kth element) from the sampling frame. Stratified random sampling selects subjects at random by stratum/strata that can be a certain characteristic we can group the total possible subjects. The stratified sampling methods can be used for research that examines differences in groups divided, grouped, or stratified by certain traits (i.e., female/male, Medicaid recipient/non-Medicaid recipient, freshman/sophomore/junior/senior, violence victims or not, racial groups, etc.). To implement stratified sampling methods, we need to divide the total possible subjects into specific sub-groups that generate a few sub-sampling frames and select the study participants from each of the sub-sampling frames with either simple or systematic random sampling methods. On the other hand, multi-stage cluster sampling indicates random picks based on multiple stages such as different sizes of geographical boundaries (streets, census tracts, zip codes, cities, counties, and states) or other ranks such as a class within a major, college, and university.

        Non-random sampling methods include convenience (availability), snowball, quota, and purposive sampling. Convenience sampling is called availability sampling as we collect data from any available people who meet the eligibility criteria to participate in research. Snowball sampling begins with a few study participants, and through referrals from those participants, bigger samples are obtained. Snowball sampling is a useful method to recruit research participants especially when it is hard to identify research subjects due to rare occurrences of such people (i.e., people with rare diseases and other rare experiences). Purposive sampling, also known as judgmental sampling, refers to a technique in which only people with specific characteristics are selected for the sample. In other words, units of sample are selected on purpose relying on the researcher’s or other experts’ judgment. Quota sampling is a non-probabilistic version of stratified sampling. Similar to stratified sampling, in quota sampling, a population is divided into sub-groups, and then non-probability sampling strategies, such as convenience, purposive, and/or snowball sampling techniques are used to recruit study participants.  

        To identify the sampling method the original research utilized, we need to understand the process of selection and recruitment of the study participants, especially when the study did not describe the sampling procedure in detail.

Data collection. Many students are confused about the sampling methods and data collection. As briefly explained above, the sampling method is to recruit the study participants, while data collection is about the logistics of how the study participants complete and provide their information such as how they received, completed, and returned surveys, explaining how the original study collected the data. The survey can be done in person with a hard copy of the survey questionnaire, by telephone interview, online, and by mail.  For each type of survey data collection, we need to provide specific logistics of data collection. For example, an online survey can be done as researchers could obtain email addresses and send out the survey to the study participants, or researchers provide a survey link on social network sites or research websites or a QR code on flyers and the study participants can access the survey questions. Identifying and recruiting the study participants are not enough when we collect data since we are not sure whether they will participate in the data collection for the study. So the data collection process requires detailed logistics in terms of reaching out and getting permission from the study participants. Thus, data collection requires precise information about the methods of obtaining consent forms, the location and ways the study participants can complete surveys, and the number of the study participants who actually completed the survey out of the total possible people who were reached out (also known as the response rate^[1]).

Instruments/Measures. This section provides information about all of the variables we will include for our study by listing and explaining the actual survey questions the original research developed and collected the data. We also provide information about how we clean, modify, recode, and transform the original survey questions or variables for our study in detail, especially what we did in steps for Chapter 4.

Data analysis. This section explains what we did for data analyses in terms of univariate, bivariate, and multivariate analyses covered in Chapters 4 to 7. It will be helpful to revisit this chapter after all of the analyses are actually done.

Writing the Results Section

This section provides information about study findings obtained from data analyses detailed in Chapters 4 to 7. We present a narrative sample write-up, corresponding tables and figures for each of the univariate, bivariate, and multivariate analyses. Once we have findings, we organize them in order of univariate, bivariate, and multivariate analyses. Univariate analysis begins with frequencies and percentages reporting independent variables first and then the dependent variable. Next is bivariate analysis to address an association or relationship between the IVs and DV. The last part is multivariate analysis if we go this far. This analysis compares the dependent variable with the independent variables all at once while controlling for each other.

Results are presented in written text as well as in tables, which is great for readers to catch key points of lengthy textual information presented in table format. Presentation of findings in the text and in tables should follow the APA format styles as presented in Chapters 4 to 7. The APA formatting and some writing tips are also described in Chapter 9.  

Writing the Discussion and Conclusion Sections

The Discussion section is composed of four parts: 1) the significance of the findings, 2) the graduate research project’s limitations and strengths, 3) the implication of the findings, and 4) the Conclusion. First, the significance of the findings provides key findings of our study with the meaning of the findings. We present key findings one by one based on our study hypotheses or the findings we want to highlight.  

Significance of the Study Finding. We compare our study findings with existing studies we reviewed in Literature Review in terms of whether our study findings are expected as the existing studies found and reported. In case our study findings are consistent with the existing studies’ findings, then we discuss both our findings and existing study findings together using the same or additional citations regarding what the results mean. In case our study findings are inconsistent with the existing study findings, we discuss why our findings are different from the existing study findings regarding insignificant findings or different directions of the relationship with differences in study methods or other reasons, justifications, or other confounding factors that may affect our findings or other study findings. We can also think of the original study methodology that may have led to unexpected findings in comparison with other existing studies. In the Discussion section, there is room for some of the personal and editorial comments that are not allowed in other sections. It is always better to provide citations to support your comments in this section.

We do not repeat the findings presented in Results with statistical values, but we explain the meaning of the study findings and how the findings are consistent or inconsistent with existing studies we write in Literature Review. It is best to make notes in preparation for this section as we carry out and proceed with the study, analyze data, write up the Results section, and compare our findings with the existing studies' findings reported in the Literature Review section. This will help us with what to write in the Discussion section. We can use the same citations or add more citations for our justification when we compare our findings with the existing study findings.

        Example 1 below compared and confirmed their study finding of females’ hopelessness with existing study findings as both studies found the vulnerabilities of female adolescents’ mental health affected by parental mental health. Example 2 discussed that their study findings are inconsistent with existing study findings as their study hypotheses were built with existing study findings. They further discussed possible explanations justifying why their findings are different based on the limitation of their study as their study did not collect other variables that may have affected their findings. Example 3 compared findings with other study findings by stating that their findings are similar to existing study findings but different from other studies.    

Example 1. Females’ hopelessness was high when their caregiver showed high levels of depression and low levels of traumatic stress at age 12, but female’s hopelessness level increased over time as the caregiver’s levels of depression decreased and caregiver traumatic stress increased. These findings confirm that female adolescents are more vulnerable to caregiver mental health, which is consistent with the empirical literature base (Bouma et al., 2008). [From Hooper et al., 2023, p.14]  

Example 2. In contrast to our hypotheses, we did not find evidence that preadolescents’ perceptions of positive aspects of parenting, such as consistent discipline and acceptance, were protective in the presence of maternal depressive symptoms. There could be several reasons why we did not find evidence that positive parenting attenuated the effects of maternal depressive symptoms on child outcome. First, it may be that children who perceive their mothers as accepting and consistent are at lower risk for emotional and behavioral problems, regardless of whether their mother was depressed. It is also possible that the preadolescent children who perceived their mothers as more accepting and consistent were doing better in ways that were not measured in our study. [From Zalewski et al., 2017, p.569]

Example 3. We found that adolescent girls reporting recent dating violence were 60% more likely to report 1 or more suicide attempts in the past year. This association is similar in magnitude to that reported among predominantly white adolescent girls from Massachusetts (Silverman et al., 2001) and Vermont (Kreiter et al., 1999). Our findings differ from those of Borowsky et al (2009) who found, in a national sample of teenagers, much stronger associations between baseline violence victimization and suicide attempts during a 1-year follow-up (odds ratio, 8.9 for black girls, 7.7 for white girls, and 2.7 for Hispanic girls). However, the increased odds ratios in their study may have been observed because the authors did not distinguish between dating violence and other types of violence reported by teenagers. [From Olsen, 2007, p.543]

Limitations and Strengths of the Study. We need to identify all of the ways in which the study design limits the certainty of our findings, such as methodological, ethical, and cultural considerations and/or limitations by both the original study and our secondary study. Although all studies have limitations, it is important to address the strengths of both the original study and our study as all studies also have strengths in methodological, ethical, and cultural aspects.

        The following examples addressed the strengths of the study focusing on their study purposes, study populations, and the study design such as longitudinal data although the study is a secondary data analysis. It also addressed the study limitations based on the original study design such as limitations of the original data and variable collections.

Example 1. The greatest strength of the current study is that the study focused specifically on Black American youth living in impoverished neighborhoods. A second strength is the longitudinal nature of the study….. The study was also limited based on the timing of the data collection between the youth and the caregivers. The caregiver sample survey was collected in conjunction with the youth survey results, in that they were collected while the youth was participating in the MYS study….   [From Hooper et al., 2023, p.15]  

Example 2. The study had several strengths, which include adding to the small literature on the effects of maternal depression symptoms during preadolescence (Lovejoy et al., 2000). First, by controlling for initial levels of co-occurring behavior problems while testing multiple parenting behaviors simultaneously, we were able to examine the unique predictors of initial levels and trajectories of internalizing and externalizing problems. Second, ….. Despite these strengths, the study had several limitations. Other important variables were either not measured or not included due to power constraints…. Another limitation….  [From Zalewski et al., 2017, p.569]

Example 3. One strength of this study was that we were able to examine the relationship between violence victimization in a population at high risk…….  Limitations of this research should be noted. First, as a cross-sectional study, we were not able to assess the temporal relationships between dating violence, sexual assault, and suicide attempts…. Second, our analyses of dating violence, sexual assault, and suicide attempts were all based on single-item responses….. [From Olsen, 2007, p.543 - 544]

Implications of the Study. Despite the study's limitations, all studies have a purpose of conducting research. In this part, we discuss and emphasize the implications of the study and study findings in terms of practice, policy, and research. We also recommend further practice, policy, and future research. This can be done in terms of overcoming methodological problems, clarifying findings, and/or pursuing research in new directions.

        The following examples addressed implications for treatment and future research (Example 1), and implications of the study findings applicable to the general populations (Example 2).

Example 1. …..The results of this study indicate that four specific areas be addressed: (a) caregiver mental health, (b) youth mental health, (c) attachment to peers, and (d) attachment to school. While caregiver and youth mental health must be considered as components of a comprehensive treatment approach, there are numerous evidence-based interventions for treating mental illness. The focus, which is supported by the research described above, is on positive attachment to peers and school. Attachment to peers is a critical component to human psychosocial development. Strong, positive peer relationships have the potential to lessen the negative effects of poor caregiver-child dynamics. Strategies that encourage positive peer relationships should be thoroughly explored in future research [From Hooper et al., 2021, p.16]  

Example 2. …Our study is unique in evaluating these relationships in a population of predominantly minority urban youth and in evaluating violence victimization, while controlling for multiple potential confounding factors, in a single model. While our study focused on public high school students in a single urban area, our results are likely generalizable to urban youth across the United States. Our findings are of concern because dating violence and sexual assault are highly prevalent among urban youth. Teenagers reporting violence victimization are at increased risk for suicide attempts….. [From Olsen, 2007, p.544]

Conclusion. In closing, we provide a brief overview of our study in terms of primary purpose, findings, the significance of the study, and/or future recommendations. Following are the examples that provide a summary and implication/significance of the study findings, and future direction of research.

Example 1. ….The current longitudinal study offered insight into significant changes in levels of hopelessness over time for youth and a significant positive relation between caregiver distress and youth level of hopelessness. ….. Taken together, these findings underscore the importance of identifying educational junctures whereby school attachment can be enhanced. Future research ought to consider at which points should interventions be put into place that encourages development of strong, positive school attachment. In the US, the school system is a culturally relevant and ecologically valid context to address the combined effects of education and health on youth outcomes. [From Hooper et al., 2021, p.16]  

Example 2. This study provides modest evidence that internalizing and externalizing problems in preadolescents differ as a function of maternal depressive symptoms and parenting. Clinical implications of this work include the potential benefit of addressing parenting in the context of maternal depressive symptoms, as the interaction of a child’s experience of maternal depressive symptoms and rejecting parenting may exacerbate emotional problems. [From Zalewski et al., 2017, p.569]