On this appendix page you will find resources and links based on my article for the Journal of Economic Education:
Citation: T. Dvorak, S. D. Halliday, M. O’Hara, and A. Swoboda, “Efficient empiricism: Streamlining teaching, research, and learning in empirical courses,” The Journal of Economic Education, vol. 50, no. 3, pp. 242– 257, 2019. doi: 10.1080/00220485.2019.1618765. Available: https://doi.org/10.1080/00220485.2019.1618765.
Corresponding Author: shalliday@smith.edu
These exercises spanned macroeconomic data such as labor force participation rates from FRED, micro data from particular markets such as the Saumaty fish market in Marseille, survey data, such as the World Values Survey, and laboratory data, such as the data from decision experiments over risky choices and public goods games with individuals or teams.
[To be completed - instructor is facilitating study abroad and will provide details in January]
These links will be provided upon acceptance.
In recent years, the learning curve for instructors and students has been flattened by the release of R Studio and a variety of packages that facilitate teaching and learning R. We recommend the following packages.
The tidyverse is a collection of packages written by developers affiliated with R Studio (Wickham 2017) that provides a variety of functions for data manipulation that are as easy, if not easier, to use than functions Stata-users typically use. Demonstrating the use of these functions to students, alongside the all-in-one visualization function , provides the basic infrastructure for teaching R in a way that does not require the steep learning curve that previously existed.
mosaic is another useful package developed to teach statistics to undergraduates (Pruim, Kaplan, and Horton 2017). It provides a variety of functions that Stata-users would be accustomed to using, for example, tally is roughly equivalent to tab in Stata. It also incorporates some drop-down menu choices for creating graphics to teach the basics of plotting data and shows the commands that generate the graphical output.
stargazer is a package that helps to produce publication quality tables in R Markdown. It offers a friendly way to produce tables of summary statistics and regression results similar to Stata’s outreg, but with the added advantage that the integrated structure of Markdown inserts these tables directly into the text with no cutting or pasting necessary.
remedy is a package that facilitates the learning (and production) of R Markdown by integrating drop-down and point-and-click menus to bold, highlight, or otherwise edit text. A brief introduction is available here: https://www-r–bloggers-com.cdn.ampproject.org/v/s/www.r-bloggers.com/remedy-is-now-on-cran/
Ariely, Dan, Anat Bracha, and Stephan Meier. 2009. “Doing Good or Doing Well? Image Motivation and Monetary Incentives in Behaving Prosocially.” American Economic Review 99 (1): 544–55. doi:10.1257/aer.99.1.544.
Berry, James, Lucas C Coffman, Douglas Hanley, Rania Gihleb, and Alistair J Wilson. 2017. “Assessing the Rate of Replication in Economics.” American Economic Review 107 (5). American Economic Association 2014 Broadway, Suite 305, Nashville, TN 37203: 27–31.
Carr, Michael D, and Phil Mellizo. 2013. “The Relative Effect of Voice, Autonomy, and the Wage on Satisfaction with Work.” The International Journal of Human Resource Management 24 (6). Taylor & Francis: 1186–1201.
Giné, Xavier, Dean Karlan, and Jonathan Zinman. 2010. “Put Your Money Where Your Butt Is: A Commitment Contract for Smoking Cessation.” American Economic Journal: Applied Economics 2 (4): 213–35. doi:10.1257/app.2.4.213.
Godlonton, Susan, and Rebecca L Thornton. 2013. “Learning from Others’ Hiv Testing: Updating Beliefs and Responding to Risk.” The American Economic Review 103 (3). NIH Public Access: 439.
Guryan, Jonathan, and Melissa S. Kearney. 2008. “Gambling at Lucky Stores: Empirical Evidence from State Lottery Sales.” American Economic Review 98 (1): 458–73. doi:10.1257/aer.98.1.458.
Hargreaves Heap, Shaun P., and Daniel John Zizzo. 2009. “The Value of Groups.” American Economic Review 99 (1): 295–323. doi:10.1257/aer.99.1.295.
Hsieh, Chang-Tai. 2003. “Do Consumers React to Anticipated Income Changes? Evidence from the Alaska Permanent Fund.” American Economic Review 93 (1): 397–405. doi:10.1257/000282803321455377.
Jensen, Carsten, and Michael Bang Petersen. 2017. “The Deservingness Heuristic and the Politics of Health Care.” American Journal of Political Science 61 (1): 68–83. doi:10.1111/ajps.12251.
Kube, Sebastian, Michel André Maréchal, and Clemens Puppe. 2012. “The Currency of Reciprocity: Gift Exchange in the Workplace.” The American Economic Review 102 (4). JSTOR: 1644–62. Malmendier, Ulrike, and Klaus M. Schmidt. 2017. “You Owe Me.” American Economic Review 107 (2): 493–526. doi:10.1257/aer.20140890.
Sautmann, Anja. 2013. “Contracts for Agents with Biased Beliefs: Some Theory and an Experiment.” American Economic Journal: Microeconomics 5 (3): 124–56.
Sutter, Matthias, Martin G. Kocher, Daniela Glätzle-Rüetzler, and Stefan T. Trautmann. 2013. “Impatience and Uncertainty: Experimental Decisions Predict Adolescents’ Field Behavior.” American Economic Review 103 (1): 510–31. doi:10.1257/aer.103.1.510.
Tanaka, Tomomi, Colin F. Camerer, and Quang Nguyen. 2010. “Risk and Time Preferences: Linking Experimental and Household Survey Data from Vietnam.” American Economic Review 100 (1): 557–71. doi:10.1257/aer.100.1.557.
Voors, Maarten J, Eleonora EM Nillesen, Philip Verwimp, Erwin H Bulte, Robert Lensink, and Daan P Van Soest. 2012. “Violent Conflict and Behavior: A Field Experiment in Burundi.” The American Economic Review 102 (2). American Economic Association: 941–64.
The instructor gleaned the following lessons. First, though data are readily available in most circumstances, in many cases some of the data may not be available for reproduction. Second, even when data are available, the paper may be too hard for students to reproduce in the available time unless the instructor has substantial amounts of time to facilitate the students’ work and provide substantial assistance. Lastly, and most reassuringly, many students are motivated by the quest to confirm or overturn existing research results as they become deeply engaged with the research process and committed to understanding whether and how to verify the results. The projects were improved by the students’ ability to use the free-of-charge resources of R Markdown and R Studio (and using online resources when trying to understand a given statistical problem). Students spent time over fall break and Thanksgiving break collaborating on their projects, which the instructor has not been able to have students do previously because of the licensing constraints over other packages, demonstrating how free and open resources are useful in the context of statistics and behavioral economics instruction.
Students were able fully to reproduce the following papers: Voors et al. (2012), Kube, Maréchal, and Puppe (2012), and Sautmann (2013). In the case of Godlonton and Thornton (2013), though students were able to access the Malawi Longitudinal Study of Families and Health (MLSFH), they could not obtain the paired proprietary data and were unable to reproduce all of the analysis as a result, but they were able to reproduce some of the results from the paper (partial reproducibility). In the case of Guryan and Kearney (2008), the instructor had incorrectly assessed how hard the paper would be to reproduce in the time available, and so changed course mid-way and asked the students to do a partial reproduction with some new work based on the data (they produced some original data visualizations to explain the research question and its corollary results). In the second iteration of the course, students were able fully or partially to reproduce Malmendier and Schmidt (2017), Tanaka, Camerer, and Nguyen (2010), Sutter et al. (2013), Hargreaves Heap and Zizzo (2009), Jensen and Petersen (2017), Ariely, Bracha, and Meier (2009), and Giné, Karlan, and Zinman (2010).
We have selected all comments that referred to R and R Studio. Other comments were often content specific, e.g. commenting on papers, the textbook, etc, so they were excluded.
2015-16: In Response to the question: “What features of this course made the most valuable contributions to your learning?”
2017-18: In Response to the question: “What features of this course made the most valuable contributions to your learning?”
| Program Capability | R and R Markdown | Stata |
|---|---|---|
| Notebook ability: produce output in document as you proceed | Yes, R Markdown as a Lab Notebook | No |
| Produces multiple export formats | Yes, all using R Markdown with markdown syntax | do files with different syntaxes, format-specific syntax in, e.g. dyndoc. Limited format types. |
| Move between code and prose easily | Yes, code is in “code chunks” and can use R or other types of code, e.g. Python; prose is in between the code chunks. | No. Output produced then cut and paste into documents or used with scripted LaTeX |
| Connecting different functions to each other | Yes, tidyverse package uses the magrittr pipe | No |
| Point and click figures | Yes, in the mosaic package (code produced); else code in ggplot package |
Yes, built in; else code for twoway |
| Point and click importing of data | Yes, in RStudio (using the tidyverse) and shows code | Yes, similarly and built in; shows code |
| Server/Cloud Edition (to ease teaching/collaboration) | Yes, R Studio Server (free to educational institutions) and R Studio Cloud | No, not as yet. |
There are a variety of benefits for using language and documents, like R and R Markdown, that many departments outside of economics use. At the institutions where the authors teach there are a variety of support structures to assist students in learning statistics and statistical programming software. For example, at one institution there are tutors in the quantitative learning center who all have experience with R Markdown and who can act as tutors for statistical and econometric concepts. Furthermore, the statistics department has additional resources: a statistics fellows program where the fellows have drop-in hours available both to faculty and to students. Though institutions may not have such infrastructure in place at first, as it develops over time it provides excellent opportunities for students to hone their skills and to work as peer tutors.