JEREMY Y WANG http://jeremyywang.com a grad student's thoughts on mind, brain, education and science posterous.com Thu, 24 May 2012 11:11:00 -0700 Magnifying the Universe http://jeremyywang.com/magnifying-the-universe http://jeremyywang.com/magnifying-the-universe

Copyright 2012. Magnifying the Universe by Number Sleuth.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 22 May 2012 11:44:00 -0700 Gauging Public Opinion on Climate Change Policy : NPR http://jeremyywang.com/gauging-public-opinion-on-climate-change-poli http://jeremyywang.com/gauging-public-opinion-on-climate-change-poli

FLATOW: Yeah. And you say that by a margin of 3-1, Americans say they would be more likely to vote for a political candidate who supports a, quote, "revenue neutral tax shift." What does that mean?

LEISEROWITZ: Yeah. Really interesting. So we all remember now that there was this big fight in this country over cap and trade, you know, this market mechanism that was put forward a couple of years ago which did not pass. At the same time, there's been a whole other community of scholars that have looked and then proposing a different approach, which is basically that you increase taxes on something that we think is, in this case, is a bad, fossil fuel use, and decrease an equivalent amount in taxes on something that we all think of is good, which is income, income taxes. And so this is often called a tax swap that, basically, we don't let the elected officials, we don't give Congress a cent more, but that we increase taxes on fossil fuels and decrease taxes on people's income tax by an equivalent amount.

And what's interesting about that is who supports that? We see everyone from Al Gore, on the one hand, to very conservative former Representative Bob Inglis, on the other, liberal institutions like Brookings Institute, to the very conservative American Enterprise Institute, support this exact same idea. And what we, again, find is that, as you just said, 3-1 American support that, including Republicans, 2-1, say that they would be more likely to vote for a candidate who supports that kind of a policy.

"Revenue neutral tax shift" doesn't quite have a ring to it... can we ge some marketers on that?

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 22 May 2012 10:48:00 -0700 [article] How Reliable Are the Social Sciences? - NYTimes.com http://jeremyywang.com/article-how-reliable-are-the-social-sciences http://jeremyywang.com/article-how-reliable-are-the-social-sciences
Public policy debates often involve appeals to results of work in social sciences like economics and sociology.  For example, in his State of the Union address this year, President Obama cited a recent high-profile study to support his emphasis on evaluating teachers by their students’ test scores.  The study purportedly shows that students with teachers who raise their standardized test scores are “more likely to attend college, earn higher salaries, live in better neighborhoods and save more for retirement.”

The perennial question about the value of social science research for policy decisions, revisited.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 22 May 2012 10:48:00 -0700 [article] How Reliable Are the Social Sciences? - NYTimes.com http://jeremyywang.com/article-how-reliable-are-the-social-sciences http://jeremyywang.com/article-how-reliable-are-the-social-sciences
Public policy debates often involve appeals to results of work in social sciences like economics and sociology.  For example, in his State of the Union address this year, President Obama cited a recent high-profile study to support his emphasis on evaluating teachers by their students’ test scores.  The study purportedly shows that students with teachers who raise their standardized test scores are “more likely to attend college, earn higher salaries, live in better neighborhoods and save more for retirement.”

The perennial question about the value of social science research for policy decisions, revisited.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Wed, 09 May 2012 09:24:00 -0700 Done deal: Connecticut Senate approves education overhaul http://jeremyywang.com/done-deal-connecticut-senate-approves-educati http://jeremyywang.com/done-deal-connecticut-senate-approves-educati
Media_httpbloximagesn_cfjuw

This is a big time overhaul of a state education system. It will be interesting to see what happens in the coming years.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Thu, 22 Mar 2012 23:00:00 -0700 The Carbon Blanket http://jeremyywang.com/the-carbon-blanket http://jeremyywang.com/the-carbon-blanket

Research has shown that people respond to vivid images the evoke emotional responses (Duh, 0001).

One example that comes up in the environmental education and environmental psychology literature is the "ozone hole" (Kollman & Agyeman, 2002). You probably remember being taught about this in school at some point, and it probably made sense to you, at least intuitively.

Holes in things = generally bad (like, holes in a boat or holes in a pocket).

Therefore, holes in atmosphere = probably bad. 

Well, this emotional response can be good or bad. If there is some change that can be made that is quick and requires little effort, people will make the change. This was the case for CFCs and the ozone hole, which has more or less been mitigated.

On the other hand, understanding the greenhouse effect and global climate change is complicated. There's no hole to patch up or dirt to scrub away (well, you can do carbon scrubbing in smoke stacks), making the whole thing kinda... blah, whatever.

A potential solution is to create an image to describe how our actions influence climate: The Carbon Blanket.

OK, so "blanket" doesn't necessarily have the impact of "hole," and it's not the most scientifically accurate model, but it is a lot more accessible than a greenhouse (actually, it's probably a better analogical model). 

Here are some ways you can help spread this terminology:

"Wow, this Carbon Blanket sure made the ice melt fast this year."

"That Hummer sure is quilting the Carbon Blanket something fierce."

"This summer is going to suck if the Carbon Blanket keeps getting thicker."

Greenhouse Effect is so 90s.

Carbon Blanket is the new Ozone Hole.

 

(a quick Google search brings up only a handful of people using this terminology:

so, it's not an original idea, but I'm promoting it!)

 

 

 

Kollmuss, A., & Agyeman, J. (2002). Mind the Gap: Why do people act environmentally and what are the barriers to pro-environmental behavior. Environmental Education Research, 8(3), 239-260. doi:10.1080/1350462022014540

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Wed, 22 Feb 2012 12:50:00 -0800 [article] Harvard seeks to jolt university teaching - The Chronicle of Higher Education http://jeremyywang.com/article-harvard-seeks-to-jolt-university-teac http://jeremyywang.com/article-harvard-seeks-to-jolt-university-teac

February 5, 2012

Harvard Conference Seeks to Jolt University Teaching

By Dan Berrett

Cambridge, Mass.

A growing body of evidence from the classroom, coupled with emerging research in cognitive psychology and neuroscience, is lending insight into how people learn, but teaching on most college campuses has not changed much, several speakers said here at Harvard University at a daylong conference dedicated to teaching and learning.

Too often, faculty members teach according to habits and hunches, said Carl E. Wieman, a Nobel Prize-winning physicist and associate director of the White House Office of Science and Technology Policy, who has extensively studied how to improve science education.

In large part, the problem is that graduate students pursuing their doctorates get little or no training in how students learn. When these graduate students become faculty members, he said, they might think about the content they want students to learn, but not the cognitive capabilities they want them to develop.

"It really requires someone to be doubly expert," Mr. Wieman said. As sometimes happens in some disciplines and departments, a few people develop deeper knowledge of pedagogy. These doubly expert faculty members, he said, can show colleagues how to apply new approaches to teaching the discipline.

Such approaches would demand much more of students and faculty. Students should be made to grapple with the material and receive authentic and explicit practice in thinking like an expert, Mr. Wieman said. Faculty would need to provide timely and specific feedback, and move beyond lectures in which students can sit passively receiving information.

"We assume that telling people things without asking them to actively process them results in learning," Mr. Wieman said.

The conference, which also featured demonstrations of innovative approaches to teaching, was the first event in a new Harvard Initiative for Learning and Teaching, a project supported by a $40-million grant from two benefactors, Gustave M. and Rita E. Hauser. In addition to the conference, the money will pay for the redesign of classrooms at Harvard and for a grant program that will finance innovative ideas. More than 250 Harvard faculty, staff, and students have submitted letters of interest for projects costing nearly $10-million. Awardees will be selected in April.

Many colleges routinely hold seminars on teaching and learning. But the fact that Harvard is focusing on the subject—and that many speakers referred worryingly to the growth of online and for-profit providers—suggests a growing concern at even the most elite institutions that the classroom experience is not all it could be.

The Hausers wanted their money to have a broad effect across Harvard's departments and disciplines. They also wanted the university to respond to changes in students. "You can see there will be a fundamental break in how students are learning," Mrs. Hauser said in an interview, "and we thought Harvard should be at the forefront of that."

Confronting Misconceptions

Students are indeed changing, some speakers said. Their level of curiosity has declined over the past two decades, said Clayton M. Christensen, a professor of business administration at the Harvard Business School.

Mr. Christensen also drew an analogy between Harvard and the for-profit world, and General Motors and Toyota, describing how new businesses often enter the bottom of a market and claim untapped customers whom they reach through some new technological advance. Eventually, they move up-market and overtake the dominant player.

Higher education once was immune, he said, until the spread of online learning, which will allow lower-cost providers to extend into the higher reaches of the marketplace. "Higher education," he said, "is vulnerable to disruption."

And, while students are changing, several speakers described conventional teaching approaches as being ineffective.

Take, for example, the lecture, which came up for frequent shellacking throughout the day. It is designed to transfer information, said Eric Mazur, professor of physics at Harvard. But it does not fully accomplish even this limited task.

Lectures set up a dynamic in which students passively receive information that they quickly forget after the test. "They're not confronted with their misconceptions," Mr. Mazur said. "They walk out with a false sense of security."

The traditional lecture also fails at other educational goals: prodding students to make meaning from what they learn, to ask questions, extract knowledge, and apply it in a new context.

And yet, many speakers acknowledged, faculty members harbor their own misconceptions about learning, which still hold sway at Harvard and beyond.

One, said Mahzarin R. Banaji, a professor of psychology at Harvard, is what she called a "myth" about different learning styles, in which it is thought that some students learn best visually, others by hearing, and still others kinesthetically.

"There's no evidence, zero, that teaching methods should be matched up with different learning styles," Ms. Banaji said. "It's intuitively appealing, but not scientifically supported."

Assessing as Learning

Another commonly held notion, that studying is how learning occurs and testing is an afterthought, was upended by Henry L. "Roddy" Roediger III, a professor of psychology at Washington University in St. Louis, who has studied what is known as the "testing effect."

In an experiment, he broke students into three different groups: One studied a list of words eight consecutive times without taking any tests; the second studied the list six times and was tested twice. The last studied the words four times and took four tests. Two days later, they were asked to recall as many words as they could. Those who took four tests recalled words at up to twice the rate of those who only studied.

"Taking a test on something is a very effective way to learn about it," Mr. Roediger said.

But frequent quizzes—which he said should be low-stakes and not "deadly" multiple choice—often hit a wall of disdain among both faculty and students, he noted. "There's a kind of a conspiracy in higher education that professors don't like to give tests," Mr. Roediger said. "We hate grading tests. Students don't like taking them, so we don't give them very much."

But there are other ways to get students to truly learn, other speakers said. Asking students to explain concepts or to teach one another the material they have just learned are also effective.

Writing is often an effective pedagogical tool, too, several speakers said. For his history of psychology course, Mr. Roediger asks his students to send him short essays before each class meets. They respond to the reading. (Others at the conference who use this method said they sometimes ask their students to identify outstanding questions or relevant areas of their reading that have been left unexplored.) Mr. Roediger reads the one-page essays before class and works their thoughts into his comments.

But writing is also more than a means to convey content. It is a core skill that faculty members often hope their students will carry with them after they graduate, said Steven Pinker, a professor of psychology at Harvard who studies language and cognition. But even here, students and faculty often fail.

Students are trained to write in jargon-heavy language that obscures rather than reveals the underlying ideas. Mr. Pinker drew an analogy to teaching, saying that obtuse writing and poor teaching both reflect what he called the "curse of knowledge."

Having this curse means that a writer or professor often assumes knowledge the reader or student does not have. More important, the writer or teacher usually forgets that the reader or student is struggling to learn the material for the first time, which often was long ago for the teacher.

"It's hard to know what it is like for someone else not to know something that you know," Mr. Pinker said. "It's the chief driver of bad writing and, I would argue, bad teaching."

Hardly novel, nor a "jolt," but certainly good press for improving education at the college/university level.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Thu, 16 Feb 2012 15:42:00 -0800 Charter-school group seeks legislative changes to ease replication, ensure accountability | MinnPost http://jeremyywang.com/charter-school-group-seeks-legislative-change http://jeremyywang.com/charter-school-group-seeks-legislative-change
Media_httpminnpostcom_lzcvg

I visited HLA on Monday and can report that Shannon and his group are working incredibly hard - and it's paying off. They have a great vibe at the school and teachers are serving high-need students very well.

I hope that our legislators will see that policies like this will help replicate schools like HLA, while also shutting down charters that aren't working.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Fri, 03 Feb 2012 23:07:00 -0800 A Classroom Software Boom, but Mixed Results Despite the Hype - NYTimes.com http://jeremyywang.com/a-classroom-software-boom-but-mixed-results-d http://jeremyywang.com/a-classroom-software-boom-but-mixed-results-d
The federal review of Carnegie Learning’s flagship software, Cognitive Tutor, said the program had “no discernible effects” on the standardized test scores of high school students. A separate 2009 federal look at 10 major software products for teaching algebra as well as elementary and middle school math and reading found that nine of them, including Cognitive Tutor, “did not have statistically significant effects on test scores.

So much large-scale education research ends with "no discernable effects." I mean, like almost all of it. Seriously.

The linked article examines some of the research claims made about Cognitive Tutor, made and distributed by Carnegie Learning. I'm familiar with this software, and it is probably the best of what is out there in terms of educational software. The content is solid and the system is adaptive to the students. But the research results are mixed, depending on who you talk to.

Looking at the research, one could come to the conclusion that all educational interventions are crap. After all, if an intervention doesn't stand up to the rigor of scientific investigation, we probably shouldn't use it, right?

Well, another way to look at this is that maybe the "gold standards" of scientific rigor are not appropriate for educational contexts. Large-scale, randomized trials make sense for medical trails, but not for educational interventions. Why? Because educational systems are complex and have multiple inputs. But wait, isn't your body a complex system too.

Dr. David Agus was the guest on The Daily Show last night (2 Feb 2012) and suggested that medical thinking has been stunted ever since the discovery of pathogens. They make us think that all diseases have a cause and a cure. Dr. Agus suggests that diseases like cancer and Alzheimer's are different. They don't have a single cause; the body has to be treated as a whole.

Similarly, if we expect to improve educational outcomes for all students, we aren't going to do it by giving the same treatment to all students. These educational treatments need to be delivered at the right time, to the right student, by a trained professional (or computer that can aid a teacher).

The frustrating thing is, who's going to get on board with a program that doesn't meet rigor, especially when the bar might be unattainable?

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 17 Jan 2012 14:25:00 -0800 Gates Foundation Report On Measuring Teacher Effectiveness Suggests Three-Pronged Approach http://jeremyywang.com/gates-foundation-report-on-measuring-teacher http://jeremyywang.com/gates-foundation-report-on-measuring-teacher
Media_httpihuffpostco_ochfg

I'd rank "Figuring out Teacher Evaluation" as a Top 3 education problem.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Mon, 16 Jan 2012 14:28:00 -0800 American Museum of Natural History Will Groom School Teachers http://jeremyywang.com/american-museum-of-natural-history-will-groom http://jeremyywang.com/american-museum-of-natural-history-will-groom
Media_httpgraphics8ny_rqbat

Interesting approach to science teacher ed. AMNH does some good professional development, so I hope their master's program follows in that success.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 13 Dec 2011 09:23:00 -0800 [article critique] Videogames and the Science of Expertise http://jeremyywang.com/article-critique-videogames-and-the-science-o http://jeremyywang.com/article-critique-videogames-and-the-science-o

[article] Scientific American: How a Computer Game Is Reinventing the Science of Expertise

*Warning: The article is more about gaming than the science of expertise.

Chess experts have given cognitive scientists a great model for understanding expertise. The work of cognitive scientists such as Simon, de Groot, Glaser, & Chi have helped us understand some of the basic differences between experts and non-experts. Their methodologies have been extended to other domains as well (such as physics problem-solving and chicken sexing - look it up).

What we know from these studies is that experts literally see the world differently. Well, I should qualify that... Experts perceive things in their domain of expertise differently than non-experts. For example, they are able to "chunk" perceptual information into bigger pieces that allows them to process more information than non-experts. They can also perceive subtle differences in features that others don't.

Think about sitting next to an elderly person struggling to use a computer. They don't see the same things that you see - all they see is a bunch of squares and words and symbols on a fancy TV. You, the expert, see a Twitter feed in a web browser, which just spoiled who got kicked off Top Chef this week.

Here's the key quote from the article for me:

For decades, a different game, chess, has held the exalted position of “the drosophila of cognitive science”—the model organism that scientists could poke and prod to learn what makes experts better than the rest of us. StarCraft 2, however, might be emerging as the rhesus macaque: its added complexity may confound researchers initially, but the answers could ultimately be more telling.

Now, I don't know much about StarCraft 2, but I do remember playing WarCraft back in the day. From what I remember, this game is about developing resources and building an army to defeat your opponent. Based on the description in this article, the goals of StarCraft 2 (SC2) aren't significantly different.

So, what is it about this game that is going to "reinvent" the science of expertise? Unfortunately, the article doesn't tell us much about this besides some possible cognitive activities that are involved. It is quite strong to say that this game with "reinvent" the way we study expertise. That said, I'll take a crack at some of the candidate reasons why SC2 could "reinvent" cognitive science of expertise. 

(1) "Multitasking"

Over and over again, scientists have shown that people are generally really bad at this. It is really difficult to divide attention between two tasks. However, with training, some tasks can be done more automatically. This is likely what is going on with SC2 - expert players are able to do menial tasks really quickly and automatically, and can therefore spend more of their mental effort doing other, more cognitively demanding tasks. I suspect that there is little transfer of whatever small amount of "multitasking" that is going on in SC2.

(2) Log Files

The replay files from SC2 are supposedly a rich source of data. But only if you know what's going on in a game and why a player is doing certain things. Chess games have been recorded for hundreds of years. Just because you have them doesn't mean you understand them, or that they will tell you anything about how humans think.

(3) Development of Expertise

This is probably the most promising offering for SC2 to cognitive science. If we examine log files for the same player over the course of their development, we might gain insight into how expertise develops. We could look at how good players develop and how not-so-good players develop. We could see if there are different courses of development or multiple paths to expertise.

Will videogames "reinvent" our understanding of expertise? Probably not. But they might help us understand how expertise develops and how we can foster that development.

 

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Fri, 21 Oct 2011 12:00:00 -0700 A Multimillion Dollar Education Bet http://jeremyywang.com/a-multimillion-dollar-education-bet http://jeremyywang.com/a-multimillion-dollar-education-bet
Media_httpblogsfgatec_jejvh

An interesting approach to educational reform: "money with a catch" (or two or three).

In one way, I like this approach - it allows the school leadership to take a multi-faceted approach to improving eduction. We know that teachers, curriculum, technology, infrastructure, and food (among a multitude of other things) are all important for school outcomes. So, money can be put in different baskets, as opposed to one.

A problem that may arise is that freedom to choose how to spend the money also means freedom to choose incorrectly. There probably isn't a "right" and "wrong" way to spend $1.6 million, but a lot of it could go to waste.

I guess this is the sense in which this is a multimillion dollar "bet." The bet is on whether local school leaders can figure out how to spend their money in the best possible way - something that we currently have more questions than answers about.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Mon, 12 Sep 2011 22:50:00 -0700 Ig Nobel Prize - Wikipedia, the free encyclopedia http://jeremyywang.com/ig-nobel-prize-wikipedia-the-free-encyclopedi http://jeremyywang.com/ig-nobel-prize-wikipedia-the-free-encyclopedi
The Ig Nobel Prizes are an American parody of the Nobel Prizes and are given each year in early October for ten unusual or trivial achievements in scientific research. The stated aim of the prizes is to "first make people laugh, and then make them think". Organized by the scientific humor magazine Annals of Improbable Research (AIR), they are presented by a group that includes Nobel Laureates at a ceremony at Harvard University's Sanders Theater, and they are followed by a set of public lectures by the winners at MIT.

Media_httpuploadwikim_aoaem

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Thu, 18 Aug 2011 07:53:00 -0700 For you AERA reviewers... Twelve Tips from Roediger http://jeremyywang.com/for-you-aera-reviewers-twelve-tips-from-roedi http://jeremyywang.com/for-you-aera-reviewers-twelve-tips-from-roedi

The Academic Observer

Twelve Tips for Reviewers

An occasional column by Henry L. Roediger, III

Many critical skills needed for becoming a successful academic are typically not taught in graduate school, at least not in any formal way. One of these is how to review journal articles. Few students coming out of graduate school have much experience reviewing papers, and yet, at least for those students continuing on in research, reviewing is a skill that will be increasingly critical as their careers develop. In fact, being a good reviewer can greatly help a career. If a young psychologist becomes known as an excellent reviewer, he or she may be selected as consulting editor, then associate editor, and then perhaps the primary editor of a journal.

How do people learn to review? I suspect most newly minted PhDs learn to review papers in the same way that the children in Albert Bandura’s famous 1960s studies learned aggressive responses, which is to say, by imitation. Just as children who watched one boy smack a BoBo doll with a bat (and received praise for doing so) would tend to do the same when it was their turn, so do young academics learn to review. Unfortunately, the analogy with the BoBo doll experiment is apt in another way: Because reviews are often highly negative, the new researcher implicitly learns from the negative reviews received on his or her own submitted papers that reviews are supposed to be negative. It is as if the implicit message is: “A reviewer’s job is to criticize the manuscript. Find any faults in the logic, method, results, and conclusions that the paper might have and then communicate these to the editor.” Hence, the cycle of negative reviews is perpetuated across academic generations.

In 2002, Robert Sternberg wrote a column in the Observer titled “On civility in reviewing,” decrying the often hostile nature of reviews in psychology. This led him to edit a collection of essays on different types of reviewing and how to go about them. The book Reviewing Scientific Works in Psychology (American Psychological Association, 2006) is excellent, and I urge readers to purchase it (although for truth-in-advertising purposes, I should note that I wrote one of the chapters, on reviewing chapters in edited books). The purpose of this column is not to replace that book but simply to provide some guidelines for reviewing papers that may be helpful to some, especially younger, readers. Abraham Tesser and Leonard Martin have a fine chapter in Sternberg’s book that provides further thoughts on reviewing articles for journals.

1. Know your mission. A reviewer’s job is to evaluate a submitted article, not (necessarily) to criticize it and certainly not to trash it. The editor is seeking your advice on whether or not to publish the paper, so you should point out both strengths and weaknesses of the paper and come to some balanced conclusion. Keep the big picture in mind: Is the problem addressed an important one in the context of the field? Does the current paper push knowledge forward in a substantive way? The question at issue is not “does this paper answer all the questions about this particular problem?” because the answer to that question is always “no.” The critical issue is whether the paper under review advances knowledge on the issue under consideration enough to be published. Yes, this requires human judgment, a notoriously fallible quality, but that is why there are two or three reviewers plus an editor. (Sometimes there are even more reviewers — too many, in fact — but I’ll save that problem for another column.) Always keep in mind that, as a reviewer, you are just one piece in the puzzle. You offer advice to the editor, and the editor decides on the publishability of the paper.

2. Be speedy. Your reviews should be timely. Don’t wait until the last minute to begin the review (or worse, after you get a reminder that it is overdue). If you want to establish a reputation as a good reviewer, be prompt and get your review in ahead of schedule. When I was editing the Journal of Experimental Psychology: Learning, Memory, and Cognition, I tried a young reviewer just out of graduate school. To my surprise, he returned the review in five days and it was excellent — fair, balanced, pithy, and wise. Just then, I had another paper that a reviewer had returned after holding it for a few weeks, pleading lack of time. I sent it to the same young reviewer and got another immediate turnaround (and this was in the days before e-mail made delivery of papers and the return of reviews instantaneous). I quickly added the person to the editorial board, having suddenly discovered that I couldn’t live without him. When you are young and new to the field, you might not be asked to review much. However, if you are a good reviewer, you may be discovered and suddenly you will have an overflowing reviewing pile. Don’t agree to review a paper unless you think you can do so in a timely fashion.

3. Read carefully. Yes, you need to be speedy, but not at the expense of accuracy. Read the paper carefully, and go back over parts that are not clear. This advice should go without saying, but I am amazed at how often reviewers will raise a point that is based on a misreading of the paper or, worse yet, raise a point as if it were new when, in fact, the point was addressed and rebutted in the paper. I confess to having made this mistake myself on occasion, and having the mistakes pointed out by the editor or other reviewers has made me more careful.

4. Say positive things in your review. Many reviews I read (both those of my papers and those of others’ papers when I am one of the companion reviewers) have nary a kind word. Rather, the review consists of an unrelenting series of criticisms that might leave the author ready to hang him- or herself by the end of the review. (Sometimes I will even see editors apologize in the action letter for the tone of a review.) Usually, even the worst paper has some redeeming features — the issue addressed is interesting, the literature review is excellent, or whatever. (Don’t push the compliments, though. No need to comment on the excellent choice of typography — “one hardly ever sees Franklin Gothic used in a manuscript” — if that is the only good thing you can think to say.) In short, even if you are recommending that the paper not be published for one reason or another, try to include some kind words.

5. Don’t exhibit hostility or mean-spiritedness in your review. The field suffers from too many hostile reviews, as Sternberg has noted in his essays on this topic. If there is a confounding that undermines the research in question, point it out directly but without being rude about it. You can say, “Unfortunately, I believe the authors’ conclusions may be limited by the fact that Factor X covaried with the critical Factor Y,” rather than “The authors obviously were poorly trained, as evidenced by their blunder of confounding Factor X with Factor Y, so this entire line of research was a waste of time.” (I have seen much worse statements than this one.)

6. Keep it brief. The editor is a busy person. He or she wants your opinion, but does not want a stream-of-consciousness record of every pithy thought you had while reading the manuscript. Give your conclusion about the publishability of the paper and provide reasons for it, but don’t go on and on. Reviews in most fields of psychology seem to have grown longer over the years (that is my subjective impression, at any rate), although great individual differences exist among reviewers. During my times as editor, I would sometimes get reviews whose length would rival that of the paper. Some of my own have been too long as well, especially in the days when I dictated them. I now try to keep my reviews to one or two pages, which probably makes both the editor and the author happy. Remember, the editor has to read the paper and the several reviews received and then write an action letter. No one wants your logorrheic review to gum up the process.

7. Don’t nitpick. Don’t go through the paper searching for misspellings and grammatical infelicities. Keep to the main point, the big picture. God created copyeditors for a reason, so you needn’t do their job. If a paper is unusually well or poorly written, that might be worth pointing out; and if a passage or point is difficult to follow, it is fine to note that too. However, there is no reason to go through the paper line by line, as some reviewers do, saying that trail on page 16, line 4, should actually be trial. (Spellcheck cannot catch this type of error; though it does take away some of my favorites from the old days, when people would write about their “important resluts in Experiment 2.”)

8. Develop a good reviewing style. There is no universally agreed-upon reviewing style, so you must develop one that is comfortable for you. Some people begin with a summary of the paper. Personally, I think this is ok if the paper is confusing and if the editor might need a little help, but there is no need to provide a long summary for a paper that has a straightforward message. One typical style for reviews is to have an introductory paragraph stating the paper’s main point and the reviewer’s initial reaction. The reviewer might then make the case for publishing the paper, listing its strengths and discussing its importance. Next might come a section on the manuscript’s shortcomings and criticisms, as well as advice on how they might be overcome (if they can be). Finally, the review may conclude with a recommendation about the paper’s publishability, with the realization that this is advice for the editor (as discussed in Point 11). This schema is only a suggestion. Many other reviewing styles exist.

9. Be careful in recommending further experimentation. One of the easiest bad habits for reviewers to develop is to routinely recommend that further research be conducted before the paper is published. For most papers, you will be able to think of further experiments that might be done on the topic. However, if the author has provided several experiments already and made a coherent contribution, the critical question is not “can further experiments be done?” (the answer is “yes”), but “does the author need to do further experiments to support the conclusions being made?” (Of course, this question presupposes that one thinks that the conclusions are of interest to the field.) When I get back a review suggesting several further experiments I might do, I usually think “Those are fine experiments. Let the reviewer do them. I submitted the four experiments that (in my opinion) tell an interesting story on their own.” If you write about further research that might be done, use a high threshold and be sure to specify to the editor whether you think it is imperative for publication of the paper or rather just an idea for some future paper. Yes, sometimes a further study is absolutely required to clinch a point and make a paper publishable, but these cases are not as routine as some reviewers and editors make them out to be.

10. Watch for egocentrism. If you received a paper to review, chances are you yourself have published on that topic. Nearly every member of any scientific field is subject to the feeling of citation neglect — “The author should be citing my work more often.” Somehow, we feel compelled to remind the author that we made a similar point to hers, even if we embedded it on page 646 in footnote 2 of that article we published in an obscure journal in 1991. If you find yourself writing statements like that — which are read as “please cite me more” — you know you’ve stepped over the line. Yes, you know your own work better than anyone, and yes, you would like authors to cite it appropriately, but try to resist constant advice to cite your own work. Occasionally you might want to point out one of your papers that is overlooked by the current authors, but don’t make a habit of it.

11. Make a recommendation about the paper, unless the instructions from the editor tell you not to. Most journal editors want to know what your bottom line is. Rather than just discussing features of the paper and letting the editor guess what your overall opinion is, state it at the end of the review. You may have listed positive and negative factors, but how do you weight them? Conclude with a statement such as “On balance, I do not think this paper should be published in its current form because of [reiterate the two main reasons],” or “I believe the current paper should be published if the author can address [two or three major criticisms].” Don’t be dogmatic about it, though, to give the editor leeway. Sometimes, editors permit you to write a review and then make a statement or recommendation for only the editor’s eyes. Although I have encouraged kinder reviews, you should never write a flattering review of the paper and then write the editor a note that says “I tried to be nice to the authors, but this paper is really terrible and should be rejected.” You put the editor in an untenable position with this practice, so make sure your recommendation is consistent with your review, especially if only the editor sees your recommendation.

12. Sign your review. Or, if you can’t bring yourself to do that, at least write your review as if the author will learn your identity and you wouldn’t be embarrassed. I sign all of my reviews and have done so for many years. I think if everyone did, most of the problems of nastiness in reviewing would disappear. As psychologists have repeatedly shown (e.g., Zimbardo’s prison experiment), human beings do not display their best behavior when they are cloaked behind the mask of anonymity. Signed reviews will usually be more polite and diplomatic, with much less tendency for brutal, unvarnished criticism. Of course, you still want to give your honest opinion, but (as discussed above) there are helpful and unhelpful ways of relating that opinion. Nonetheless, many discussions over the years have convinced me that people object to signing their reviews for all sorts of reasons. If you fall into this category, my advice is to still write the review as if you were going to sign it. This makes it more likely that you will follow the golden rule of “review unto others as you would have them review unto you.” You may still frequently need to criticize papers, but you can learn to do so in ways that are not blatantly offensive. Signed reviews may not win friends because often you are saying “don’t publish this paper,” but it’s the right course of action, at least for me. Be willing to stand behind your words, not snipe from behind the hills. Also, if you blow a point in your review, you can be sure that the author will let you know and you can be more careful in the future.

Peer review is critical to the scientific process. Although we do not teach courses on reviewing, we should all be mindful of ways to improve the process. For further reading, let me once more recommend Sternberg’s book on Reviewing Scientific Works in Psychology.

P.S. If you have a review of mine in which I broke one or more of the rules above, no need to send it to me. I know I haven’t always followed my own advice in the past, but I’m trying to reform.

Henry L. Roediger, III is James S. McDonnell Professor at Washington University in St. Louis. This column benefited from the advice of David A. Balota, Kathleen B. McDermott, Robert J. Sternberg, Endel Tulving, and John Wixted.


Voices of Experience

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Thu, 21 Jul 2011 20:18:00 -0700 Don't you love interactions? http://jeremyywang.com/dont-you-love-interactions http://jeremyywang.com/dont-you-love-interactions

Sf_condxefinteract1

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 05 Apr 2011 19:08:40 -0700 Citations are people too. http://jeremyywang.com/citations-are-people-too http://jeremyywang.com/citations-are-people-too One lesson I've learned in grad school is "Citations are people, too."

Early in my grad career, I'll admit that I put citations on a pedestal. There's good reason to do so; a published paper represents a fair amount of work and some degree of consensus among (some) experts in the field. (That isn't to say that all citations are equal.)

As I've progressed and had the opportunity to meet the people behind the citations at conferences and other professional events, I've been encouraged by knowing the people behind the research. It allows you to think about the researchers' motivations, biases, and quirks. It frees you up to consider how you might do things differently.

Most importantly, it opens up a dialogue between you and the once-monolithic literature. 

You are among your peers.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Wed, 16 Mar 2011 14:01:00 -0700 [audio] New Words, New Worlds http://jeremyywang.com/audio-new-words-new-worlds http://jeremyywang.com/audio-new-words-new-worlds
[via radiolab.org]

Story about research on a group of deaf children in Nicaragua that developed their own signing language that, in turn, appeared to change their thinking.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Wed, 02 Feb 2011 15:28:00 -0800 A primer on the problem with fMRI http://jeremyywang.com/a-primer-on-the-problem-with-fmri http://jeremyywang.com/a-primer-on-the-problem-with-fmri

"There are so many bad brain imaging studies, it's hard to believe," says Nikos K. Logothetis, director of the Max Planck Institute for Biological Cybernetics in Germany. "Too many of these experiments are being done by people who, unfortunately, don't really understand what the technology can and cannot do."

Logothetis and others believe that much of the misuse stems from the visual nature of the data. One study, by researchers at Colorado State University, showed that simply giving neuroscience students images from an fMRI machine, even if the images were redundant or irrelevant, made the students significantly more likely to find the data credible. According to Paul Bloom, a psychologist at Yale, this is because fMRI "has all the trappings of work with great lab-cred: big, expensive, and potentially dangerous machines, hospitals and medical centers, and a lot of people in white coats."

The data looks rigorous - it has the veneer of cutting-edge science - and people assume it's valid, even when the reasoning is shoddy.

"You can't just put people in a scanner and ask them whatever question you want," Logothetis says. "Many of these [fMRI] papers are such oversimplifications of what's happening in the brain as to be worthless."

A typical fMRI experiment goes like this: a subject is slipped into a tight space that's about the size of a coffin. The person is told to lie perfectly still, as even the slightest movement will muddy the results. At first, the subject does nothing. Then, he or she performs the experimental task, which might involve looking at a picture or making a decision.

While this is happening, noisy magnets whir overhead, as the machine detects the slightly different magnetic properties of blood with and without oxygen. The underlying assumption is that more active neurons require more oxygenated blood. Follow the oxygen, and you can construct precise maps of the brain at work.

Last year, the New York Times published an op-ed that used fMRI to investigate the brains of swing voters as they stared at photos and videos of presidential candidates. For instance, the scientists found that pictures of Mitt Romney led to activity in the amygdala, while pictures of Hillary Clinton activated the anterior cingulate. (Interestingly, the only two candidates who inspired "little activity in areas of the brain associated with thought or feeling" were Barack Obama and John McCain.)

Within days, 17 prominent cognitive neuroscientists signed a letter criticizing the study. "We are distressed," the scientists wrote, "by the publication of research . . . that uses flawed reasoning to draw unfounded conclusions about topics as important as the presidential election." The critics pointed out that a specific brain area, such as the amygdala, can be involved in the production of a wide variety of emotions, from fear to pleasure. This makes it extremely difficult, if not impossible, to decipher the hidden feelings of people based on brain scans alone.

This 2008 article from Jonah Lehrer, one of my favorite science journalists, gives a nice commentary on the problems associated with fMRI. For you non-science geeks, this is a technique used to ostensibly take pictures of your brain while you do stuff, like make decisions or view images. It sounds cool, right? Well, that may also be the problem.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang
Tue, 23 Nov 2010 13:46:00 -0800 [article] The Shadow Scholar - The Chronicle Review http://jeremyywang.com/article-the-shadow-scholar-the-chronicle-revi http://jeremyywang.com/article-the-shadow-scholar-the-chronicle-revi
Media_httpchronicleco_tnzim

I experienced a range of emotions while reading this article...

(1) Anger at the people that do this for a living, on both sides. I work hard to write papers and am proud of my work. The fact that others are getting away with this and earning degrees potentially takes away from the credibility of my work and others that are actually trying to be scholars.

(2) Sympathy for Ed Dante, the pseudonym for the shadow writer in this story. Like him, I've burned the candle at both ends to get a paper done on a deadline, fueled by caffeine and sleep-deprivation-induced "almost-out-of-body experience."

(3) Sadness at the sheer number of people, particularly in nursing and education, that get away with this. At the very least, this warrants that all educators re-evaluate how they evaluate students.

(4) Respect for my international graduate student colleagues that don't succumb to the pressure by cheating, and instead learn to write at an advanced level in their non-native language. They have to work harder than I do to produce the same level of work here in the US, and I have a lot of respect for that.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/30770/SouthParkSuit.png http://posterous.com/users/KPcWHxRVlf Jeremy Wang jeremyywang Jeremy Wang