Useful News for You

Neurologist Andrew Budson and neuroscientist Elizabeth Kensinger not only explain how memory works, but also share science-based tips on how to keep it sharp as we age in their new book, “Why We Forget and How to Remember Better: The Science Behind Memory.” The book came out Wednesday. The Gazette interviewed Budson, M.D. ’93, and Kensinger ’98 about the neuroscience of memory and tips for improving our recall. This interview has been condensed and edited for length and clarity.

Q&A

Andrew Budson and Elizabeth Kensinger

GAZETTE: What are the most common misconceptions about memory?

KENSINGER: One of the most common errors is in the metaphors that we use to talk about memory that imply that there is a memory that sits somewhere in the brain, like a file that we can retrieve without effort. Memory is an active and effortful process. Every time that we’re bringing a past event to mind, we have to use effort to rebuild that memory. A second, related misconception is that there is such a thing as photographic memory, which is this ability to effortlessly remember everything that you just saw. It might be that it feels to us that we remember random things that we weren’t trying to remember, but there are reasons why we remember them; we were enjoying a song that we were listening to, or we were thinking about how bizarre something was, and those feelings or thoughts allowed that content to get into memory.

The third thing is that many people think that forgetting is bad and that an optimal memory system is one where forgetting doesn’t occur. Forgetting is important because if every time that we were trying to make a prediction about the future or understand what is going on right now, we had to sift through everything that’s ever happened to us, it would be inefficient. There’s tremendous utility in pruning because it allows us to use the pieces of our past that are most likely to be relevant for understanding what’s going on right now or what might happen tomorrow or next year.

GAZETTE: Why do we forget things?

KENSINGER: At the most basic level, we want to think about memory as having three different phases that must happen for us to have access to past content. The first is to get the information into memory, a process that is referred to as encoding. Then, you must keep that information around, and this is called storage or consolidation. It’s akin to pressing the save button on the document that you’ve just created on your computer, but unlike that analogy with a computer, you must continually re-store that content in the brain. And then finally, you must be able to bring that information to mind in the moment that you need it. Memory failures can reflect errors at any of those different stages. One of the most common times when errors arise is in that initial encoding phase, where often what happens is that we’re just not devoting enough effort or paying enough attention.

GAZETTE: How can we make sure we remember things we need to remember?

KENSINGER: Throughout the book we use the mnemonic device of FOUR, which stands for four critical things that we must do to get information encoded into memory. First, you must Focus attention; second, you must Organize the information, then you must Understand the information, and lastly, you need to Relate it to something else that your brain already knows. It’s much easier said than done. Often, when someone says, “I went to a party, and I met all these people, and I don’t remember any of their names,” the breakdown was at that first stage, not paying enough attention. At the moment of retrieval, we can also have failures. Any student has experienced this, where they know the content, but during an exam, they’re not able to come up with it. Or you’re looking at someone’s face, you know this person’s name, but right there in that moment, you’re not able to retrieve it. In those moments, you want to avoid the urge to generate possible answers and instead use general retrieval cues such as thinking about the last time you saw that person, the context, and the possible connections.

GAZETTE: How can sleep, or lack thereof, affect our memory?

KENSINGER: When we’re talking about storing the information so that we have longer-term access to it, getting enough sleep is one of the most important things that we can do. Sleep helps information to move from being briefly accessible to being stored in long-term ways, and it guides the transition from something we remember from a specific event, like remembering when the teacher said the boiling point of water was 212 degrees Fahrenheit, to a fact we just know.

BUDSON: Sleep is important to consolidate memories so that they can be retrieved later, but sleep is also at least theorized to help us flush away the amyloid beta protein, at night. This protein is thought to trigger Alzheimer’s disease dementia. It is still an active area of research, but there is some good evidence that when we sleep, our brain cells and the synapses shrink a little bit, and it allows us to flush away this protein that accumulates during the day. No one knows exactly what the normal function of the amyloid protein is, but some people believe, and I’m one of them, that it is an important part of the brain’s immune system that defends it against foreign invaders like bacteria, viruses, parasites, and fungi. Now, in addition to sleep, we know that to keep our brains healthy — and for good health in general — we need to eat right, engage in regular aerobic exercise, keep a healthy body weight, and be socially active.

GAZETTE: Sudoku or crossword puzzles? Which one helps keep our brain healthy and memory strong?

BUDSON: The short answer to that is that when you do computerized brain games or Sudoku, you get better at brain games and Sudoku, but that doesn’t generally translate into overall brain function. Having said that, we know that when one engages in novel cognitively stimulating activities, there have been shown to be benefits. One study that came out recently looked at crossword puzzles compared to computerized brain training games, and they found that the people who did crossword puzzles did better. Crosswords may be something that’s beneficial because they are always a bit different, and they require you to think about words and your knowledge in different and novel ways. But having healthy social interactions has been shown to be important. Our brains did not evolve to do crossword puzzles or computer games; they evolved in large part for social interactions. This is one of the reasons why staying socially active is so important. In summary, you must eat right, exercise, keep yourself cognitively stimulated, stay socially active, and sleep.

GAZETTE: How does memory change with aging?

KENSINGER: With aging that is not Alzheimer’s disease or other pathological aging, memory annoyances are very common. We forget things like proper nouns; we can’t think of someone’s name; we can’t think of the title of the book that we read last week. We also are more prone to forgetting some of the specifics, because with aging, there’s a transition toward the brain prioritizing the gist of what happened. The brain embraces the similarities across events rather than trying to hold on to each individualized event. That can lead to a lot of memory frustrations, and it also can make us prone to some types of memory distortions or false memories where we think something happened, but it was something slightly different. It’s also important to point out that there are some upsides to this transition. It is thought that this tendency for the brain to recognize the similarities across events may be one of the contributors to what we think of as wisdom that comes with old age.

GAZETTE: Of the many tips to improve memory you share in your book, which has been the most helpful to you?

BUDSON: The first thing I would say is that there’s nothing wrong with outsourcing your memory or using memory aids. Anybody who wants to remember a shopping list or an appointment that’s coming up, write it down, put it in your phone or a planner; use reminders and calendars.

I offload my memory as much as possible. I have all my passwords written down in a secure digital place. I use calendars, planners, and lists. In terms of trying to remember things better, day to day, I work at trying to be present and pay attention to what I’m doing and trying to multitask less. When I park my car in a parking garage, I’m trying to be conscious of exactly what I’m doing. If I’m going for a run in an unfamiliar location, I’m really going to pay attention. I also work very hard to turn things into habits and routines as much as possible so that it’ll just become automatic, a habit.

KENSINGER: I really like the FOUR mnemonic we came up with. That has helped me to think about all those different steps I need to take each time something is important for me to remember. I think the small mnemonics that we create in the moment are also helpful. For example, for passwords, I create a sentence so an alphanumeric code makes sense to me, and I can remember it over long periods of time. Those sorts of mnemonics that we’re generating ourselves are powerful because they demand that you do those four things, and especially that you invest effort in creating the memory. We’ve all had that frustrating moment where we don’t know where we left our phones. For me, that is often an encoding failure because I set my phone down somewhere when I wasn’t paying any attention. Now when I’m putting my phone down, I say it out loud: I’m putting my phone on the counter. It’s a very simple strategy, but because it’s simple, I remember to do it. You must focus your attention on those early actions to save yourself from those annoyances of forgetting later.

And lastly, one of the most important pieces of advice for students who are studying for a test is this: Do not cram. When you’re staying up to study before the exam, you don’t sleep. As we already talked about, sleep is important for consolidating memories. And, if you realize you don’t understand something as you’re pulling an all-nighter, there’s often not enough time to build that understanding. Also, you want to be studying information in lots of different ways and in lots of different contexts. You don’t just want to be studying in your dorm room or always at the same seat at the library; you want to study in lots of different places at lots of different times of day because all of that is going to help you remember the information. It is because of that variability, that need for sleep, and the time that it can take to reach understanding that it is important that students start their preparation early and keep it going ideally throughout the semester rather than cramming right before a big test.

Sean Carroll bought a green screen and a camera during the first pandemic lockdowns of early 2020. Each week, the theoretical physicist and acclaimed science writer shot two hour-long videos explaining fundamental ideas in modern physics, including conservation, quantum mechanics, gravity, and black holes.

Carroll is turning that series, “The Biggest Ideas in the Universe,” into a namesake book trilogy. This Friday he will give an in-person talk at the Harvard Book Store about volume one, “The Biggest Ideas in the Universe: Space, Time, and Motion.” Both video and book series are conceived as mash-ups of popular science romp and college-level textbook and aimed at an oft-neglected reader: one not so interested they’d want a degree but enough to crave a detailed grasp of how it all really works.

“Secretly, the real audience for this book is my 16-year-old self. I would have loved to have this,” said Carroll, who received his Ph.D. in astronomy from Harvard in 1993. Today, he is the Homewood Professor of Natural Philosophy at Johns Hopkins University. The Gazette spoke with Carroll about his project and why he dreams of a world where people can debate their favorite dark-matter candidates — and other physics mysteries — at a local pub.

Q&A

Sean Carroll

GAZETTE: You said you started your video series to help keep people connected during the surreal early days of the pandemic. Did it accomplish what you hoped?

CARROLL: Yes, they kept people connected, and the pandemic was a great pleasure. No, I was definitely a little bit less ambitious than that. Maybe it did more good for me than for the rest of the world. But the reception was pretty good. The most popular video has something like 800,000 views. And it’s an hour and a half of me writing down tensor equations for general relativity. It’s not the usual viral hit on YouTube.

GAZETTE: So then, why turn the videos into a book trilogy? Why aren’t the videos enough?

CARROLL: Like I said, 800,000 people watched the most popular video. Let me tell you, it’s not 800,000 people who are buying the book. But that doesn’t mean that the people who do buy the book, even though it’s a smaller number, have watched the videos. Some people just like reading books. And honestly, you can learn more from the books because it’s a slower process.

GAZETTE: It also seems like you’ve invented a whole new genre, something between a popular science book and a textbook. Was that intentional?

CARROLL: That was absolutely intentional. I’ve written a few popular books. But for the most part, my goal is not just to be pedagogical. My books make an argument that you can disagree with, whether it’s about the nature of reality or the interpretation of quantum mechanics. Whereas these books try very hard to stick to things that everyone agrees with. There’s a vast audience in between the ones who read popular books and the ones who read textbooks. That’s a gap I’m happy to try to fill.

GAZETTE: Throughout the book, you present a lot of abstract concepts, like infinity, that are challenging for most people to conceptualize, even if they’re important to questions about our natural world. Why are these concepts still valuable to explore?

CARROLL  I’m not just trying to explain the bare bones of the physics. I brought in little historical stories, philosophical background, mathematical niceties, and so forth. In the back of my mind, I’m always thinking like, “What is it that got me really interested?” With infinity, we use it in the continuum and our best descriptions of the natural world without really being sure whether it’s necessary. But I honestly don’t know whether we’re on the right track. So, even though I tried to stick with things that everyone agrees on, I’m happy to point out places where we don’t agree.

GAZETTE: Are there any physics concepts that are commonly misunderstood?

CARROLL: Oh, my goodness, there are so many. Within the scope of this book, I talked about space and time and space time. Classical mechanics up through general relativity. There are obviously a lot of misimpressions about general relativity, curved space time, the expanding universe, and things like that. But I don’t dwell on that. The tiny bit I’ve learned from academic research into misinformation and learning is that if you try to teach something by saying, “Here’s the wrong thing that you might think is true. And now I’m going to fix it,” people end up thinking the wrong thing because you put the idea in their heads. I try to spend most of my time thinking about the correct conceptions.

GAZETTE: For this first book, you travel all the way from Aristotle and Newton to modern research on black holes. What’s next?

CARROLL: I’m working on book two as we speak, and that’s all about quantum mechanics and quantum field theory. The third book will be complexity and emergence. Some of that is just a rubric that covers thermodynamics and cosmology, but the ideas of complex systems, criticality, and networks are hugely important to identify the ways in which the microscopic rules of the universe build up together to make this crazy complex macroscopic world we live in.

GAZETTE: You’ve said you’d love to hear people discussing physics at the local pub or in line at the grocery store. Apart from making you and other theoretical physicists happy, what might that accomplish?

CARROLL: I’m not an expert on science education. But from what little I do understand, we focus way too much on a set of facts. Science is not just a set of final answers; it’s a way of thinking about the world and learning new things. So, if people understood how and why we got to specific ideas, they would, I think, have a better idea of how to keep uncertainty in mind, judge new evidence, think about the world in a quantitative way, and be open to changing their minds. All those things are crucially important to science and certainly serve people more broadly.

GAZETTE: Which physics topic would you be most excited to hear people chatting about at the pub?

CARROLL: The cheeky response would be Hamiltonian mechanics. In the book, we learn Newtonian mechanics, then a whole other way of doing it, called Hamiltonian mechanics, and then another way, Lagrangian mechanics. Three different ways to do the same underlying physics. Is one of them more right? We don’t know. But it’s important to conceptualize the same ideas from different angles because one might suggest a different way forward. We’re not done yet. We’re still thinking about what happens next. So, in this book, I’m trying to provide both the ideas that are going to be used for the next 500 years and what to look out for as we move forward.

While each person gets dressed at least 29,000 times in the course of their lives, empirical science has paid little attention to why we select the everyday clothes that help mold our image.

In one of the first studies of its type, researchers at Massachusetts General Hospital and London College of Fashion, University of the Arts­, have explored the aesthetics of fashion to uncover preferences like style (shape and cut), color (hue, brightness, and saturation), and personality differences that motivate people to buy and wear clothing for work, leisure, and more formal occasions. The study was published in Empirical Studies of the Arts.

“Preference has been studied for ages across a wide range of art and aesthetic domains, from paintings to music, but almost never in the context of fashion, which is unquestionably a social experience and mode of self-expression,” says co-author Nancy Etcoff, director of the Program in Aesthetics and Well Being at MGH and Harvard Medical School.

“We asked in our research whether fashion aesthetics could be studied empirically, and not only discovered it can, but that it opens the door to better understanding factors that guide people in their everyday clothing preferences. This knowledge is valuable to designers and marketers of clothing as well as to consumers themselves by putting them in better touch with their aesthetic tastes and sensibilities.”

From the research team’s online survey of 307 females and 191 males in the United Kingdom, a novel preference structure emerged across four types of clothing styles: essential, comfortable, feminine, and trendy.

Further analysis found that preferences across each of these styles were associated with the color preferences and the self-reported traits (e.g., personality) of study participants.

More specifically, prominent color preferences for those who liked and owned feminine clothing (e.g., dresses and skirts) consisted of lilac, violet, pink, turquoise, and dark red, and for people who wore essential clothing (e.g., shirts and jackets) dark blue, blue, and brown were the favored colors.

In addition, the study showed that people with a propensity for feminine clothing displayed high levels of fashion leadership, an appreciation for the importance of dressing well and, on the behavioral scale, tended to show more compassion.

On the other hand, individuals who liked and owned essential clothing tended to be sociable, exhibited higher energy, and were emotionally stable.

Wearers of comfortable clothes (e.g., hoodies, sweatpants, tracksuits) were also identified with fashion leadership and interest, while those whose taste ran to trendy articles (e.g., dungarees, polo shirts, boiler suits) tended to be young and had an appreciation for the visual arts.

“The more one knows about and appreciates the aesthetics of fashion, the more attention they will pay to what they’re wearing and what excites them when viewing clothing,” says lead author Young-Jin Hur, with London College of Fashion/UAL.

“Because aesthetic experiences seem linked with well-being, our findings may provide an important commentary on how this could impact the wearer’s self-confidence.”

Adds senior author Emmanuel Sirimal Silva with LCF/UAL, “Our findings underscore that clothing preferences are closely linked to fashion experience and to the individual’s identity.”

Etcoff, who authored the book “Survival of the Prettiest: the Science of Beauty,” believes her team’s latest research opens the door to an exciting new body of discovery around fashion behaviors and preferences, such as clothing textures and patterns as well as the impulses that drive constant change in the multi-billion-dollar clothing field.

“Each of us spends a lot of money on clothing,” she observes, “and what we’re attempting to do through our work is more fully inform the decisions we make to look better and to complement our personalities.”

Data collection for this study was funded by Fashion Business Research at LCF/UAL’s Fashion Business School.

Few would argue right now that physics doesn’t matter, barely a month after scientists at Lawrence Livermore National Laboratory achieved fusion ignition, a breakthrough step toward unlocking a new source of abundant, clean energy. Australian physicist Suzie Sheehy wants to go further, however, making the experimental side of the science accessible and reconnecting us with forgotten pioneers who helped change the ways we understand the world.

She discussed her debut book, “The Matter of Everything: How Curiosity, Physics, and Improbable Experiments Changed the World,” with Greg Kestin, Ph.D. ’14, associate director of science education and a lecturer on physics, in an online event last Wednesday presented by the Division of Science and Harvard Library with Harvard Book Store. Sheehy offered a rapid-fire overview of the history of the science, along with an introduction to some unsung heroes of the field and some peeks into where it may be heading next.

Sheehy, who oversees research groups at the universities of Oxford and Melbourne and currently focuses on medical applications, laid out five basic points. First, she said, “How we know is just as important as what we know.”

For that reason, “I celebrate experiments,” said Sheehy, whose book is organized around 12 key experiments from the last 120 years. Acknowledging that theoretical physics, practiced by such luminaries as Albert Einstein, may be better known, she described her experimental colleagues as having “a more nuanced job,” requiring “good questions, persistence, and a whole lot of luck.” As an example, she recalled the 1897 experiment into cathode rays that resulted in the discovery of electrons and “gave birth to the entire electronics industry.” Without that, she noted, “rock ’n’ roll would never have happened.”

Her second point — “The results in curiosity-driven research grow in usefulness over time” — was reflected in the 1896 discovery of the X-ray. Not only did it allow doctors to look beneath a patient’s skin, it also gave photographers a new artistic tool and has become crucial to airport security. “New discoveries make new imaginings possible,” she said.

“Science may be objective, but scientists are not,” was her next point. Even great physicists have blind spots, she noted, quoting physicist Albert Michelson, who said in 1894, “It seems probable that most of the grand underlying principles have been firmly established.” This was before the discovery of X-rays, radioactivity, and the electron — and before quantum mechanics would completely upend the field. Sheehy quipped, “It’s hard to predict the future.”

Following up on scientists’ very human failings, Sheehy delivered her fourth point in the form of a question: “Who gets to be a physicist?”

“Curiosity is a human trait,” she said. “It’s not racist or sexist, but we’ve been restricting this field.” To counter the “strong white man” narrative too often championed in her field, Sheehy briefly introduced some of the female physicists featured in her book. These include Harriet Brooks, who helped decipher how radioactive elements change, as well as Marietta Blau, whose work led to a new kind of particle detector, and Bibha Chowdhuri, an Indian particle physicist who researched cosmic rays.

Ultimately, “collaboration is the human force of nature,” said Sheehy, making her final point. Citing “the power of collaboration,” she pointed out the great strides being made at CERN, the European Organization for Nuclear Research. The organization, which has 23 member states, was designed to foster such collaboration — and invented the World Wide Web in order to do so. Currently, the Swiss-based main lab not only brings international teams together, it houses the Large Hadron Collider, allowing for the kind of experiments very few, if any, of the member countries would be able to afford on their own.

Following up Sheehy’s presentation with a discussion that included questions from audience members, Kestin asked about the future of the field. Sheehy reflected back on Michelson’s remark of more than a century ago, and how easy it is to assume that we’re reaching the end of human knowledge. “It feels like we’re done with physics, and yet we know there’s more,” she said. In particular, she pointed out that even the most recent discoveries, such as those into the nature of subatomic particles like muons, only account for roughly 4 percent of all matter. Much of what else makes up our universe, known as dark matter, remains a mystery.

“It’s exciting to think that over 90 percent of matter is not understood,” said Kestin.

Results from a new study by Harvard researchers just published in Cell offer insights into the relationship between inflammation and the cognitive impairment we experience as we age, and suggest the possibility that it may be a result of a kind of cellular chain reaction.

“Understanding aging is one of the most important goals in biomedicine,” said Xiaowei Zhuang, David B. Arnold Jr. Professor of Science in the department of chemistry and chemical biology, professor of physics,  Howard Hughes Medical Institute (HHMI) investigator, and one of the paper’s authors. “It is also a very challenging problem. One reason is because the brain is very complex. It contains an exceptionally high diversity of cells, with many different types of neurons and non-neuronal cells forming intricate interaction networks.”

To study such an intricate system, researchers used an imaging method known as MERFISH, developed by the Zhuang lab, which has extensive expertise in inventing novel imaging methods and applying them to study biological systems. MERFISH (for Multiplexed Error Robust Fluorescence In Situ Hybridization) is capable of simultaneously measuring not only thousands of types of RNA, or thousands of genes, in cells, it also reveals the spatial relationships between them.

MERFISH allowed researchers to generate “atlases of gene expression,” looking at “neighborhood relationships” between cells, explained Catherine Dulac, Samuel W. Morris University Professor in the department of Molecular and Cellular Biology, HHMI investigator,  and another of the paper’s authors. With MERFISH, said Dulac, “one can look not only at changes in gene expression across different ages, but also changes in gene expression, in particular cell types, related to their spatial relationship.”

Projections created internally by ExxonMobil starting in the late 1970s on the impact of fossil fuels on climate change were very accurate, even surpassing those of some academic and governmental scientists, according to an analysis published Thursday in Science by a team of Harvard-led researchers. Despite those forecasts, team leaders say, the multinational energy giant continued to sow doubt about the gathering crisis.

In “Assessing ExxonMobil’s Global Warming Projections,” researchers from Harvard and the Potsdam Institute for Climate Impact Research show for the first time the accuracy of previously unreported forecasts created by company scientists from 1977 through 2003. The Harvard team discovered that Exxon researchers created a series of remarkably reliable models and analyses projecting global warming from carbon dioxide emissions over the coming decades. Specifically, Exxon projected that fossil fuel emissions would lead to 0.20 degrees Celsius of global warming per decade, with a margin of error of 0.04 degrees — a trend that has been proven largely accurate.

“This paper is the first ever systematic assessment of a fossil fuel company’s climate projections, the first time we’ve been able to put a number on what they knew,” said Geoffrey Supran, lead author and former research fellow in the History of Science at Harvard. “What we found is that between 1977 and 2003, excellent scientists within Exxon modeled and predicted global warming with, frankly, shocking skill and accuracy only for the company to then spend the next couple of decades denying that very climate science.”

“We thought this was a unique opportunity to understand what Exxon knew about this issue and what level of scientific understanding they had at the time,” added co-author Naomi Oreskes, Henry Charles Lea Professor of the History of Science whose work looks at the causes and effects of climate change denial. “We found that not only were their forecasts extremely skillful, but they were also often more skillful than forecasts made by independent academic and government scientists at the exact same time.”

Allegations that oil company executives sought to mislead the public about the industry’s role in climate change have drawn increasing scrutiny in recent years, including lawsuits by several states and cities and a recent high profile U.S. House committee investigation.

Harvard’s scientists used established Intergovernmental Panel on Climate Change (IPCC) statistical techniques to test the performance of Exxon’s models. They found that, depending on the metric used, 63-83 percent of the global warming projections reported by Exxon scientists were consistent with actual temperatures over time. Moreover, the corporation’s own projections had an average “skill score” of 72 percent, plus or minus 6 percent, with the highest scoring 99 percent. A skill score relates to how well a forecast compares to what happens in real life. For comparison, NASA scientist James Hansen’s global warming predictions presented to the U.S. Congress in 1988 had scores from 38 to 66 percent.

The researchers report that Exxon scientists correctly dismissed the possibility of a coming ice age, accurately predicted that human-caused global warming would first be detectable in the year 2000, plus or minus five years, and reasonably estimated how much CO₂ would lead to dangerous warming.

The current debate about when Exxon knew about the impact on climate change carbon emissions began in 2015 following news reports of internal company documents describing the multinational’s early knowledge of climate science.  Exxon disagreed with the reports, even providing a link to internal studies and memos from their own scientists and suggesting that interested parties should read them and make up their own minds.

“That’s exactly what we did,” said Supran, who is now at the University of Miami. Together, he and Oreskes spent a year researching those documents and in 2017 published a series of three papers analyzing Exxon’s 40-year history of climate communications. They were able to show there was a systematic discrepancy between what Exxon was saying internally and in academic circles versus what they were telling the public. “That led us to conclude that they had quantifiably misled the public, by essentially contributing quietly to climate science and yet loudly promoting doubt about that science,” said Supran.

What can take the place of real-life, tactile experience when that would mean allowing whole classes of students to paw through an illustrated manuscript nearly 250 years old?

The International Image Interoperability Framework, or IIIF.

Just ask art history Professor Jinah Kim, who last semester took students in her “Painting of India” class to Houghton Library to view a Persian text from 1778. The nearly 500-page book is filled with vibrant full-color images and handwritten adventure stories.

Students couldn’t examine the entire manuscript during class: There was too little time, and it would have been hard on the document. Instead, they were able to engage more deeply with it online, flipping through its pages and zooming in so close on illustrations they could make out brush strokes. Kim told them to “play” with the manuscript, compare its pages or place it side by side with similar works. They could add their own annotations on top of the illustrations and share their analyses with Kim in a shared workspace.

Engaging with images has come a long way since Kim, the George P. Bickford Professor of Indian and South Asian Art, was a student herself. At that time, more than 20 years ago, students could only view images of the art they were studying printed in books or projected as slides in a classroom.

Today, Kim said, IIIF gives students as-good-as, or better-than, in-person examination of the works they’re studying. IIIF is an open-source technological framework that has become the universal standard for online, high-resolution viewing of cultural heritage images like paintings or manuscripts.

Developed in 2011, IIIF was originally intended for viewing digitized medieval manuscripts, Harvard Associate University Librarian and Managing Director of Library Technology Services Stuart Snydman explained. But it was soon clear that its applications went far beyond this one use.

Since its inception, the database of cultural heritage images available for free online with IIIF capability has continued to grow. In 2022, the IIIF community estimated that between all their participating cultural heritage institutions, they’ve made available more than 1 billion items available.

“With IIIF, we’re investing in the cultural heritage image community,” Snydman said. “Our goal is global, universal, as open as possible. It’s not just about Harvard’s images; it’s about enabling students and faculty to interact in the very same way with images at Oxford, the Library of Congress, or the Vatican that they do with images held at Harvard. The code word for this is interoperability.”

Of the 1 billion IIIF-compatible items, about 6 million are held in Harvard’s library collections. Everything from 500-year-old maps to modern photographs are viewable in high resolution by anyone with an internet connection. Emily Dickinson’s pencil strokes can be magnified and examined, and Persian manuscripts like the one studied by Kim’s class can be compared with illustrations from the same region and period held at the Library of Congress.

Rashmi Singhal, at HUIT Academic Technology, has been working on IIIF at Harvard for 10 years. She was hired in 2013 as a software engineer to build the 2.0 version of Mirador, the IIIF image-viewing application Harvard uses, and to make Harvard’s existing digital collections IIIF-compatible.

Her unit and Harvard Library Technology Services worked closely with Harvard faculty and with staff at the Harvard Art Museums to build out Harvard’s IIIF compatibility. Library chief of staff Franziska Frey said the coalition around IIIF was collaborative across the University.

“From the beginning, we were adamant about it being a true collaboration,” said Frey.

With continued cooperation, Harvard has built out its IIIF capabilities over the last decade and created several applications for scholars to make the most of IIIF.

Singhal said she sees some researchers using the Mirador image viewer’s “deep zoom” feature to transcribe difficult-to-read handwritten manuscripts, while others use its “workspace” feature to pull together pages of the same manuscript held at different institutions to examine them at once.

“You’re not going to be able to see with the same level of detail in person as you can with deep zoom,” she said, “and if a manuscript’s pages are held separately, you can’t ‘page through’ it physically, but you can digitally. IIIF has really revolutionized scholarly examination of some of these materials.”

IIIF also enables interactive virtual exhibitions and collaborative online research projects, like Kim’s “Mapping Color in History.” A key part of the project, which is a searchable database of pigment analysis in Asian paintings, was possible because of her ability to annotate on IIIF images, Kim said.

“With digital access and IIIF, you can see an image, manipulate it, and even add your own things to it,” she said. “It really opens a door for interesting possibilities to engage with images.”

While IIIF-savvy scholars like Kim think creatively about future uses for IIIF, those on the technical side also appreciate its implications in terms of educational access.

“The fact that IIIF has been able to become a universal standard, and that it’s all open-source — that has exciting implications for democratized learning,” said Snydman. “Students and scholars of all ages have the opportunity to learn with images — not just in a physical classroom or library, not just during certain hours, and not just on Harvard’s campus. This is a great example of how technology can be used to minimize inequalities in education and give open access to knowledge.”