Five years ago, information designer Mahima Pushkarna joined Google to make data easier to understand.as Senior Interaction Designer Human + AI research (Pair) Team, she designed data card Help everyone better understand the context of the data they are using. The Data Cards Playbook is Google’s AI Principles Put it into action by providing feedback, relevant clarifications, and an opportunity to appeal.
Recently, Mahima paper on Data Cards (co-authored with Googlers Andrew Zaldivar and Oddur Kjartansson), the ACM Conference on Fairness, Accountability, and Transparency (ACM FACCT). Let’s catch up with her and find out more about why she ended up on her Google.
How did your background lead to your current job?
I’ve always been fascinated by remembering solutions to things. The kinds of questions that I found meaningful are those that are never truly answered or that have no single correct answer. (A question that pisses us off!) These are the questions that I always gravitate towards.
Early in my career, I realized the power of visualizing data, but spreadsheets were intimidating. I wondered if the design could easily convey complexity. So I found myself studying information design and data visualization in graduate school in Boston. I focused on how people experience data and how our interactions and our context are mediated.
With no background in artificial intelligence or machine learning, I first joined Google Brain as a full-time visual designer. This created room to explore human-AI interactions and make his AI more accessible to a wider class of developers. My work at PAIR is focused on making the information experience more meaningful for developers, researchers, and others building AI technologies.
What is it like to have a unique background as a designer in a technical AI research team?
When you’re an engineer and you’re immersed in building technology, it’s easy to assume that everyone has a similar experience to yours. Especially when you’re surrounded by peers who share your expertise. The actual user experience is highly personal and highly dependent on the user and context. That particular clarity is what the designers bring to the table.
I was able to get my engineers and research colleagues to ask simple, person-centered questions at first. How are people using AI tools? What are they learning from it? Who else might be involved in the conversation? Do they have the capabilities we assume?
Quote quote: “Identifying what you don’t know about your data is just as important as clarifying what you do know.”
What inspired you to start designing data cards?
This project started while I was working on another visualization toolkit. facet, convey skewnesses and imbalances in datasets to help machine learning practitioners make informed decisions. At the time, transparency was a moving goal. Andrew, Tulsee Doshi, and I began to think proactively about data fairness and realized that there were significant gaps in documenting the human decisions that dot the lifecycle of datasets.
This “invisible” information shapes how the data is used and the results of models trained on it. For example, a model trained on a dataset that captures ages with 2 or 3 buckets will give very different results compared to a dataset with 10 buckets. The goal of data cards is to make both visible and invisible information about datasets available and understandable. This enables people with different backgrounds to make informed decisions.
as we cover FACCT paper, Andrew and Oddur, and I have come to two insights. First, identifying what you don’t know about your data is just as important as clarifying what you do know. Capturing these nuances helps fill knowledge gaps before data is collected. The second surprise was the sheer number of people involved in the dataset life cycle and the vulnerability of knowledge. Context is easily lost in documents, emails, people, and translations over time, both between and within teams.
Datacards stand on the shoulders of giants data sheet (Gebble et al.) and model card (Mitchell et al.). We are very fortunate to have the support of many original authors on these important papers that paved the way for FAccT.
How would you like this paper to be used across the tech industry?
Imagine a world where finding testable information about the motivations of dataset creators and model performance is as easy as learning about the ethical beliefs of celebrities or movie ratings. Our vision for data cards is for them to become a mainstay of our culture. In other words, although it is invisible, machine learning practitioners will miss the lack of data cards.
This white paper introduces a framework that other teams can use to do their work.In addition to that, we have open sourced Data card playbookso we try to lower the barrier of access in every possible way.
