Model Cards: A Nutrition Label for Predictive Models

Background

AI is a prevalent part of our day-to-day lives. Whether we are checking our e-mails, going to the doctor, receiving a diagnosis, signing up for insurance, applying for loans to buy a house, or being identified as traffic violators for speeding, these are all areas that AI, and more specifically machine learning, is being applied to.

Anyone using basic applications like e-mail will be impacted by AI (e.g. spam filters, what is classified as priority or not, etc.). [Image from Unsplash]

In addition to the integration of AI into our daily lives and its continued commercial applications, we are starting to see increasing cases of unethical use or applications of AI that negatively impact specific groups of people. This can be based on gender, ethnicity, political belief, socioeconomic status, or any number of factors that are biased and should not be incorporated into the decision-making process, much less systematized at scale through machine learning.

One of the most prominent areas we are seeing this in is law enforcement and the use of computer vision for facial recognition, where algorithms are producing incorrect results biased against minorities, especially African Americans. Some examples include:

• The ACLU did a study of Amazon’s Rekognition services, which struggled to correctly identify people of color and women.

• The incorrect identification of Robert Williams as a criminal and his subsequent arrest in Detroit.

• (non-law enforcement) Google’s image recognition algorithm identified an African American as a gorilla.

Now, this is a multidimensional and complex problem with many failure points along the way. In addition to the entire process around predictive models (e.g. hypothesis forming, data input, testing, etc.), models themselves are also often described as ‘black boxes’ and the decision-making process is opaque.

Not everything can be covered in this article, but I hope to shed some light on one of the many methods starting to be adopted in organizations that are oriented around responsible AI.

Model Cards: The Nutrition Label

The model cards concept was founded by Dr. Timnit Gebru, one of the most renowned AI ethics researchers that was controversially fired from Google after writing a paper that exposes the inherent dangers of large language models like the ones companies like Google uses. Model cards were first introduced in one of Dr. Gebru’s papers, and as a result, there is increasing adoption of either better documentation or model cards across many companies and in academia. They are also still publicized on Google’s website.

Original paper for model card proposal. Source: arxiv

Model cards, like nutrition labels, provide information on the contents (and more) of a model. This means that anyone at a given company or other organization should be able to go into a given model’s documentation, read its corresponding model card, and quickly understand the problem statement, requirements, hypothesis formulation, data inputted (including any identified biases in the data, especially with demographic or other sensitive fields), implementation of predictive modeling, testing, remaining gaps, etc. Examples of model cards can be found here from different companies.

Sample model card from Salesforce GitHub repository. Source here.

While model cards should include technical details, they should also include enough language and information that allows any business or other user as well to quickly understand the intent around building the model, as well as any gaps or ethical considerations as in the image above. Many also contain data charts that show the breakdown of total records in a training/test set, in addition to any class imbalance or skewing of the data for sensitive fields like gender, etc. This promotes transparency and understanding of AI to both internal and external (if you publish them) users.

Implementation

For organizations just getting started with AI and machine learning, model cards are a great way to begin establishing AI ethics best practices. Again, they do not solve everything, but they are a good way to start and have a low barrier to entry.

To implement them, there are many model card toolkits, examples, and frameworks out there to follow. They can be done programmatically in a couple of different programming languages (e.g. Python, Tensorflow), or they can be done as part of the product/algorithm development process.

More mature organizations that may have many machine learning models and associated model cards have a central repository and way to quickly update this documentation and find relevant information.

Challenges

Despite their low barrier to entry, there are some common challenges to implementing the practice of using model cards, including but not limited to:

• Since many organizations may have challenges with data silos, implementing a common practice such as this can be difficult. Therefore, I do see them being pushed out either through a data ethics office (if the company has one) or from some other central part of the organization.

• Lack of leadership support will also result in inconsistent adoption at a given organization. AI ethics, transparency, and trust must be important to key members of a business for it to be a core part of a company’s culture and policies.

Future Opportunities

Once there is a common practice and foundation of using model cards to drive AI ethics and transparency in a company or other institution, there are a lot of areas of opportunity to build upon it. Some examples include:

• Incorporate into workflows - With Robotic Process Automation (RPA), Boomi, and many other integration and workflow automation tools, it will become easier to incorporate the creation of model cards into a given process.

• Adoption through public policy - As more public institutes, professional associations, and regulatory agencies become involved in the development and management of AI, they will emphasize adoption of better documentation practices to promote ethical AI. Model cards or similar may be part of that, and adopting them now will help organizations get a head start.

• New developments in explainability - As algorithms that help audit and increase the explainability of deep learning models are created (e.g. counterfactuals, causal inference), these can and should be incorporated into future model cards/documentation processes to ensure as much transparency as possible.

In summary, ethical issues in the application of AI to various use cases is an increasing and complex problem as the technology becomes more commercialized; however, model cards provide an easy way for AI developers and owners to begin providing transparency and awareness to others on potential issues or gaps. More needs to be done on this front, but given how easy it is to start publishing these with any relevant model(s), I recommend anyone involved in this field to incorporate model cards into the development process.

Reference

ACLU. "Amazon’s Face Recognition Falsely Matched 28 Members of Congress With Mugshots.” 2018. Link.

MIT Technology Review. “There is a crisis of face recognition and policing in the US.” 2020. Link.

The Verge. “Google ‘fixed’ its racist algorithm by removing gorillas from its image-labeling tech.” 2018. Link.

Stanford. Timnit Gebru profile. Link.

The Verge. “Timnit Gebru was fired from Google — then the harassers arrived.” 2021. Link.

Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. “Model Cards for Model Reporting.” 2019. Link.

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmar- garet Shmitchell. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 2021. Link.

Google. Model Card page. Link.

GitHub. Model Cards and Datasheets. Link.

GitHub. Salesforce Model Card Master. Link.