Historically, during the period of 2000-2016, MDL data were collected manually through publicly available announcements of gifts of $1 million and above. The school reviewed news articles and press releases issued by donors and recipients, and then manually entered relevant information into the MDL database. While effective, this approach required significant time and human resources and may still not be able to capture the full picture of million-dollar-plus giving due to the limited scope of media coverage.
Starting with the 2021 to 2023 data, MDL data collection was modernized through the use of an artificial intelligence (AI)–driven model. Funded by the Bill and Melinda Gates Foundation, and developed in partnership with Catalyst Balkans, a philanthropy infrastructure organization based in Serbia, this system fully automated the initial data collection process from publicly available sources.
Data Collection and Classification
The AI system was designed to automatically retrieve philanthropy-related news and articles from a wide range of online sources and databases. Its primary data source was the GDELT Project (https://www.gdeltproject.org), which collects more than 60 million English-language news articles annually. GDELT was primarily used to gather article titles and summaries, while full article content was retrieved separately using custom-built web scrapers. Additional web sources not covered by GDELT were also identified and scraped to ensure broader coverage. All collected articles were filtered to include only those relevant to U.S.-based donors, whether giving domestically or internationally.
The AI model was then trained using semantic clustering techniques to identify and classify key data points within each article. Through iterative training and human validation, the system achieved a high level of accuracy in:
- Identifying gifts of $1 million or more;
- Determining whether a gift is qualified as a unique philanthropic donation;
- Classifying key attributes, including donor, recipient, gift type, and cause area; and
- Generating a brief summary of a donation.
Exclusion Criteria
Both the AI model and human reviewers applied the following criteria to exclude the following types of records:
- Cumulative giving over time (e.g., $5 million over 7 years) rather than a single donation;
- Donations involving multiple donors or recipients without sufficient detail to separate them into distinct records;
- Investments or non-philanthropic financial instruments (e.g., bonds, social impact financing, mixed loans);
- Estimates of community impact rather than actual donations;
- Public pledges, earmarks, or announcements of funds without a specific, discrete donation;
- Crowdfunding or pooled contributions attributed to a single entity; and
- Donations for political purposes, including contributions to 501(c)(4) organizations.
Database Structure and Standardization
Using the above criteria, the AI model organized the MDL database into three primary components:
- Article records (including source, date, and time of publication);
- Donation records; and
- Legal entity records (for both donors and recipients).
As part of the de-duplication process, legal entities were matched using both exact and fuzzy matching techniques. Where possible, entities were linked to their Employer Identification Number (EIN), which improved consistency and streamlined future data integration. This process also standardized entity names according to their legal registrations.
Because individual donors do not have unique legal identifiers, standard naming conventions were applied. For example, “and” was used instead of “&,” and full names of donors were used. These conventions improved consistency and helped identify potential duplicate records.
Human Validation
Despite the high accuracy of the AI model, human validation remained essential. The school reviewed a subset of records to ensure accuracy, particularly in cases where:
- Articles lacked complete information;
- The AI attempted to infer missing details; and
- Articles referenced multiple donations but provided limited specifics.
During validation, the school compared AI-classified data against the full source article. When necessary, donor and recipient websites were consulted to verify or supplement missing information.
Insights from human review were fed back into the AI system, allowing for continuous refinement through updated rules and sub-rules. This iterative process improved overall data quality and enabled the system to flag anomalies for further review.
Analytical Considerations
In analyzing the data, the school evaluated which classification variables were most meaningful. While geographic data (e.g., donor location, recipient location, and location of impact) were often available, its analytical value was sometimes limited due to the complexity of organizations operating across multiple regions.
All dollar values in the MDL are reported in nominal terms and are not adjusted for inflation. This reflects the value of each donation at the time it was made and ensures consistency with the MDL database as an ongoing, longitudinal resource.
