AI evaluation framework
Rate each criterion to evaluate an AI provider for ethics, security, and value.
Based on our AI evaluation training for nonprofits.
Select a rating for each criterion to see detailed descriptions and actionable recommendations.
Criterion
Description
Recommendation
Reputation
Positive reviews, clean track record
Limited information available
No credible mentions or red flags
---
Verify reviews are from credible sources and recent
Search more thoroughly: check tech news sites and ask in nonprofit communities
This is a major warning sign: consider well-established alternatives instead
---
Access
Free account or generous trial
Limited trial period
Must pay or request demo
---
Check what features are limited in the free tier before committing
Ask for an extended trial to properly evaluate if it meets your needs
Avoid providers who won't let you test the product: this is a red flag
---
Technical info
Extensive documentation available
Basic information only
No technical details
---
Review their documentation to understand limitations and capabilities
Ask the provider directly for more details about how their system works
Lack of transparency is concerning: they may be hiding limitations or repackaging others' work
---
Marketing claims
Specific, evidence-backed claims
Vague or exaggerated language
Unrealistic "too good to be true" promises
---
Still verify their claims match your specific use case
Ask for concrete examples and evidence to back up their claims
Unrealistic promises usually mean the tool won't deliver: look elsewhere
---
Performance data
Published benchmark results
Cherry-picked comparisons
No performance data
---
Check if benchmarks are relevant to your intended use
Ask why they only compare to certain competitors or older versions
Without performance data, you have no way to judge if this tool is any good
---
Team background
Demonstrable expertise and track record, with accountability
Limited personal info, but credibility signals present
Completely anonymous with no accountability or credibility
---
Verify claimed expertise through past work, publications, or feedback from clients
Look for credibility signals: relevant experience, transparent methodology, public work history, or organisational affiliation
No accountability or credibility signals is a major red flag: avoid providers that you cannot vet or hold accountable
---
Pricing
Transparent, competitive pricing
Opaque or hard to find pricing
Suspiciously cheap/expensive or hidden
---
Calculate total cost including any usage limits or add-ons
Demand transparent pricing before proceeding: hidden costs often appear later
Pricing red flags often indicate other problems: look for honest, transparent providers
---
Data protection
Unambiguous policy: not used for training
Vague data usage policy
Data used for training or retained
---
Read the full privacy policy: ensure it covers your specific data protection needs
Contact them to clarify exactly what happens to your data: get it in writing
Do not use tools that train on your data: this puts beneficiaries and donors at risk
---
Ethics & values
Explicit ethics policy and values alignment
No ethics information
Concerning practices (military, surveillance)
---
Verify their stated values match their actual practices (check news, partnerships)
Research their clients and partnerships: they may have problematic ties not publicly advertised
Using tools from companies with questionable ethics can damage your organisation's reputation
---
0
Good
0
Caution
0
Bad
/9
Scored
Evaluation complete
✔ Provider appears safe ❓ Requires careful review ⚠ Proceed with caution ❌ Consider alternatives
Reach out
Connect with us to find out how we can help you use data and AI for greater impact.
Instagram
BlueSky