Large language models trained on general internet text don't understand your organization's specific priorities, terminology, and values. A general LLM might explain grant funding but won't understand your foundation's unique approach. Custom AI solutions often need to adapt general models to grant-specific contexts. This lesson explores two key techniques: fine-tuning existing models and Retrieval-Augmented Generation (RAG). Understanding these approaches helps you evaluate options when designing custom solutions.
Fine-tuning takes a pre-trained model and continues training it on your organization's data, adapting it to your specific context. Rather than building models from scratch (which requires enormous computational resources), fine-tuning leverages existing model knowledge while specializing it for your purposes.
Transfer learning means taking knowledge learned from one task and applying it to a different task. A model trained on general English text has learned language patterns, common concepts, and reasoning approaches. Fine-tuning transfers that knowledge to your domain: grant administration. Domain adaptation means adjusting models to specific contexts. A model trained on general business language adapts to nonprofit language and priorities through fine-tuning.
Effective fine-tuning requires training data: examples of what you want the model to do. If fine-tuning for grant matching, you need historical data showing nonprofit profiles and matched funders with outcomes. If fine-tuning for application assessment, you need examples of applications paired with reviewer assessments. Typically, hundreds to thousands of examples enable effective fine-tuning. You need enough data that models can learn patterns; too little and models overfit (memorize your data rather than learning generalizable patterns).
Data quality matters enormously. Biased training data produces biased fine-tuned models. Historical data reflecting past funding patterns might perpetuate inequalities in future recommendations. Careful data curation—removing bias, ensuring diverse examples, documenting data provenance—improves fine-tuning quality.
You have options for adapting general models. Prompting means writing detailed instructions telling the model what to do. "Assess this grant application against these criteria" with clear criteria is prompting. Prompting is fast and free (or cheap) but limited—models can only follow instructions so precisely.
Fine-tuning adapts models by training them on your data, making them deeply familiar with your domain. Fine-tuning is expensive (requiring computational resources and time) but produces more capable, specialized models. RAG (discussed below) combines general models with your specific data, without requiring fine-tuning.
Choose based on your needs: Simple tasks with clear instructions? Prompting suffices. Need deep domain expertise? Fine-tune. Need to leverage your specific knowledge without retraining models? Use RAG.
RAG combines language models with retrieval systems. Rather than trying to encode all your knowledge into a model through fine-tuning, RAG keeps your knowledge in a database and retrieves relevant information when needed to inform model outputs.
RAG systems work in steps: User asks a question, the system searches a knowledge base for relevant information, relevant information is combined with the user's question, a language model generates a response informed by both the question and retrieved information. For grants, a user might ask "What nonprofits should we fund working on homelessness?" The system retrieves relevant grant guidelines, past funding to homelessness organizations, and programmatic research on homelessness approaches, then uses that information to generate informed recommendations.
RAG requires knowledge bases—organized collections of relevant information: grant guidelines, nonprofit profiles, research papers, policy documents, past funding data. Information must be stored and indexed for efficient retrieval. Traditional databases work for structured data. Vector databases, newer systems optimized for semantic search, work well for unstructured text. Vector databases convert text into mathematical representations (embeddings) capturing meaning, enabling search by concept rather than keyword.
RAG systems must identify which information is most relevant to questions. Search by exact keyword is easy but brittle—if you search for "poverty relief" but documents say "poverty reduction" or "anti-poverty work," keyword search misses relevant materials. Semantic search using vector databases matches documents with similar meaning even if wording differs. Ranking determines which retrieved documents are most relevant, surfacing best information.
How you ask questions (prompts) significantly affects RAG system performance. Specific, well-structured prompts get better results than vague ones. "Identify nonprofits with strong track records reducing homelessness suitable for $50,000 grants focused on policy advocacy" is more effective than "Find homelessness nonprofits." Well-designed prompt engineering maximizes RAG system value.
RAG is particularly valuable for grant-related tasks. Policy lookup: foundations can RAG-enable searches of their complex grant guidelines so staff quickly find precedents and policies. Program matching: nonprofits search foundation programs finding good fits. Evidence synthesis: RAG systems retrieve research on program approaches, helping justify funding decisions with evidence. Policy advocacy: organizations retrieve relevant research and policy documents informing advocacy campaigns.
Implementing RAG requires: identifying knowledge sources (what documents should your system know about?), converting documents into vector embeddings for semantic search, designing interfaces where users ask questions, and tuning systems so retrieved information is actually relevant and useful.
Whether fine-tuning or using RAG, data quality is paramount. Poor quality training data or knowledge bases produce poor AI systems. Good data curation involves: selecting representative, high-quality examples; documenting where data came from; identifying and addressing biases; ensuring confidential information is properly protected; and regularly reviewing system outputs for problems.
How do you know if fine-tuning or RAG is working? Define success metrics. For grant matching: Do users find recommendations relevant? Do matched nonprofits apply? Are match outcomes better than previous process? For assessment support: Do AI assessments align with human expert assessment? Do staff trust AI recommendations? Document baseline performance (how good was the previous process?) so you can measure improvement.
Fine-tuning and RAG have different cost structures. Fine-tuning requires upfront investment in data curation and model training but lower ongoing costs. RAG requires ongoing maintenance of knowledge bases but lower upfront costs. Computational costs differ: fine-tuned models may be cheaper to run; RAG systems with large knowledge bases may require more computational resources. Evaluate based on your specific situation: volume of questions asked, available training data, knowledge base scope, acceptable response latency, and total budget.
Fine-tuning adapts pre-trained models to your domain through training on your data. RAG combines general models with retrieved information from knowledge bases. Both approaches specialize AI to grant contexts better than general models alone. Choose between them based on your specific needs, available data, and resources.
Design a RAG system for your organization. Identify: What questions should the system answer? What knowledge sources should it draw from? How would you measure if it's working? What challenges would you encounter building this system? Discuss with technical staff to assess feasibility and cost.
Design a complete RAG system your organization could implement. Specify: the user questions it would answer, knowledge sources it would retrieve from, how you'd prepare documents for retrieval, how you'd evaluate relevance, and how you'd measure system success. Create a proposal a technical team could use to implement your design.
General AI models serve general purposes. Custom solutions for grants work better when specialized through fine-tuning or RAG. Understanding these approaches helps you work effectively with vendors building custom systems and evaluate their technical proposals realistically.
Specialize AI systems for grant-specific applications.
Explore Full Course