Research notes usually fail in one of two ways. Either they are raw archives, full of PDFs and copied links that are easy to store but hard to reuse, or they are polished notes that take too much time to maintain. An LLM wiki is an interesting middle ground: keep the sources as they are, but ask a language model to compile them into small, linked Markdown pages that can be opened in Obsidian.

The idea is simple. A folder contains the original material: papers, technical reports, meeting notes, links exported as Markdown, or old project notes. Another folder contains the wiki itself. Each page explains one concept, method, dataset, equation, software component, or decision. Pages link to one another using [[wiki links]], so Obsidian can display the structure as a graph and make navigation natural.

This is different from using a chatbot as a search box over a pile of files. Retrieval-augmented generation is useful when the goal is to answer a question from documents. The LLM wiki pattern adds a compilation step: after a new source is added, the model updates existing pages, creates missing pages, links related concepts, and flags contradictions. The knowledge base changes over time instead of being recomputed from scratch at every question.

For research work, that distinction matters. A PhD topic, numerical method, or code base is not a static set of documents. It is a growing network of assumptions, experiments, failures, definitions, and decisions. A good wiki page on a numerical method should not only summarize one paper; it should also point to the equations used in the code, the boundary conditions that caused trouble, the datasets used for validation, and the open questions that remain.

A practical folder layout

I would start with a deliberately small structure:

research-wiki/
  raw/
    papers/
    notes/
    meetings/
  wiki/
    index.md
    log.md
    concepts/
    methods/
    projects/

The raw/ folder is append-only. It is the evidence base: PDFs, Markdown exports, notes, and transcripts. The wiki/ folder is the compiled layer. It should be readable without opening the raw files every time.

The two most useful files are index.md and log.md. The index gives a short map of the wiki: main topics, important pages, and unresolved questions. The log records what was added, when, and which pages were modified. Without a log, it becomes difficult to know whether a page reflects last week’s sources or last year’s understanding.

What to ask the model to do

The prompt should be explicit about structure and uncertainty. A useful instruction is not “summarize these papers”; it is closer to:

Read the new files in raw/.
Update the Markdown wiki in wiki/.
Create one page per concept, method, dataset, software component, or important result.
Use Obsidian-style [[links]] between related pages.
When updating an existing page, preserve useful prior content and add new source-backed details.
Flag contradictions, missing definitions, and claims that need verification.
Update index.md and log.md.
Do not invent citations or results that are not present in the sources.

The last line is important. An LLM wiki is only useful if it remains humble. It should expose weak links, not hide them behind fluent prose.

Why Obsidian fits well

Obsidian works because it does not require a database or a special export format. The wiki is just Markdown files in folders. This keeps the notes portable and easy to version with Git. The graph view is also useful, not as decoration, but as a debugging tool: isolated pages reveal topics that are not connected; over-connected pages often mean that a concept should be split; clusters show where a project has become mature enough to deserve its own index.

Obsidian also keeps the human in the loop. The model can draft and maintain pages, but the researcher should still review important claims, rename pages, merge duplicates, and decide which contradictions are real. The workflow is better understood as assisted editorial work than as automatic knowledge management.

Maintenance rules

The wiki needs occasional linting. Every few additions, ask the model to inspect the compiled pages and report:

• duplicate pages with different names;
• pages that mix several concepts and should be split;
• pages with no incoming or outgoing links;
• claims without a source;
• contradictions between pages;
• obsolete statements that newer sources have superseded.

This audit pass is where the system starts to compound. The value is not only in producing more notes, but in making old notes easier to reuse.

Where this is useful

I would use this pattern for subjects that evolve over months: literature reviews, numerical methods, software architecture notes, experiment logs, or teaching material. It is less appropriate for information that changes daily or for situations where every sentence needs a precise citation trail. In those cases, a conventional search index or a stricter reference manager is still better.

The promising part is the combination of boring files and active maintenance. Markdown keeps the system inspectable. Obsidian makes it navigable. The LLM does the repetitive compilation work. The researcher keeps responsibility for judgment.

Inspired by the Data Science Dojo tutorial on the LLM wiki pattern, but adapted here for a research notebook workflow with Obsidian and Git.

I was initially skeptical about NotebookLM, mostly because “chat with your documents” sounds like a feature every AI product claims to have. After using this kind of tool more seriously, I think the interesting part is not the chat. It is the fact that the conversation is constrained by a pile of sources that you chose yourself.

That changes the feeling of the workflow. Instead of opening a generic chatbot and hoping it remembers the right thing, you build a small working library: papers, web pages, YouTube videos, audio files, Google Docs, slides, Markdown notes, CSV files, images, or PDFs. Then you ask questions inside that library. The answers are still generated by an AI model, so they still need checking, but the citations make the checking much less painful.

I would not use it as a place to store my final notes. I would use it as a temporary desk: a space where I can throw a focused set of documents, ask messy questions, generate first drafts of summaries, and then move the useful parts back into Obsidian, Git, or a project README.

Where it fits in a research workflow

The unit that makes sense to me is one notebook per real question, not one notebook for everything. A notebook for a numerical-methods paper could contain the submitted manuscript, two or three related papers, implementation notes, validation cases, and meeting notes. A teaching notebook could contain slides, exercises, references, and the questions that tend to come back every year.

The workflow I like is simple:

1. Add only the sources that define the current question.
2. Ask for a rough map: main concepts, disagreements, missing definitions, and recurring assumptions.
3. Use follow-up questions to compare sources, not only summarize them.
4. Generate a few artifacts: a briefing doc, a mind map, a quiz, or an audio overview.
5. Copy the useful conclusions somewhere durable.

The last step matters. If a conclusion is important, it should not only live in a NotebookLM chat history. It should end up in a note, a commit, a paper draft, or a task list.

Small habits that help

Use fewer sources than you think. NotebookLM supports quite large notebooks, but the answers are better when the source set is intentional. For a literature review, I prefer comparing two or three papers first, then asking for a global synthesis once the local comparisons are clear.

Ask for disagreement. “Summarize this” is rarely the best prompt. More useful questions are: “Which assumptions differ across these sources?”, “Which results are not directly comparable?”, “What would break if this boundary condition changed?”, or “Which claims are repeated but not really justified?”

Mention source names. When several documents are active, naming the documents in the prompt makes the answer easier to audit. It also reduces the risk of getting a smooth average of several incompatible texts.

Use the generated artifacts for different jobs. Audio Overviews are good for passive review, especially before a meeting or during a commute. Mind Maps are useful for seeing whether the source set has a structure. Flashcards and quizzes are better when the goal is recall. None of these outputs should be treated as final, but they are good first passes.

Customize the output. Audio Overviews can be steered by format, language, length, focus, and expertise level. A two-minute briefing for a collaborator should not use the same prompt as a critical review of a draft paper.

Keep a verification pass. NotebookLM can still make mistakes, and Google explicitly warns that generated audio may contain inaccuracies. I treat every generated summary as an index into the sources, not as a substitute for reading them.

Prompts I would actually use

For papers, start with:

Extract the research question, model assumptions, numerical method,
validation cases, limitations, and claims that require checking.
Return citations for each important claim.

For meetings:

Turn these notes into decisions, open questions, owners, and follow-up tasks.
Separate facts from interpretation.

For teaching:

Create a study guide from these sources.
Then create ten quiz questions at three difficulty levels.
For each answer, cite the source section that supports it.

For writing:

Compare this draft with the source material.
Find unsupported claims, missing citations, repeated ideas, and unclear transitions.

Limits to remember

The free tier has finite limits: Google lists 100 notebooks, up to 50 sources per notebook, sources up to 500,000 words, and daily limits on chat and audio generations. Local uploads can be up to 200 MB per source. These limits are generous enough for many research tasks, but they still push toward a useful discipline: keep notebooks focused.

The more subtle limit is conceptual. NotebookLM is strong when the source set is well chosen. It is weaker when the notebook becomes a vague dumping ground. The quality of the answer depends first on the quality of the small library you build.

For me, that is the right mental model: not an oracle, not a replacement for a reference manager, not a final note-taking system. Just a useful workbench for thinking with sources before writing something more permanent.

Sources

• Google NotebookLM Help: Learn about NotebookLM
• Google NotebookLM Help: Add or discover new sources
• Google NotebookLM Help: Frequently asked questions
• Google NotebookLM Help: Audio Overviews
• Google NotebookLM Help: Mind Maps
• Google Blog: NotebookLM learning features
• Google Blog: Video Overviews and Studio upgrades
• Google Blog: Better research with NotebookLM

AI agents have usage limits (tokens, iterations, budget, context window). They are normal, but you can push them back with good session hygiene and a few simple technical choices.

Why You Hit the Limits

• Cache misses: switching models or tools too often invalidates the cache.
• Context bloat: too many files reread, too much unnecessary history.
• Wrong effort: using a large model for a grep or formatting task.
• Bad input format: unoptimized images/PDFs burn tokens.

The Real Lever: Protect the Cached Prefix

• Choose tools/MCP at the start and stick with them.
• Choose the model at the start and do not change /model mid-session.
• Keep AGENTS.md / CLAUDE.md short.
• Move the details into MATH.md, TESTING.md, SKILL.md.

Session Hygiene: 6 Simple Habits

1. /clear between topics to avoid stacking context.
2. /compact after a task (or around 50-80% of the context).
3. /rewind if the session goes bad instead of continuing to patch it.
4. Sub-agents for large reads, logs, PDFs, and file search.
5. Do not reread the whole repo for every request: point to the right folder/file.
6. Split tasks up: one clear action per message.

Choose the Right Model for the Right Task

• Small model: grep, renaming, formatting.
• Medium model: local exploration, tests, short synthesis.
• Large model: architecture, tradeoffs, long-term decisions.

Bad Format = Burned Tokens

• Web: prefer text / the accessibility tree over screenshots.
• PDF: extract the text (pdftotext) before sending images.
• Large repo: start from a map or graph rather than reading everything.

TL;DR

• Fix the model and tools at the start.
• Keep the context light.
• Clean up sessions.
• Use the right model at the right time.
• Optimize the source format.

If you want to export or edit it, the source file is available in the repository.

Turning a paper into code is never just “implement the paper.” At least, it should not be. A paper is full of equations, tables, assumptions, conventions, and shortcuts that made sense to the authors, but not necessarily to the next person trying to reproduce one part of it.

This is one place where AI agents are genuinely useful. Not because they magically understand the science, but because they are patient with the annoying first layer of work: reading PDFs, extracting equations, doing OCR when the text layer is bad, converting sections to Markdown, listing symbols, and comparing a paper against an existing code base.

The example I have in mind is my work on PenguinReactiveTransferClimentBenchmarks, a Julia benchmark package around scalar and coupled reactive transfer in the PenguinxCutCell ecosystem. The goal was not to make an impressive demo. The goal was more modest and more useful: take benchmark material from the literature, translate it into small reproducible cases, and make the result auditable.

The first pass is reading

The first useful task for the agent is not coding. It is reading the paper in a way that prepares coding.

I want it to extract:

• the equations;
• the definitions of symbols;
• the nondimensional numbers;
• the boundary and interface conditions;
• the reference tables and figures;
• the assumptions that are easy to miss;
• the cases that are actually comparable with my solver.

That last point matters. A table from a finite-Reynolds DNS paper is not automatically a validation target for a low-Re prescribed-velocity solver. It can be useful context, but it should not become an acceptance criterion if the physics solved by the code is different.

This is where PDF handling matters. Sometimes the equation text is selectable. Sometimes it is not. Sometimes a table needs OCR. Sometimes the cleanest path is to ask the agent to convert a few pages into Markdown, then manually check the equations before going further. That manual check is not optional.

Then comes the implementation map

Once the paper is readable, I want a map from the paper to the repository. In the Climent benchmark package, this means identifying whether a case belongs near an existing scalar spherical benchmark, a two-phase conjugate transfer problem, a prescribed Stokes-flow diagnostic, or a multi-species reaction block.

The repository already has a useful shape:

• benchmark scripts such as scripts/run_benchmark.jl;
• source modules in src/;
• tests in test/;
• reference data in data/reference and data/references;
• generated result CSVs in data/results;
• plotting and documentation scripts.

The agent should read that structure before proposing code. If a new case resembles an existing one, the right answer is usually to reuse the existing post-processing conventions and add a small benchmark path. The wrong answer is to invent a separate framework because the prompt only mentioned the paper.

Ask for comparisons, not just code

The prompts that work best are not “write the implementation.” They are comparative:

Compare the equations in this paper with the existing B1 and B5 benchmark code.
Which terms are already represented?
Which assumptions differ?
Which reference quantities can be tested on the current grid?
Which rows should be marked diagnostic rather than valid?

This forces the agent to reason across three things at once: the paper, the current code, and the validation layer. It also gives me something to review before any file changes happen. If the mapping is wrong, the code will be wrong too.

Build a ladder

A good implementation should climb a ladder:

1. formula checks;
2. geometry and post-processing checks;
3. scalar smoke benchmarks;
4. local convergence or resolution studies;
5. coupled or multi-species benchmarks;
6. paper-level campaigns and plots.

This ladder is important because it prevents overclaiming. A smoke benchmark only says that the case is wired correctly. A local run gives more confidence. A paper-level campaign is slower and should be reserved for results that are meant to support a figure, a table, or a claim.

In the benchmark repository, this shows up through result levels, usable flags, under-resolution flags, geometry checks, and stored reference tables. Those details may look bureaucratic, but they are what make the output honest.

Keep the final result reviewable

For scientific code, “it runs” is not enough. At the end of an agent session, I want a plain report:

• changed files;
• commands run;
• numerical results;
• uncertainty or limitations.

If the agent added a benchmark, I want to know which CSV row changed, which figure can be regenerated, which test passed, and what still should not be claimed. The final answer should not sound like a press release. It should sound like a lab notebook.

The useful unit

The productive unit is not “paper to code.” It is smaller:

one equation -> one implementation path -> one testable quantity -> one result row

Once that path is clear, adding more benchmarks becomes less mysterious. The paper provides a case. The code provides operators. The tests provide constraints. The CSV files keep the record of what was actually checked.

AI agents are useful here because they can do the repetitive reading and glue work: extract equations, OCR tables, convert fragments to Markdown, inspect nearby code, draft a runner, and summarize discrepancies. They are dangerous when they flatten scientific distinctions.

For this kind of work, I would rather have a cautious agent that says “this reference is out of scope for the current solver” than a confident agent that makes every table look validated.

Obsidian is attractive for research because it does not try to own the notes. A vault is mostly a folder of Markdown files. That sounds boring, but for long projects it is exactly the right kind of boring: readable files, local control, links that still make sense outside the application, and a structure that can be versioned with Git.

I do not think a research note system should become a second research project. It should be simple enough to survive deadlines, paper revisions, teaching, and the weeks where nothing is perfectly organized. My rule is: if I cannot maintain the system when I am tired, the system is too complex.

A small vault structure

I would start with something like this:

research-notes/
  00-inbox/
  10-projects/
  20-literature/
  30-methods/
  40-meetings/
  50-teaching/
  assets/

The exact names do not matter much. What matters is that the folders describe the role of a note, not a perfect taxonomy of the world. A note about a paper goes in 20-literature/. A note about a numerical scheme goes in 30-methods/. A note about an ongoing article or code base goes in 10-projects/.

The inbox is useful because not every note deserves classification immediately. A copied idea, a meeting remark, or a quick equation can live there until it becomes clear whether it belongs to a project, a method, or nowhere.

Notes should be small enough to link

The temptation is to make one giant note per project. That works for the first week, then it becomes a document you never want to open. I prefer smaller notes:

• one note for a paper;
• one note for a method;
• one note for a benchmark;
• one note for a decision;
• one note for a recurring problem.

Links are useful when the notes have real boundaries. A project note can link to [[space-time cut-cell method]], [[manufactured solution]], [[Stefan problem]], or [[ICCFD 2026]]. The graph view is not the point, but it becomes a helpful diagnostic: isolated notes are probably unprocessed; over-connected notes may need to be split.

Git changes the way I write notes

Git is not only backup. It gives a history of thinking.

For research notes, commits do not need to be perfect. A useful rhythm is:

git add .
git commit -m "Add notes on reactive transfer benchmarks"

The value appears later. When a paper draft changes, when a benchmark result is replaced, or when a hypothesis quietly disappears, the history is still there. It is also easier to be honest in notes when old versions are not lost. You can rewrite a page aggressively because Git keeps the previous version.

What I would keep out of the vault

Not everything belongs in Obsidian. Large raw datasets, generated figures, build artifacts, and simulation outputs should usually stay elsewhere. The note should link to them, describe them, and explain why they matter. It should not become a fragile archive of everything.

For PDFs, I prefer a pragmatic approach. Keep the PDF somewhere stable, create a Markdown note for the paper, and write the useful content there: main question, assumptions, equations, numerical method, limitations, and why the paper matters for the current work.

A note template for papers

# Paper title

## Why I opened it

## Main contribution

## Model and assumptions

## Numerical method

## Results worth reusing

## Limitations

## Links

The important section is “Why I opened it.” Without that, literature notes become summaries detached from the project. A paper is rarely useful in the abstract; it is useful because it answers a question, gives a method, provides a benchmark, or contradicts something.

The role of AI

LLMs fit naturally around this workflow, but only as assistants. They can turn rough notes into a cleaner page, extract assumptions from a paper, propose links to existing notes, or produce a first draft of a meeting summary. They should not silently rewrite the record.

The best pattern is explicit:

Read this note and propose links to existing concepts.
Do not delete information.
Separate source-backed claims from interpretation.
List uncertain points at the end.

That keeps the vault as a human research memory, not just a pile of generated summaries.

My preferred rule

Every important idea should eventually have three things: a note, a link, and a reason. The note stores the idea. The link places it in the project. The reason explains why it was worth writing down.

Obsidian provides the writing surface. Git provides the memory. The rest is discipline.

The prompt template I keep coming back to is deliberately structured:

Objective
  Add a manufactured-solution convergence test for <equation/method>.

Context to read first
  Read src/<operator>, test/<nearest existing test>, docs/MATH.md.

Scientific constraints
  The exact solution is ... Evaluate L2 error at refinements [...].

Implementation contract
  Start with a plan. Make the smallest patch. Run <command>.

Final response
  List changed files, commands run, numerical results and uncertainty.

The important part is not the formatting. The important part is that the model receives the same information a careful collaborator would need before touching the code.

Objective

The objective should be narrow. “Improve the solver” is too vague. “Add a manufactured-solution convergence test for the diffusion operator” is better because it defines a target and suggests what kind of evidence will count.

For numerical code, I try to include the equation or method name directly in the objective. That helps the model search locally and prevents it from drifting toward unrelated tests.

Context to read first

This section is the antidote to generic code. The model should read the operator, the nearest existing test, and the mathematical notes before writing anything. If the repository already has a pattern for grids, tolerances, fixtures, or error norms, the new test should follow it.

The phrase “nearest existing test” is useful. It tells the model to adapt local style instead of inventing a new testing framework.

Scientific constraints

This is the part that normal software prompts often miss. Scientific code needs more than input/output behavior. It needs the mathematical claim:

• the exact solution;
• the forcing term;
• the boundary conditions;
• the refinement sequence;
• the norm used for the error;
• the expected convergence rate;
• the tolerance and why it is reasonable.

If these are not written down, the model may choose convenient values that make the test pass without testing the intended property.

Implementation contract

I usually ask for the smallest patch. Not because small patches are morally better, but because they are easier to audit. If the goal is a convergence test, I do not want an unrelated refactor of the solver, the plotting code, and the documentation in the same change.

The command matters too. A coding agent should know how success will be checked:

Run julia --project=. -e 'using Pkg; Pkg.test()'

or, for a narrower pass:

Run julia --project=. test/test_diffusion_operator.jl

The command is part of the task, not an afterthought.

Final response

The final response should be boring and factual:

• changed files;
• commands run;
• numerical results;
• uncertainty.

That last item is important. A convergence rate estimated on three grids is not the same thing as a proof. A passing smoke test is not a paper-level validation. A result that depends on a coarse geometry should say so.

Why this works

The template forces the model to operate inside a scientific boundary. It tells it what to read, what to preserve, how to make the change, and how to report evidence.

It also makes review faster. If the final answer does not list numerical results, something is missing. If no uncertainty is mentioned, the result is probably oversold. If the patch is large, it probably ignored the implementation contract.

For research software, a good prompt is not just a request. It is a small protocol for keeping the code, the mathematics, and the evidence aligned.

Context and Goal

I wanted to test fine-tuning on a domain I know well: finite volume methods in numerical simulation. The goal was to make Zephyr-7B able to explain the concepts and schemes clearly, so it could answer technical questions on the topic.

Data Preparation: The Heart of the Project

The quality of a fine-tune depends on the training data. I built a dataset of a few question/answer pairs covering:

1. Theoretical concepts:
   • “What is the CFL condition and how do you apply it to an explicit scheme?”
   • “What is the difference between an upwind scheme and a centered scheme for discretizing a convective term?”
2. Practical cases:
   • “How do you handle a Neumann boundary condition in a finite volume solver?”
   • “What are the stability criteria for an implicit scheme applied to the heat equation?”
3. Method comparisons:
   • “What are the advantages and disadvantages of explicit vs implicit schemes?”
   • “When should an upwind scheme be preferred over a centered scheme?”

Data format: Each example follows a standard format for SFTTrainer:

{
  "text": "### Instruction: [Question about finite volumes]\n### Response: [Detailed technical answer]"
}

Example:

{
  "text": "### Instruction: Explain the CFL condition for an explicit finite volume scheme.
### Response: The CFL (Courant-Friedrichs-Lewy) condition is a stability criterion for explicit numerical schemes. For a convection problem (∂u/∂t + c∂u/∂x = 0), it is written as Δt ≤ Δx/|c|, where Δt is the time step, Δx the spatial step, and c the convection speed. For diffusion (∂u/∂t = ν∂²u/∂x²), it becomes Δt ≤ Δx²/(2ν). These conditions ensure that numerical information does not propagate faster than the physics of the problem."
}

Why this format?

• Clear structure: The model learns to distinguish the question (Instruction) from the answer (Response).
• Compatibility: trl’s SFTTrainer is optimized for this format, which simplifies training.

Fine-tuning Methodology: Zephyr-7B + LoRA in 4-bit

Model and Technique Choice

I selected Zephyr-7B for its instruction-following ability and its size, which is suitable for a consumer GPU. To reduce memory usage, I used:

• LoRA (Low-Rank Adaptation): A technique that trains only a few additional weights (the “adapters”), without modifying the base model.
• 4-bit quantization: Reduces weight precision to save memory (via bitsandbytes).

Technical Setup

1. Model loading:

   from transformers import AutoModelForCausalLM, BitsAndBytesConfig
   
   bnb_config = BitsAndBytesConfig(
       load_in_4bit=True,          # 4-bit quantization
       bnb_4bit_quant_type="nf4",  # Optimal quantization type
       bnb_4bit_compute_dtype=torch.float16,  # FP16 computation
   )
   
   model = AutoModelForCausalLM.from_pretrained(
       "HuggingFaceH4/zephyr-7b-beta",
       quantization_config=bnb_config,
       device_map="auto",  # Automatically uses the GPU
   )
   

2. LoRA setup:

   from peft import LoraConfig
   
   lora_config = LoraConfig(
       r=8,                     # LoRA matrix rank (smaller = fewer parameters)
       lora_alpha=32,           # Scaling factor
       target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # Target layers
       lora_dropout=0.05,       # Dropout to reduce overfitting
       bias="none",             # No bias in the adapters
       task_type="CAUSAL_LM",   # Task type (language model)
   )
   

3. Preparing the model for LoRA:

   model = prepare_model_for_kbit_training(model)  # Prepares the model for LoRA
   model = get_peft_model(model, lora_config)      # Applies LoRA
   

Memory Optimizations

To avoid CUDA out of memory errors on a T4 GPU (15 GB of VRAM), I:

• Reduced the batch size: per_device_train_batch_size=2.
• Used gradient accumulation: gradient_accumulation_steps=8 (equivalent to a batch size of 16).
• Limited the sequence length: max_seq_length=256.
• Cleared memory before training:

  import torch, gc
  torch.cuda.empty_cache()
  gc.collect()
  

Training: Parameters and Results

Training Parameters

from transformers import TrainingArguments

training_args = TrainingArguments(
    output_dir="zephyr-volumes-finis",      # Output directory
    per_device_train_batch_size=2,          # Batch size
    gradient_accumulation_steps=8,          # Gradient accumulation
    learning_rate=2e-4,                     # Learning rate
    num_train_epochs=3,                     # Number of epochs
    fp16=True,                              # Mixed precision (FP16)
    save_steps=50,                          # Periodic checkpoints
    logging_steps=10,                       # Periodic logs
    optim="paged_adamw_8bit",                # Memory-efficient optimizer
    report_to="none",                       # Disable Weights & Biases
)

Running Training

from trl import SFTTrainer

trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset,          # Prepared dataset
    peft_config=lora_config,        # LoRA configuration
    max_seq_length=256,             # Maximum sequence length
)

trainer.train()  # Start training

Results and Observations

• Duration: ~ minutes for 3 epochs on a few examples (T4 GPU).
• Loss progression:
  • Epoch 1: Loss ~0.5 (the model is learning the answer structure).
  • Epoch 2: Loss ~0.25 (the answers become more precise).
  • Epoch 3: Loss ~0.15 (stabilization, correct generalization).
• Answer quality:
  • The model generates precise technical explanations, even for questions not present in the dataset.
  • The responses are structured and context-aware (for example, convection/diffusion distinctions).

Example generated answer for “What are the advantages of an implicit scheme?”:

*“An implicit scheme has three major advantages:

1. Unconditional stability: Unlike explicit schemes, it is not subject to the CFL condition, which allows larger time steps (Δt ≫ Δx²/(2ν) for diffusion).
2. Time accuracy: By avoiding restrictions on Δt, it better captures slow dynamics (for example, diffusion over long periods).
3. Robustness: Less sensitive to numerical instabilities, even for stiff problems (for example, equations with strong source terms). Drawback: It requires solving a linear system at each time step, which increases the computational cost (tridiagonal matrix for 1D problems).”*

Validation and Limits

To validate the model, I tested it on questions not present in the dataset:

1. “How do you discretize a source term in the convection-diffusion equation?” → A coherent answer explaining how the source term is integrated into the flux balance.
2. “What is the influence of the mesh on the accuracy of a finite volume scheme?” → A detailed answer about truncation error and convergence.

Identified limits:

• Limited precision for very specific problems (for example, 3D unstructured meshes).
• Dataset dependence: The model generalizes well within its training domain, but may lack detail for uncovered cases.

Future Improvements

1. Expand the dataset:
   • Add examples on non-uniform meshes, 2D/3D problems, and high-order schemes.
   • Include cases with source terms or chemical reactions (for example, a reaction-diffusion equation).
2. Optimize training:
   • Test with more epochs (5-10) to improve accuracy.
   • Use a validation set to evaluate generalization.
3. Practical integration:
   • Develop an API to interact with the model through a web interface.
   • Create a Jupyter notebook with interactive PDE-solving examples.

Conclusion

This project shows that a large model like Zephyr-7B can be adapted to a specific technical domain with little data and training time. The model already answers finite volume questions correctly and explains the concepts clearly.

Why is this useful?

• Saves time when looking for explanations.
• Gives pedagogical answers adapted to the audience.
• Can be extended to other numerical methods.

Next steps:

• …

Project link: GitHub Gist