You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Why not CP (Continue Pretraining) / How about base model know nothing about the domain knowledge ?
When SFT data is sufficient, the model can also learn new knowledge. Completing "data cleaning-CP-SFT-DPO" from scratch costs hundreds of thousands of dollars. As an engineer rather than an academic researcher, GraphGen is the best solution we can give.
Activity
tpoisonooo commentedon Apr 27, 2025
Why not CP (Continue Pretraining) / How about base model know nothing about the domain knowledge ?
When SFT data is sufficient, the model can also learn new knowledge. Completing "data cleaning-CP-SFT-DPO" from scratch costs hundreds of thousands of dollars. As an engineer rather than an academic researcher, GraphGen is the best solution we can give.
[-]FAQ[/-][+][doc] FAQ[/+]tpoisonooo commentedon Apr 27, 2025
Why all of output loss are same value?
See #17 and try
vllm servewithv0.6.6post1version.logprobsinv0.8.3still fails vllm-project/vllm#16838unlimited0715 commentedon Dec 30, 2025
What should I fill in for the tokenizer if I use a large language model deployed locally via Ollama as the Synthesizer Model?
ChenZiHong-Gavin commentedon Dec 30, 2025
You can use the default tiktoken tokenizers like "cl100k_base".