Add suuport to params.lora_adapters in newer llama.cpp (after b3534)

[xunuohope1107]

Llama.cpp has updated the code if (arg == "--lora") { CHECK_ARG params.lora_adapter.emplace_back(argv[i], 1.0f); return true; } in common/common.cpp to if (arg == "--lora") { CHECK_ARG params.lora_adapters.push_back({ std::string(argv[i]), 1.0, }); return true; }, which is likely a vector of structures that contain both the adapter name (a std::string) and a scaling factor (1.0 in the default case).

Could you please update llama.cpp version and the data structure to fit it? Thanks!

[kherud]

Hey @xunuohope1107 I'm not sure I understand your problem, you can already specify multiple lora adapters and their scaling factor using ModelParameters#setLoraAdapters(Map<String, Float>). I will however update to the newest llama.cpp version in the next few days.

[xunuohope1107]

Actually, this issue happen when i use cmake to build the binary lib, under src/main/cpp/server.hpp (line 2737) params.lora_adapter = json_value(jparams, "lora_adapter", default_params.lora_adapter);, I would get something like 'Cannot found "lora_adapter", do you mean "lora_adapters"?', because llama.cpp has change params.lora_adapter to params.lora_adapters under common/common.cpp

[kherud]

Ah I see what you mean now. If you use the cmake file of this project (java-llama.cpp) it should automatically use a compatible llama.cpp version (see here). You probably want to use a newer llama.cpp version, though, which then of course doesn't work. But as I said, I'll update to the newest llama.cpp version soon!

[vaiju1981]

@xunuohope1107 can you check if this works now, the code is updated to latest.

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125

[bernardladenthin]

Resolved in the downstream fork at https://github.com/bernardladenthin/java-llama.cpp.
ModelParameters.addLoraAdapter(String) and
addLoraScaledAdapter(String, float) (ModelParameters.java:906,918) map to
--lora and --lora-scaled, which the upstream parser already converts into
the vector-of-lora_adapters form; setLoraInitWithoutApply() covers the
matching flag. llama.cpp pin is b9284, well past b3534.