If create a new LlamaModel and then close it then nothing is freed by native code.
To reproduce the issue it is enough to run this code:
public static void testCloseLoop(String modelPath)
{
ModelParameters modelParameters=new ModelParameters();
modelParameters.setModel(modelPath).setThreads(4).setKeep(-1).setCtxSize(1024).setGpuLayers(0);
for (int i=0;i<10;i++)
{
LlamaModel lamaModel = new LlamaModel(modelParameters);
lamaModel.close();
try { synchronized (lamaModel) { lamaModel.wait(100); } }
catch (InterruptedException e) { System.out.println(e); }
}
}
The number of cycles depends on memory size on your computer. With GPU an exception is thrown after memory limit is exceeded while on CPU after memory limit is reached the swap space is employed and things become very ugly.
Introduction of delay doesn't help, despite of hope of waiting for some time, required for OS to free the memory, could help.
If create a new LlamaModel and then close it then nothing is freed by native code.
To reproduce the issue it is enough to run this code:
The number of cycles depends on memory size on your computer. With GPU an exception is thrown after memory limit is exceeded while on CPU after memory limit is reached the swap space is employed and things become very ugly.
Introduction of delay doesn't help, despite of hope of waiting for some time, required for OS to free the memory, could help.