Have anyone tried evaluate PDrop under lmms-eval framework?

[ashun989]

I have evaluated PDrop on lmms-eval. Here are some results.
layer_list=[8,16,24], image_token_ratio_list=[0.5, 0.25, 0.125]

Model	gqa_lite	vqav2_lite	sqa_img	textvqa_val_lite	vizwiz_vqa_val_lite	docvqa_val_lite	mmmu_val	mme
llava-1.5-7b	0.684	0.716	0.696	0.496	0.459	0.290	0.366	1508
Pdrop	0.698	0.698	0.692	0.481	0.459	0.273	0.371	1499

[Cooperx521]

Sorry for the delayed response. I just finished some tasks recently, now starting to organize the PDrop code, and expect to support lmms-eval by next week.

Thanks.