How do I set different compression versions?

[liuyafeng0609]

Hello, this is excellent work.

I'd like to know how to set the different compression versions in Table 2? I want to reproduce this result.

For example, to evaluate the metrics, retaining 128, 64, and 32 tokens corresponds to compression rates of 22.2%, 11.1%, and 5.6%, respectively.

I really need this data and hope to receive a reply from the author.

Thank you very much.

[Cooperx521]

Sorry for the confusion. You should be able to see the previous discussion here: #10. I haven’t been following this topic for quite some time as I’ve been away from this field.

I completely understand your urgency, and I think this issue can be resolved quite simply. Since using straightforward settings like 8, 16, and 24, each dropping half of the tokens, cannot achieve a low enough compression ratio, you could try combinations such as 2, 8, 16 or 2, 4, 6, 8, or other similar settings. The key is to maintain the λ, λ², ... structure. As long as the final average number of tokens matches your desired target, you can determine the compression ratio for each layer by solving equations like this:

λ = 0.19
layer_list = 2, 8, 16

λ = 0.29
layer_list = 0, 2, 8

Finally, regardless of the final performance, this result can be reported as the outcome of PDrop, as long as the setup is clearly explained. Please make sure to clearly describe in your paper (either in the main text or the appendix) how you applied PDrop, that would be perfectly reasonable.

[liuyafeng0609]

Thank you very much. The problem has been solved.