CHES_CTF Dataset: Can't see reason for applied 800 shift during dataset processing in raw data

[lambdaupb]

EDIT: See latest comment after re-open! - While this issue was initially created because of a bug on my side, the Issue actually exists

Old Issue:

feature_selection_dlsca/experiments/CHESCTF/generate_dataset.py

Lines 50 to 55 in eb5b909

	""" The second file contains traces shifted by 800 samples in acquisition phase """
	if filename == f"{raw_trace_folder_chesctf}/PinataAcqTask2.2_10k_upload.trs":
	sample_offset = 800
	else:
	sample_offset = 0

I don't really understand the reasoning behind applying the 800 sample shift as done above.

Could you provide some insight on this?

This is the processed data without applying the 800 sample shift to the traces from PinataAcqTask2.2_10k_upload.trs

Below is with the 800 sample shift:

[lambdaupb]

~~Using the CHES OPOI MLP ID using the non-shifted data I get covergence with ~10 traces in comparison to the result in the feature selection paper, which lists ~1900.
Maybe this is connected.~~

[AISyLab]

Guilherme, when you have time? Stjepan From: lambdaupb ***@***.***> Sent: Sunday, May 7, 2023 6:27 PM To: AISyLab/feature_selection_dlsca ***@***.***> Cc: Subscribed ***@***.***> Subject: [AISyLab/feature_selection_dlsca] CHES_CTF Dataset: Can't see reason for applied 800 shift during dataset processing in raw data (Issue #2) https://github.com/AISyLab/feature_selection_dlsca/blob/eb5b909f8d070faa7ae69051e016acaa6dd5f12d/experiments/CHESCTF/generate_dataset.py#L50C5-L55<https://urldefense.com/v3/__https:/github.com/AISyLab/feature_selection_dlsca/blob/eb5b909f8d070faa7ae69051e016acaa6dd5f12d/experiments/CHESCTF/generate_dataset.py*L50C5-L55__;Iw!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbWinS-Kqg$> I don't really understand the reasoning behind applying the 800 sample shift as done above. Could you provide some insight on this? This is the processed data without applying the 800 sample shift to the traces from PinataAcqTask2.2_10k_upload.trs [Image removed by sender. image]<https://urldefense.com/v3/__https:/user-images.githubusercontent.com/1890613/236689503-c4ca01d1-34cd-450e-8e1c-cfd03583d7e4.png__;!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbUclC9S7A$> [Image removed by sender. image]<https://urldefense.com/v3/__https:/user-images.githubusercontent.com/1890613/236689515-3e8cd1bd-d760-48a9-84ff-684eb82e1bd4.png__;!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbXqVYdMyw$> Below is with the 800 sample shift: [Image removed by sender. image]<https://urldefense.com/v3/__https:/user-images.githubusercontent.com/1890613/236689627-bf6c06b7-5fc9-4f27-9ca5-97453ac789ce.png__;!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbUZaQLthg$> [Image removed by sender. image]<https://urldefense.com/v3/__https:/user-images.githubusercontent.com/1890613/236689631-d1aa2482-323a-4fd1-9aea-5e3613e6a08b.png__;!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbVYO-bt2Q$> — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/AISyLab/feature_selection_dlsca/issues/2__;!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbVIQb05kA$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/APDG756H4XATRXX34PN753LXE7EL7ANCNFSM6AAAAAAXY7LITI__;!!PAKc-5URQlI!5wiKpFdz8zVjGzhZruxmQceTX2du8AKtplQzSTt7co85VNr-V33Njarnoc7QolF-Ak1yp2qxaO4d3Hd3kbVIvzFLwg$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AISyLab]

More. Stjepan From: lambdaupb ***@***.***> Sent: Sunday, May 7, 2023 10:53 PM To: AISyLab/feature_selection_dlsca ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [AISyLab/feature_selection_dlsca] CHES_CTF Dataset: Can't see reason for applied 800 shift during dataset processing in raw data (Issue #2) Using the CHES OPOI MLP ID using the non-shifted data I get covergence with ~10 traces in comparison to result in the feature selection paper, which lists ~1900. Maybe this is connected. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/AISyLab/feature_selection_dlsca/issues/2*issuecomment-1537537762__;Iw!!PAKc-5URQlI!-_G3FzYGFEY1ZnM81sFI5hpYZYbo_vfepf1Jho5TRgKYUk8eXFfCwP_gZRBbnHr0CiGNfmMq3Fzyzp1P3hobXdY4QQ$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/APDG754N5YX4DTK35KI2TILXFADRFANCNFSM6AAAAAAXY7LITI__;!!PAKc-5URQlI!-_G3FzYGFEY1ZnM81sFI5hpYZYbo_vfepf1Jho5TRgKYUk8eXFfCwP_gZRBbnHr0CiGNfmMq3Fzyzp1P3hrjCyPoBw$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[lambdaupb]

Ah, ok, the paper also shows a lot of variance between different attacked bytes – and similar results for third byte.
I do get single trace attacks with the CNN ID OPOI model though.

[AISyLab]

That is nice result. Of course what we did can be even further improved. What model do you use? Stjepan From: lambdaupb ***@***.***> Sent: Wednesday, May 10, 2023 11:01 AM To: AISyLab/feature_selection_dlsca ***@***.***> Cc: AISyLab @ TU Delft ***@***.***>; Comment ***@***.***> Subject: Re: [AISyLab/feature_selection_dlsca] CHES_CTF Dataset: Can't see reason for applied 800 shift during dataset processing in raw data (Issue #2) Ah, ok, the paper also shows a lot of variance between different attacked bytes – and similar results for third byte. I do get single trace attacks with the CNN ID OPOI model though. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/AISyLab/feature_selection_dlsca/issues/2*issuecomment-1541621965__;Iw!!PAKc-5URQlI!5MOGURBqjbZpzXmJuQ6zww0m1rhNhN03ED0RB0FZ7TKf_u81-UdigStFiS5njKh6h65vibeh3S82TJeTMD0jWJDKQQ$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/APDG756XVM44NRWAQQT53LDXFNKNRANCNFSM6AAAAAAXY7LITI__;!!PAKc-5URQlI!5MOGURBqjbZpzXmJuQ6zww0m1rhNhN03ED0RB0FZ7TKf_u81-UdigStFiS5njKh6h65vibeh3S82TJeTMD3W3vo8Rg$>. You are receiving this because you commented.Message ID: ***@***.***>

[lambdaupb]

That is nice result. Of course what we did can be even further improved. What model do you use?

feature_selection_dlsca/experiments/CHESCTF/OPOI/best_models.py

Lines 72 to 99 in eb5b909

	def best_cnn_id_opoi_4000_chesctf(classes, number_of_samples):
	# Best Convolutional Neural Network for CHESCTF key dataset
	# Number of points-of-interest: 4000
	# Leakage model: ID
	# Number of parameters: 593780
	batch_size = 100
	tf.random.set_seed(855016)
	model = Sequential(name="best_cnn_id_rpoi_4000_chesctf")
	model.add(Conv1D(kernel_size=18, strides=8, filters=4, activation="selu", input_shape=(number_of_samples, 1), padding="same"))
	model.add(MaxPool1D(pool_size=8, strides=8, padding="same"))
	model.add(BatchNormalization())
	model.add(Conv1D(kernel_size=19, strides=4, filters=8, activation="selu", padding="same"))
	model.add(MaxPool1D(pool_size=2, strides=2, padding="same"))
	model.add(BatchNormalization())
	model.add(Conv1D(kernel_size=18, strides=9, filters=16, activation="selu", padding="same"))
	model.add(MaxPool1D(pool_size=6, strides=6, padding="same"))
	model.add(BatchNormalization())
	model.add(Flatten())
	model.add(Dense(400, activation="selu", kernel_initializer="he_uniform"))
	model.add(Dense(400, activation="selu", kernel_initializer="he_uniform"))
	model.add(Dense(400, activation="selu", kernel_initializer="he_uniform"))
	model.add(Dense(400, activation="selu", kernel_initializer="he_uniform"))
	model.add(Dense(classes, activation="softmax"))
	model.summary()
	optimizer = Adam(learning_rate=0.0005)
	model.compile(loss="categorical_crossentropy", optimizer=optimizer, metrics=["accuracy"])
	return model, batch_size

Your opoi cnn id model. (I removed the fixed seed though)

I think this 800 shift might have lead to poor results.

[AISyLab]

Possibly. We will also check it a bit. Thanks. Stjepan From: lambdaupb ***@***.***> Sent: Wednesday, May 10, 2023 11:07 AM To: AISyLab/feature_selection_dlsca ***@***.***> Cc: AISyLab @ TU Delft ***@***.***>; Comment ***@***.***> Subject: Re: [AISyLab/feature_selection_dlsca] CHES_CTF Dataset: Can't see reason for applied 800 shift during dataset processing in raw data (Issue #2) That is nice result. Of course what we did can be even further improved. What model do you use? https://github.com/AISyLab/feature_selection_dlsca/blob/eb5b909f8d070faa7ae69051e016acaa6dd5f12d/experiments/CHESCTF/OPOI/best_models.py#L72-L99<https://urldefense.com/v3/__https:/github.com/AISyLab/feature_selection_dlsca/blob/eb5b909f8d070faa7ae69051e016acaa6dd5f12d/experiments/CHESCTF/OPOI/best_models.py*L72-L99__;Iw!!PAKc-5URQlI!9bSJ-tKtw9Cczye81t4kUjiyARd6j6TYzYiYBDf9CtSEoPvFv18qcMH9RN1YI1wzXyrY6l8ydPGi71UeyuZlYH8m6w$> Your opoi cnn id model. I think this 800 shift might have lead to poor results. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/AISyLab/feature_selection_dlsca/issues/2*issuecomment-1541631182__;Iw!!PAKc-5URQlI!9bSJ-tKtw9Cczye81t4kUjiyARd6j6TYzYiYBDf9CtSEoPvFv18qcMH9RN1YI1wzXyrY6l8ydPGi71Ueyubc1Rn3Yw$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/APDG752PJP3OXQQOCMM44PLXFNLCDANCNFSM6AAAAAAXY7LITI__;!!PAKc-5URQlI!9bSJ-tKtw9Cczye81t4kUjiyARd6j6TYzYiYBDf9CtSEoPvFv18qcMH9RN1YI1wzXyrY6l8ydPGi71UeyuZtrZdn9w$>. You are receiving this because you commented.Message ID: ***@***.***>

[lambdaupb]

Disregard this issue, I had a bug in my dataset reading code and the same file was read 4 times instead of the actual different files. :(

I apologize for the wasted time on your end.
I can see the same shift now.

[lambdaupb]

After some more investigation I have found out that for some reason, specifically the first trace in the file PinataAcqTask2.2_10k_upload.trs is offset, but the remaining traces in that file are normal.

Here is a cumulative plot of all 40_000 traces, without applying any offset:

Easy to see are the few outliers, one of them the first trace in PinataAcqTask2.2_10k_upload.trs.

The flatlines are traces [6_025:6_048] in the same file.

I re-ran some attacks and the OPOI ID CNN for this dataset did not work, contrary to my earlier claims with the buggy dataset.

This means the 800 offset is indeed wrong as is applied to all 10'000 traces despite only a handful are actual outliers.