training with 1 GPU, tensorflow_gpu-2.3.1, transformers-3.3.1 , an error occurred: y_pred.extend(pred.values[gpu]) in ust.py
WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `experimental_run_v2` inside a tf.function to get the best performance.
WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `experimental_run_v2` inside a tf.function to get the best performance.
Traceback (most recent call last):
File "run_ust.py", line 147, in <module>
train_model(max_seq_length, X_train, y_train, X_test, y_test, X_unlabeled, model_dir, tokenizer, sup_batch_size=sup_batch_size, unsup_batch_size=unsup_batch_size, unsup_size=unsup_size, sample_size=sample_size, TFModel=TFModel, Config=Config, pt_teacher_checkpoint=pt_teacher_checkpoint, sample_scheme=sample_scheme, T=T, alpha=alpha, valid_split=valid_split, sup_epochs=sup_epochs, unsup_epochs=unsup_epochs, N_base=N_base, dense_dropout=dense_dropout, attention_probs_dropout_prob=attention_probs_dropout_prob, hidden_dropout_prob=hidden_dropout_prob)
File "UST/ust.py", line 183, in train_model
y_mean, y_var, y_pred, y_T = mc_dropout_evaluate(model, gpus, len(labels), X_unlabeled_sample, T=T)
File "UST/ust.py", line 62, in mc_dropout_evaluate
y_pred.extend(pred.values[gpu])
AttributeError: 'numpy.ndarray' object has no attribute 'values'
[[-0.59470797 -0.7103901 -0.4074441 2.2519145 -1.8890193 1.024729 ]
[ 0.79591113 0.87572926 -1.0720805 -0.9207117 -0.32005262 0.6711779 ]
[-0.3771792 -0.71912414 -1.1747787 2.134624 -1.006975 0.3743801 ]
...
[-0.5568391 -0.1446489 -0.8823348 1.9092964 -1.0569383 0.17100161]
[ 0.9426691 -0.87104434 0.3349641 0.87110806 -1.3404613 0.5784961 ]
[ 0.7570472 -1.1358421 0.9814421 -0.84206074 0.06219336 -0.15020499]]
training with 1 GPU, tensorflow_gpu-2.3.1, transformers-3.3.1 , an error occurred:
y_pred.extend(pred.values[gpu])in ust.pyactual: the value of pred is: (by
print(pred))