Training error

[darkasevgen]

Hi @LeslieZhoa ! How can you compare the dictionary and the float?

DCT-NET.Pytorch/trainer/ModelTrainer.py

Line 124 in b303f41

if loss < mn_loss:

This method is called here:

DCT-NET.Pytorch/trainer/ModelTrainer.py

Line 96 in b303f41

acc_num,mn_loss,stop_flag = self.early_stop_wait(self.get_loss_from_val(val_loss),acc_num,mn_loss,epoch)

But the val_loss obtained in CCNTrainer.evalution is an empty dictionary:

DCT-NET.Pytorch/trainer/CCNTrainer.py

Lines 196 to 206 in b303f41

	def evalution(self,test_loader,steps,epoch):
	loss_dict = {}
	with torch.no_grad():
	fake_s,_ = self.netGs([self.sample_z])
	fake_t,_ = self.gt_ema([self.sample_z])
	if self.args.rank == 0 :
	self.val_vis.display_current_results(self.select_img([fake_s,fake_t]),steps)
	# self.val_vis.display_current_results(self.select_img([fake_t]),steps)
	return loss_dict

I want to understand how to fix this and what metrics to count? And as I understand it, you trained the network without early_stopping. How many epochs/iterations did it take?

[LeslieZhoa]

Hi @darkasevgen !
I didn't use early stop. It is just a template. If you want to use it, just modify the evalution function and get_loss_from_val.
As described by Readme，when you train ccn， almost 1000 steps, you can stop it.
When you train ttn, you can stop by monitoring its condition. Maybe it is 3w steps.