Question: configuring scaled_dot_product_attention

[pfeatherstone]

it looks like from

recurrent-memory-transformer-pytorch/recurrent_memory_transformer_pytorch/attend.py

Lines 62 to 67 in 98bf309

	if device_properties.major == 8 and device_properties.minor == 0:
	print_once('A100 GPU detected, using flash attention if input tensor is on cuda')
	self.cuda_config = Config(True, False, False)
	else:
	print_once('Non-A100 GPU detected, using math or mem efficient attention if input tensor is on cuda')
	self.cuda_config = Config(False, True, True)

and

recurrent-memory-transformer-pytorch/recurrent_memory_transformer_pytorch/attend.py

Lines 93 to 99 in 98bf309

	with torch.backends.cuda.sdp_kernel(**config._asdict()):
	out = F.scaled_dot_product_attention(
	q, k, v,
	attn_mask = mask,
	dropout_p = self.dropout if self.training else 0.,
	is_causal = self.causal
	)

we manually configure F.scaled_dot_product_attention().
From the documentation it says "All implementations are enabled by default. Scaled dot product attention attempts to automatically select the most optimal implementation based on the inputs."
Can't we just let pytorch decide?