Skip to content

fix(catalog/s3tables): stop dropping endpoint_url for empty properties#2623

Open
Kurtiscwright wants to merge 1 commit into
apache:mainfrom
Kurtiscwright:fix/s3tables-endpoint-url
Open

fix(catalog/s3tables): stop dropping endpoint_url for empty properties#2623
Kurtiscwright wants to merge 1 commit into
apache:mainfrom
Kurtiscwright:fix/s3tables-endpoint-url

Conversation

@Kurtiscwright

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

This PR removes the properties.is_empty() check from the s3tables create_sdk_config. This check was causing an empty properties to be return even when an endpoint_url was provided because the endpoint wasn't passed into properties before this check ran.

Dropping the check seemed like the best approach because it only saved three (credentials, profile, region, and endpoint) checks at Catalog start up. If performance testing shows this skip is needed in the future than it can be added back in.

Are these changes tested?

Added new unit test to prevent any regressions on future PRs.

This PR removes the `properties.is_empty()` check from the s3tables
`create_sdk_config`. This check was causing an empty properties to be
return even when an endpoint_url was provided because the endpoint
wasn't passed into properties before this check ran.

Dropping the check seemed like the best approach because it only saved
three (credentials, profile, region, and endpoint) checks at Catalog
start up. If performance testing shows this skip is needed in the future
than it can be added back in.

Added new unit test to prevent any regressions on future PRs.
if properties.is_empty() {
return config.load().await;
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just move this block to after the endpoint_url check?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered that approach, but I chose the current implementation for two reasons:

  1. It read like a premature optimization to me to have this check. This validation is only ran at Catalog load time so once per session and the number of checks are only 4 at the moment.
  2. Keeping this check leaves this same class of bug for any future property flag implementations.

For both of the above reasons. I felt it was better to remove the check altogether rather than shuffle it further down. I can add it back in if there is strong disagreement with my two reasons listed above.

@dannycjones dannycjones left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion on moving or dropping the is_empty check, thanks for the fix and test!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3Tables Catalog drops endpoint when no other properties are provided.

3 participants