feat: Add PostgreSQL REAL/FLOAT4 support to Sourcedb to Spanner template#3870
Conversation
…ostgreSQLDataTypesPGDialectIT
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3870 +/- ##
============================================
+ Coverage 55.47% 63.31% +7.84%
+ Complexity 6962 2315 -4647
============================================
Files 1102 514 -588
Lines 67443 29932 -37511
Branches 7564 3298 -4266
============================================
- Hits 37413 18952 -18461
+ Misses 27587 10017 -17570
+ Partials 2443 963 -1480
🚀 New features to boost your workflow:
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enables support for migrating PostgreSQL REAL (FLOAT4) data types within the Sourcedb to Spanner template. By establishing the necessary type mappings and validating them through both unit and integration tests, the changes ensure that single-precision floating-point data is correctly processed and persisted in Spanner when using the PostgreSQL dialect. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request enables mapping support for PostgreSQL float4 (real) to Spanner float32 in AvroToValueMapper and re-enables the corresponding integration tests in PostgreSQLDataTypesPGDialectIT. Feedback suggests removing obsolete comments in the integration tests that claim the float4 tests are still commented out, and expanding the unit tests in AvroToValueMapperTest to cover null, NaN, and infinity values as described in the PR description.
The
Sourcedb_to_Spanner_Flextemplate previously lacked support for migrating data from PostgreSQLREAL(aliased asFLOAT4) columns when using the Spanner PostgreSQL dialect. This resulted in data not being populated in the target Spanner tables for columns of typeFLOAT4(Spanner's single-precision float type), causing failures inPostgreSQLDataTypesPGDialectITfor these type tests.This PR addresses this by:
Type.pgFloat4()inAvroToValueMapper.getPgMap(). This mapping utilizes the existingavroFieldToFloat32function to correctly convert incoming Avro values (originating from PostgreSQLREALcolumns) to SpannerValue.float32().AvroToValueMapperTest.javato specifically validate theDialect.POSTGRESQLmap entries forType.pgFloat4(), ensuring the conversion logic is correctly wired.REALdata types withinPostgreSQLDataTypesPGDialectIT.javathat were previously commented out. These tests now pass, confirming end-to-end functionality.With these changes, the template now correctly handles PostgreSQL
REAL/FLOAT4types, mapping them to Spanner'sfloat4type in the PostgreSQL dialect.Fixes: b/516973815