In some use cases, apis require batched data to be submitted as a raw string rather than wrapped in a JSON array.
Currently the only way to send batched raw data strings is to use the single request mode and create a pre aggregated roll up payload as a string before sinking. This operation is a bit obtuse.
|
BodyPublisher publisher; |
|
// By default, Java's BodyPublishers.ofByteArrays(elements) will just put Jsons |
|
// into the HTTP body without any context. |
|
// What we do here is we pack every Json/byteArray into Json Array hence '[' and ']' |
|
// at the end, and we separate every element with comma. |
|
elements.add(BATCH_START_BYTES); |
|
for (HttpSinkRequestEntry entry : reqeustBatch) { |
|
elements.add(entry.element); |
|
elements.add(BATCH_ELEMENT_DELIM_BYTES); |
|
} |
|
elements.set(elements.size() - 1, BATCH_END_BYTES); |
|
publisher = BodyPublishers.ofByteArrays(elements); |
Please consider adding this functionality in the near future.
In some use cases, apis require batched data to be submitted as a raw string rather than wrapped in a JSON array.
Currently the only way to send batched raw data strings is to use the single request mode and create a pre aggregated roll up payload as a string before sinking. This operation is a bit obtuse.
flink-http-connector/src/main/java/com/getindata/connectors/http/internal/sink/httpclient/BatchRequestSubmitter.java
Lines 111 to 122 in b95e7fb
Please consider adding this functionality in the near future.