There are three kinds of double-quoted strings

[jtv]

This will have its consequences for libpqxx support. I found the topic confusing, documentation didn't give me certainty, so I took some time to get clarity. I tried both psql and libpqxx calls.

Here's what I found:

Identifiers

Table names and such can be written in double quotes. I tested this by
creating a table with a quoted column name, and using PQfname().

• Doubling a double-quote escapes it: """" reads as a string containing
  one double-quote.
• Backslash is just a character. It does not escape anything.
• When standard_conforming_strings is disabled, neither seems to change.

Hstore strings

Keys and values in hstore entries should be in double quotes if they contain
certain unusual characters. (In practice, backslash-escaping works fine in
unquoted strings as well but there may be characters it does not support.)

This is a bit harder to reason about, since I'm writing these double-quoted
strings inside single-quoted strings inside C++ string literals. But AFAIK a
double-quote character is just not special at all inside a single-quoted
string.

I ran SELECT ($1::hstore)[1] with various values for parameter $1:
1=>ab, 1=>"a\"b", etc.

• Doubling a double-quote seems to be meaningless. The first one closes the
  string and you get a syntax error.
• Backslash escapes a double-quote.
• When standard_conforming_strings is disabled, neither seems to change.

Array entries

The situation with array entries is similar: entries can be in double quotes.

• Doubling a double-quote seems to be meaningless. The first one closes the
  string and you get a syntax error.
• Backslash escapes a double-quote.
• When standard_conforming_strings is disabled, neither seems to change.

Composite entries

Values within a composite are different.

• Doubling a double-quote seems to escape it.
• Backslash also escapes a double quote.
• When standard_conforming_strings is disabled, backslash no longer works as an escape character — but confusingly you get an error message saying you should use backslash as an escape character:

BEGIN;
SET standard_conforming_strings = on;  -- (This is the default)
CREATE TYPE foo AS (x text);
SELECT '("a""b")'::foo;  -- OK: ("a""b")
SELECT '("a\"b")'::foo;  -- OK: ("a""b")
SET standard_conforming_strings = off;  -- (Going back to pre-9.1 nonstandard behaviour)
SELECT '("a""b")'::foo;  -- OK: ("a""b")
-- Now we need to double the escaping backslash to get around the nonstandard string handling.
SELECT '("a\\"b")'::foo;  -- OK: ("a ""b") (with a warning).
SELECT '("a\"b")'::foo;  -- ERROR: malformed record literal
ROLLBACK;

That last check fails with:

WARNING:  nonstandard use of escape in a string literal
LINE 1: SELECT '("a\"b")'::foo;
               ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
ERROR:  malformed record literal: "("a"b")"
LINE 1: SELECT '("a\"b")'::foo;
               ^
DETAIL:  Unexpected end of input.

Conclusion

There seem to be 3 kinds of double-quoted strings: identifier strings, value strings, and composite field strings.

• Identifier strings use SQL-style escaping: to embed a double-quote character, you write two double-quote characters. Backslash has no special meaning.
• Value strings (as in array entries and hstore keys/values) use the backslash as an escape character. Writing two double-quote characters has no special meaning (and in fact if you wanted two double-quotes you'd need to escape them both with backslashes).
• Composite value strings support both SQL-style escaping (by repeating the quote character) and backslash-escaping, but the latter only if standard_conforming_strings is enabled (as is the default).

I don't think libpqxx needs a parser for identifier strings anywhere, but it does for the other kinds. Do we need specialised parsers for some of these different cases?

[jtv]

Claude's notes on where the parsing happens, abridged:

Arrays — src/backend/utils/adt/arrayfuncs.c

• ArrayCount() — first pass, counts dimensions and elements.
• ReadArrayStr() — second pass, does the actual extraction and unescaping.

Backslash escapes, doubling a quote does not.

Composite types — src/backend/utils/adt/rowtypes.c

• record_in() — single-pass parser.

Backslash escapes, but the preferred means of escaping a double-quote character is to double it.

hstore — contrib/hstore/hstore_io.c

• hstore_in() — calls parse_hstore().

Backslash escapes, doubling a quote does not.

Conclusion

The composite parser is the odd one out.

Each type of value string always occurs inside a single-quoted string, and that is the only place here where standard_conforming_strings is relevant. That processing happens before any of these double-quoted-string parsers come into play.

[jtv]

Knowing all this, I did see the odd situation with composite fields documented.

Next question: will I need to specialise the parsing of double-quoted strings to these 3 situations? In libpqxx I currently only have one parser, which accepts both kinds of escaping.

That's almost fine:

• I'm not aware of any situation where libpqxx would need to parse a double-quoted identifier.
• The existing parser would actually seem to be correct for double-quoted composite values.
• In arrays and hstores, the existing parser accepts a doubled quote as a form of escaping — which it shouldn't.

So we do need to specialise the double-quoted-strings parser for composite fields, and therefore, the whole composite-fields parser. 😦 In the hopefully exceedingly rare case where an array value contains two consecutive double-quote characters, it will currently parse those as a single double-quote. It should take the input literally as two double-quotes.

For composite values, we still want the existing parser. It may be easiest to indicate the distinction through a template parameter.

[github-actions]

\ There has been no activity on this ticket. Consider closing it.