Commit after truncate to reduce lock contention

Truncate takes an `AccessExclusive` lock on the table, which blocks all
other transactions from accessing the table, including select for
datastore_info and any other access to the table contents. Since those
operations are called from the web interface, the UI can hang for the
timeout period or untill the copy finishes. This is poor user
experience when dealing with large files.

So, truncate in it's own transaction, so that the AccessExclusive lock
is held for the least amount of time possible.

Author: EricSoroos

ckanext/datapusher_plus/jobs.py

@@ -1053,22 +1053,23 @@ def _push_to_datastore(
     else:
         cur = raw_connection.cursor()
 
-        # truncate table to use copy freeze option and further increase
-        # performance as there is no need for WAL logs to be maintained
-        # https://www.postgresql.org/docs/current/populate.html#POPULATE-COPY-FROM
+        # truncate table in case we're loading over an existing resource
         try:
             cur.execute(
                 sql.SQL("TRUNCATE TABLE {}").format(sql.Identifier(resource_id))
             )
-
+            # commit to ensure that the AccessExclusive lock is only held for the
+            # duration of the truncate, otherwise no other access to the table is
+            # allowed, blocking all selects. 
+            raw_connection.commit()
         except psycopg2.Error as e:
             logger.warning(f"Could not TRUNCATE: {e}")
 
         col_names_list = [h["id"] for h in headers_dicts]
         column_names = sql.SQL(",").join(sql.Identifier(c) for c in col_names_list)
         copy_sql = sql.SQL(
             "COPY {} ({}) FROM STDIN "
-            "WITH (FORMAT CSV, FREEZE 1, "
+            "WITH (FORMAT CSV, "
             "HEADER 1, ENCODING 'UTF8');"
         ).format(
             sql.Identifier(resource_id),