Merge pull request #16433 from CartoDB/bug/sc-246788/cc2-sync-failing-randomly-3

NEWS.md

@@ -98,6 +98,7 @@ Development
 - Add retry if a timeout is thrown when swapping the tables related with a sync process [#16430](https://github.com/CartoDB/cartodb/pull/16430)
 - Add AUTODETECT_SIZE_LIMIT to ogr2ogr process when guessing CSV file column types [#16431](https://github.com/CartoDB/cartodb/pull/16431)
 - Log pg locks if there is any problem during a sync table import process [#16432](https://github.com/CartoDB/cartodb/pull/16432)
+- Check pg locks during sync table swap and terminate locking queries [#16433](https://github.com/CartoDB/cartodb/pull/16433)
 
 4.45.0 (2021-04-14)
 -------------------

app/models/synchronization/adapter.rb

@@ -394,9 +394,9 @@ module CartoDB
         end
       rescue Exception => exception
         if exception.message.include?('canceling statement due to statement timeout')
-          # Check if the table has any lock
-          lock = user.in_database(as: :superuser).fetch(%Q{
-            SELECT pid, state, usename, query, query_start 
+          # Check if the table has any lock and cancel locking queries
+          locks = user.in_database(as: :superuser).fetch(%Q{
+            SELECT pid, query 
             FROM pg_stat_activity 
             WHERE pid in (
               SELECT pid FROM pg_locks l 
@@ -404,8 +404,14 @@ module CartoDB
               AND t.relkind = 'r' 
               WHERE t.relname IN ('#{table_name}')
             );
-          }).first
-          @logger.append_and_store "Transaction timed out as the table is blocked by query #{lock[:query]}. Retrying in 60 seconds..." if @logger && lock.present?
+          }).all
+          @logger.append_and_store "Transaction timed out as the table is blocked by other queries. Terminating locking queries and retrying in 60 seconds..." if @logger && locks.present?
+          locks.each do |lock|
+              @logger.append_and_store "Terminating query: #{lock[:query]}" if @logger
+              user.in_database(as: :superuser).execute %Q{
+              SELECT pg_terminate_backend(#{lock[:pid]});
+            }
+          end
           sleep(60) # wait 60 seconds and retry the swap
           database.transaction do
             rename(table_name, temporary_name) if exists?(table_name)