多个CSV文件复制到Postgres的

Question

我写一个SQL脚本多个.CSV文件复制到一个Postgres数据库是这样的：

COPY product(title, department) from 'ys.csv' CSV HEADER; 

我有多个文件，我想复制我不想：

COPY product(title, department) from 'ys1.csv' CSV HEADER; 
COPY product(title, department) from 'ys2.csv' CSV HEADER; 
COPY product(title, department) from 'ys3.csv' CSV HEADER; 
COPY product(title, department) from 'ys4.csv' CSV HEADER; 
COPY product(title, department) from 'ys5.csv' CSV HEADER; 

我想用for循环来代替多个拷贝命令。这可能吗？谢谢

Answer 1

在linux中管道输出文件列表到psql。让copy使用标准输入：

cat /path_to/ys*.csv | psql -c 'COPY product(title, department) from stdin CSV HEADER' 

寻找其他操作系统

Answer 2

相当于我尝试了上述答案，但有一个以上的文件，工作时我得到了一个错误。我认为在第二个文件没有切断标题。

这工作FOT我：

# get filenames 
IMPFILES=(path/FileNamepart.csv) 

# import the files 
for i in ${IMPFILES[@]} 
    do 
     psql -U user -d database -c "\copy TABLE_NAME from '$i' DELIMITER ';' CSV HEADER" 
     # move the imported file 
     mv $i /FilePath 
    done 

在我来说，我每移动文件AFER是进口的。如果发生错误，我知道在哪里寻找。如果有新文件放在该位置，我可以再次运行该脚本。

Answer 3

您可以使用pg_ls_dir遍历文件名。

DO $$ 

DECLARE file_path TEXT; -- Path where your CSV files are 
DECLARE fn_i TEXT; -- Variable to hold name of current CSV file being inserted 
DECLARE mytable TEXT; -- Variable to hold name of table to insert data into 

BEGIN 

    file_path := 'C:/Program Files/PostgreSQL/9.6/data/my_csvs/'; -- Declare the path to your CSV files. You probably need to put this in your PostgreSQL file path to avoid permission issues. 
    mytable := 'product(title,department)'; -- Declare table to insert data into. You can give columns too since it's just going into an execute statement. 

    CREATE TEMP TABLE files AS 
    SELECT file_path || pg_ls_dir AS fn -- get all of the files in the directory, prepending with file path 
    FROM pg_ls_dir(file_path); 

    LOOP  
     fn_i := (select fn from files limit 1); -- Pick the first file 
     raise notice 'fn: %', fn_i; 
     EXECUTE 'COPY ' || mytable || ' from ''' || fn_i || ''' with csv header'; 
     DELETE FROM files WHERE fn = fn_i; -- Delete the file just inserted from the queue 
     EXIT WHEN (SELECT COUNT(*) FROM files) = 0; 
    END LOOP; 

END $$;