如何搜索所有表（PostgreSQL）中的特定值？

Question

有类似的问题here，但它适用于Oracle。我对PostgreSQL有同样的问题。

简而言之，是否有可能在PostgreSQL中搜索每个表的每个字段以获取特定值？

谢谢。

Answer 1

如何转储数据库中的内容，然后使用grep？

$ pg_dump --data-only --inserts -U postgres your-db-name > a.tmp 
$ grep United a.tmp 
INSERT INTO countries VALUES ('US', 'United States'); 
INSERT INTO countries VALUES ('GB', 'United Kingdom'); 

相同的实用程序pg_dump可以在输出中包含列名。只需将--inserts更改为--column-inserts即可。这样您也可以搜索特定的列名称。但是如果我在寻找列名，我可能会转储模式而不是数据。

$ pg_dump --data-only --column-inserts -U postgres your-db-name > a.tmp 
$ grep country_code a.tmp 
INSERT INTO countries (iso_country_code, iso_country_name) VALUES ('US', 'United States'); 
INSERT INTO countries (iso_country_code, iso_country_name) VALUES ('GB', 'United Kingdom'); 

Answer 2

我知道它可以做到这一点的唯一工具是：SQL工作台/ J：http://www.sql-workbench.net/

一个Java/JDBC基础工具，它提供了一个特殊（专有）SQL“命令”，通过全搜索（或只是选择）数据库中的表：

http://www.sql-workbench.net/manual/wb-commands.html#command-search-data
http://www.sql-workbench.net/wbgrepdata_png.html

Answer 3

这是一个pl/pgsql函数，用于查找任何列包含特定值的记录。 它以文本格式搜索的值作为参数，要搜索的表名称数组（缺省为所有表）和模式名称数组（缺省为所有模式名称）。

它返回模式，表的名称，名称栏和伪列ctid的表结构（表中的行的非持久的物理位置，见System Columns）

CREATE OR REPLACE FUNCTION search_columns(
    needle text, 
    haystack_tables name[] default '{}', 
    haystack_schema name[] default '{}' 
) 
RETURNS table(schemaname text, tablename text, columnname text, rowctid text) 
AS $$ 
begin 
    FOR schemaname,tablename,columnname IN 
     SELECT c.table_schema,c.table_name,c.column_name 
     FROM information_schema.columns c 
     JOIN information_schema.tables t ON 
     (t.table_name=c.table_name AND t.table_schema=c.table_schema) 
     WHERE (c.table_name=ANY(haystack_tables) OR haystack_tables='{}') 
     AND (c.table_schema=ANY(haystack_schema) OR haystack_schema='{}') 
     AND t.table_type='BASE TABLE' 
    LOOP 
    EXECUTE format('SELECT ctid FROM %I.%I WHERE cast(%I as text)=%L', 
     schemaname, 
     tablename, 
     columnname, 
     needle 
    ) INTO rowctid; 
    IF rowctid is not null THEN 
     RETURN NEXT; 
    END IF; 
END LOOP; 
END; 
$$ language plpgsql; 

EDIT ：此代码适用于PG 9.1或更新版本。在测试数据库使用的

例子：

公共架构中的所有表中搜索：

 
select * from search_columns('foobar'); 
schemaname | tablename | columnname | rowctid 
------------+-----------+------------+--------- 
public  | s3  | usename | (0,11) 
public  | s2  | relname | (7,29) 
public  | w   | body  | (0,2) 
(3 rows) 

搜索特定表：

 
select * from search_columns('foobar','{w}'); 
schemaname | tablename | columnname | rowctid 
------------+-----------+------------+--------- 
public  | w   | body  | (0,2) 
(1 row) 

表中的一个子集搜索从一个选择中获得：

 
select * from search_columns('foobar', array(select table_name::name from information_schema.tables where table_name like 's%'), array['public']); 
schemaname | tablename | columnname | rowctid 
------------+-----------+------------+--------- 
public  | s2  | relname | (7,29) 
public  | s3  | usename | (0,11) 
(2 rows) 

得到一个结果行与相应的基台和和CTID：

 
select * from public.w where ctid='(0,2)'; 
title | body |   tsv   
-------+--------+--------------------- 
toto | foobar | 'foobar':2 'toto':1 

为了代替再次测试一个正则表达式的严格平等，如grep，这样：

SELECT ctid FROM %I.%I WHERE cast(%I as text)=%L 

可以改变为：

SELECT ctid FROM %I.%I WHERE cast(%I as text) ~ %L 

Answer 4

这里是@DanielVérité的功能与进展报告功能。 它报告三种方式的进展：

1. 通过RAISE NOTICE;
2. 通过将提供的{progress_seq}序列的值从 {要搜索的总列数}减小到0;
3. 通过将进度和找到的表一起写入位于c：\ windows \ temp \ {progress_seq} .txt中的文本文件 。

_

CREATE OR REPLACE FUNCTION search_columns(
    needle text, 
    haystack_tables name[] default '{}', 
    haystack_schema name[] default '{public}', 
    progress_seq text default NULL 
) 
RETURNS table(schemaname text, tablename text, columnname text, rowctid text) 
AS $$ 
DECLARE 
currenttable text; 
columnscount integer; 
foundintables text[]; 
foundincolumns text[]; 
begin 
currenttable=''; 
columnscount = (SELECT count(1) 
     FROM information_schema.columns c 
     JOIN information_schema.tables t ON 
     (t.table_name=c.table_name AND t.table_schema=c.table_schema) 
     WHERE (c.table_name=ANY(haystack_tables) OR haystack_tables='{}') 
     AND c.table_schema=ANY(haystack_schema) 
     AND t.table_type='BASE TABLE')::integer; 
PERFORM setval(progress_seq::regclass, columnscount); 

    FOR schemaname,tablename,columnname IN 
     SELECT c.table_schema,c.table_name,c.column_name 
     FROM information_schema.columns c 
     JOIN information_schema.tables t ON 
     (t.table_name=c.table_name AND t.table_schema=c.table_schema) 
     WHERE (c.table_name=ANY(haystack_tables) OR haystack_tables='{}') 
     AND c.table_schema=ANY(haystack_schema) 
     AND t.table_type='BASE TABLE' 
    LOOP 
    EXECUTE format('SELECT ctid FROM %I.%I WHERE cast(%I as text)=%L', 
     schemaname, 
     tablename, 
     columnname, 
     needle 
    ) INTO rowctid; 
    IF rowctid is not null THEN 
     RETURN NEXT; 
     foundintables = foundintables || tablename; 
     foundincolumns = foundincolumns || columnname; 
     RAISE NOTICE 'FOUND! %, %, %, %', schemaname,tablename,columnname, rowctid; 
    END IF; 
     IF (progress_seq IS NOT NULL) THEN 
     PERFORM nextval(progress_seq::regclass); 
    END IF; 
    IF(currenttable<>tablename) THEN 
    currenttable=tablename; 
    IF (progress_seq IS NOT NULL) THEN 
     RAISE NOTICE 'Columns left to look in: %; looking in table: %', currval(progress_seq::regclass), tablename; 
     EXECUTE 'COPY (SELECT unnest(string_to_array(''Current table (column ' || columnscount-currval(progress_seq::regclass) || ' of ' || columnscount || '): ' || tablename || '\n\nFound in tables/columns:\n' || COALESCE(
     (SELECT string_agg(c1 || '/' || c2, '\n') FROM (SELECT unnest(foundintables) AS c1,unnest(foundincolumns) AS c2) AS t1) 
     , '') || ''',''\n''))) TO ''c:\WINDOWS\temp\' || progress_seq || '.txt'''; 
    END IF; 
    END IF; 
END LOOP; 
END; 
$$ language plpgsql; 

Answer 5

如果有人认为这可能会有帮助。这里是@DanielVérité的函数，另一个参数接受可用于搜索的列的名称。这样可以减少处理时间。至少在我的测试中，它减少了很多。

CREATE OR REPLACE FUNCTION search_columns(
    needle text, 
    haystack_columns name[] default '{}', 
    haystack_tables name[] default '{}', 
    haystack_schema name[] default '{public}' 
) 
RETURNS table(schemaname text, tablename text, columnname text, rowctid text) 
AS $$ 
begin 
    FOR schemaname,tablename,columnname IN 
     SELECT c.table_schema,c.table_name,c.column_name 
     FROM information_schema.columns c 
     JOIN information_schema.tables t ON 
     (t.table_name=c.table_name AND t.table_schema=c.table_schema) 
     WHERE (c.table_name=ANY(haystack_tables) OR haystack_tables='{}') 
     AND c.table_schema=ANY(haystack_schema) 
     AND (c.column_name=ANY(haystack_columns) OR haystack_columns='{}') 
     AND t.table_type='BASE TABLE' 
    LOOP 
    EXECUTE format('SELECT ctid FROM %I.%I WHERE cast(%I as text)=%L', 
     schemaname, 
     tablename, 
     columnname, 
     needle 
    ) INTO rowctid; 
    IF rowctid is not null THEN 
     RETURN NEXT; 
    END IF; 
END LOOP; 
END; 
$$ language plpgsql; 

波纹管是上面创建的search_function的使用示例。

SELECT * FROM search_columns('86192700' 
    , array(SELECT DISTINCT a.column_name::name FROM information_schema.columns AS a 
      INNER JOIN information_schema.tables as b ON (b.table_catalog = a.table_catalog AND b.table_schema = a.table_schema AND b.table_name = a.table_name) 
     WHERE 
      a.column_name iLIKE '%cep%' 
      AND b.table_type = 'BASE TABLE' 
      AND b.table_schema = 'public' 
    ) 

    , array(SELECT b.table_name::name FROM information_schema.columns AS a 
      INNER JOIN information_schema.tables as b ON (b.table_catalog = a.table_catalog AND b.table_schema = a.table_schema AND b.table_name = a.table_name) 
     WHERE 
      a.column_name iLIKE '%cep%' 
      AND b.table_type = 'BASE TABLE' 
      AND b.table_schema = 'public') 
); 

Answer 6

不存储新过程，您可以使用代码块并执行以获取出现的表。您可以按架构，表或列名称筛选结果。

DO $$ 
DECLARE 
    value int := 0; 
    sql text := 'The constructed select statement'; 
    rec1 record; 
    rec2 record; 
BEGIN 
    DROP TABLE IF EXISTS _x; 
    CREATE TEMPORARY TABLE _x (
    schema_name text, 
    table_name text, 
    column_name text, 
    found text 
); 
    FOR rec1 IN 
     SELECT table_schema, table_name, column_name 
     FROM information_schema.columns 
     WHERE table_name <> '_x' 
       AND UPPER(column_name) LIKE UPPER('%%')     
       AND table_schema <> 'pg_catalog' 
       AND table_schema <> 'information_schema' 
       AND data_type IN ('character varying', 'text', 'character', 'char', 'varchar') 
     LOOP 
    sql := concat('SELECT ', rec1."column_name", ' AS "found" FROM ',rec1."table_schema" , '.',rec1."table_name" , ' WHERE UPPER(',rec1."column_name" , ') LIKE UPPER(''','%my_substring_to_find_goes_here%' , ''')'); 
    RAISE NOTICE '%', sql; 
    BEGIN 
     FOR rec2 IN EXECUTE sql LOOP 
      RAISE NOTICE '%', sql; 
      INSERT INTO _x VALUES (rec1."table_schema", rec1."table_name", rec1."column_name", rec2."found"); 
     END LOOP; 
    EXCEPTION WHEN OTHERS THEN 
    END; 
    END LOOP; 
    END; $$; 

SELECT * FROM _x;