Postgres - 大jsonb列的性能

Question

我们在我们的数据库表中使用了Postgres jsonb类型。表结构如下所示：

CREATE TABLE T (
    id UUID NOT NULL PRIMARY KEY, 
    payload JSONB 
); 

CREATE INDEX ON T USING gin (payload jsonb_path_ops); 

Payload是一个复杂的JSON字符串。下面是一个例子：

{ 
    "business": { 
     "taxId": "626642071", 
     "legalName": "Jon's Deli", 
     "phoneNumbers": [ 
      { 
       "phoneType": "Business", 
       "telephoneNumber": "8384407555" 
      }, 
      { 
       "phoneType": "Work", 
       "telephoneNumber": "6032255248" 
      } 
     ], 
     "addresses": [ 
      { 
       "city": "San Francisco", 
       "state": "CA", 
       "postalCode": "94101", 
       "countryCode": "USA", 
       "addressLine1": "123 Market St" 
      } 
     ] 
    }, 
    "stakeholders": [ 
     { 
      "person": { 
       "taxId": "540646815", 
       "firstName": "GdXFouh", 
       "lastName": "IlUAcgCGz", 
       "dateOfBirth": "1980-12-11", 
       "emailAddress": "jywxsijgix@qaqmlz.com", 
       "phoneNumbers": [ 
        { 
         "phoneType": "Mobile", 
         "telephoneNumber": "4901371573" 
        } 
       ], 
       "addresses": [ 
        { 
         "city": "San Francisco", 
         "state": "CA", 
         "postalCode": "94101", 
         "countryCode": "USA", 
         "addressLine1": "123 Market St" 
        } 
       ] 
      } 
     } 
    ] 
} 

注意phoneNumbers，addresses和stakeholders是数组，这意味着可以有阵列中的多个元素。

我尝试在表中插入一百万行。生成​​的每个字段随机。最初测试程序运行速度非常快。但插入大约800,000行后，每隔1000行就卡住 - 插入1000行，然后测试程序挂起2分钟，然后返回并插入另外1000行，...

我们怀疑这个是由大量的jsonb索引更新引起的。因为单行索引中有很多字段需要更新。我们只是想确认是否有人遇到同样的问题。

---

其实我们并不需要索引整个​​列。只需要某些字段：business->taxId,business->phoneNumbers-> telephoneNumber,stakeholders->person->taxId和stakeholders->person->emailAddress。

我已经尝试了以下两个指标：

CREATE INDEX ON T USING gin ((payload->'business'->'taxId') jsonb_path_ops); 
CREATE INDEX ON T USING gin ((payload ->'stakeholders'->'person'->'taxId') jsonb_path_ops); 

并运行两个语句：

explain select * from T where payload->'business'->'taxId' @> '"123456789"'; (1) 
explain select * from T where payload->'stakeholders'->'person'->'taxId' @> '"123456789"'; (2) 

第一个语句使用索引。但第二个是做一个非常缓慢的全表扫描。这就是为什么我们转向索引整个​​列。

---

任何建议是值得欢迎的。

顺便说一句，我们正在使用Postgres 9.5.4。

Answer 1

您的疑问：

select * from T where payload->'stakeholders'->'person'->'taxId' @> '"123456789"'; 

不起作用。这是因为'利益相关者'是数组。工作查询是：

select * from T where payload->'stakeholders' @> '[{"person": {"taxId": "54"}}]'::jsonb 

但在这种情况下，postgres可以使用整个利益相关者的使用指数。

             QUERY PLAN              
------------------------------------------------------------------------------------------------------------------------ 
Bitmap Heap Scan on t (cost=1388.08..1425.90 rows=10 width=36) (actual time=1.959..1.959 rows=1 loops=1) 
    Recheck Cond: ((payload -> 'stakeholders'::text) @> '[{"person": {"taxId": "54"}}]'::jsonb) 
    Heap Blocks: exact=1 
    -> Bitmap Index Scan on t_expr_idx3 (cost=0.00..1388.08 rows=10 width=0) (actual time=1.946..1.946 rows=1 loops=1) 
     Index Cond: ((payload -> 'stakeholders'::text) @> '[{"person": {"taxId": "54"}}]'::jsonb) 
Planning time: 0.071 ms 
Execution time: 1.978 ms 

对于使用更具体的指标我用修改的办法：How do you create a Postgresql JSONB array in array index?

CREATE OR REPLACE FUNCTION extract_taxids(a_json jsonb). 
RETURNS jsonb AS $BODY$ 
    SELECT jsonb_agg(j) FROM (SELECT jsonb_array_elements(a_json->'stakeholders')->'person'->'taxId' AS j) AS j 
$BODY$ LANGUAGE sql IMMUTABLE; 
CREATE INDEX ON T USING gin (extract_taxids(payload)); 

而且vuala：

EXPLAIN ANALYZE select * from T where extract_taxids(payload) @> '["54"]'; 

                  QUERY PLAN               
-------------------------------------------------------------------------------------------------------------------------------- 
Bitmap Heap Scan on t (cost=12.08..52.38 rows=10 width=36) (actual time=0.101..0.102 rows=1 loops=1) 
    Recheck Cond: (extract_taxids(payload) @> '["54"]'::jsonb) 
    Heap Blocks: exact=1 
    -> Bitmap Index Scan on t_extract_taxids_idx (cost=0.00..12.07 rows=10 width=0) (actual time=0.008..0.008 rows=1 loops=1) 
     Index Cond: (extract_taxids(payload) @> '["54"]'::jsonb) 
Planning time: 0.128 ms 
Execution time: 0.117 ms