在分区上进行联接与在整个表上进行联接时,以下联接具有非常不同的行估计:
CREATE TABLE m_data.ga_session (
session_id BIGINT NOT NULL,
visitor_id BIGINT NOT NULL,
transaction_id TEXT,
timestamp TIMESTAMP WITH TIME ZONE NOT NULL,
day_id INTEGER NOT NULL,
[...]
device_category TEXT NOT NULL,
[...]
operating_system TEXT
);
对于所有分区:
CREATE TABLE IF NOT EXISTS m_data.ga_session_20170127 ( CHECK (day_id = 20170127) ) INHERITS (m_data.ga_session);
-- the identifier are theoretically invalid, but they get truncated to 63 chars and nevertheless work
CREATE INDEX IF NOT EXISTS "ga_session__m_tmp.normalize_device_category(ga_session.device_category)" on m_data.ga_session_20170127 USING btree (m_tmp.normalize_device_category(device_category)) ;
CREATE INDEX IF NOT EXISTS "ga_session__m_tmp.normalize_operating_system(operating_system)" on m_data.ga_session_20170127 USING btree (m_tmp.normalize_operating_system(operating_system)) ;
ANALYZE m_data.ga_session_20170127;
EXPLAIN analyse
SELECT *
FROM m_data.ga_session_20170127 ga_session
JOIN m_dim_next.device ON
device.device_category_name = m_tmp.normalize_device_category(ga_session.device_category)
AND device.operating_system_name = m_tmp.normalize_operating_system(ga_session.operating_system);
这些分区上这些索引的统计信息是可见的:
SELECT * FROM pg_stats WHERE tablename ilike 'ga_session_20170127%';
schemaname |tablename |attname |inherited |null_frac |avg_width |n_distinct
-----------|----------------------------------------------------------------|---------------------------|----------|------------|----------|-------------
m_data |ga_session_20170127__m_tmp.normalize_device_category(device_cat |normalize_device_category |false |0 |10 |3
m_data |ga_session_20170127__m_tmp.normalize_operating_system(operating |normalize_operating_system |false |0 |7 |14
这(带有有关分区索引的统计信息)将导致以下(精细)查询计划估计:估计为80146,实际为77503
Hash Join (cost=1.95..6103.53 rows=80146 width=262) (actual time=0.121..117.204 rows=77503 loops=1)
Hash Cond: ((COALESCE(initcap(ga_session.device_category), 'Unknown'::text) = device.device_category_name) AND (COALESCE(replace(ga_session.operating_system, '(not set)'::text, 'Unknown'::text), 'Unknown'::text) = device.operating_system_name))
-> Seq Scan on ga_session_20170127 ga_session (cost=0.00..2975.03 rows=77503 width=224) (actual time=0.010..9.203 rows=77503 loops=1)
-> Hash (cost=1.38..1.38 rows=38 width=38) (actual time=0.064..0.064 rows=38 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 11kB
-> Seq Scan on device (cost=0.00..1.38 rows=38 width=38) (actual time=0.006..0.019 rows=38 loops=1)
Planning time: 1.460 ms
Execution time: 120.098 ms
无效的是整个表上的联接,该联接估计行数完全错误(估计为832,实际为876237)。
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hash Join (cost=1.95..60056.78 rows=832 width=262) (actual time=0.037..1065.778 rows=876237 loops=1)
Hash Cond: ((COALESCE(initcap(ga_session.device_category), 'Unknown'::text) = device.device_category_name) AND (COALESCE(replace(ga_session.operating_system, '(not set)'::text, 'Unknown'::text), 'Unknown'::text) = device.operating_system_name))
-> Append (cost=0.00..33759.37 rows=876238 width=225) (actual time=0.005..132.070 rows=876237 loops=1)
-> Seq Scan on ga_session (cost=0.00..0.00 rows=1 width=319) (actual time=0.000..0.000 rows=0 loops=1)
-> Seq Scan on ga_session_20170125 ga_session_1 (cost=0.00..3648.38 rows=94438 width=226) (actual time=0.005..10.606 rows=94438 loops=1)
-> Seq Scan on ga_session_20170126 ga_session_2 (cost=0.00..3185.81 rows=82581 width=225) (actual time=0.014..8.982 rows=82581 loops=1)
-> Seq Scan on ga_session_20170127 ga_session_3 (cost=0.00..2975.03 rows=77503 width=224) (actual time=0.002..8.797 rows=77503 loops=1)
-> Seq Scan on ga_session_20170128 ga_session_4 (cost=0.00..2936.83 rows=76083 width=225) (actual time=0.003..7.873 rows=76083 loops=1)
-> Seq Scan on ga_session_20170129 ga_session_5 (cost=0.00..3716.18 rows=96618 width=224) (actual time=0.002..9.318 rows=96618 loops=1)
-> Seq Scan on ga_session_20170130 ga_session_6 (cost=0.00..3833.19 rows=99619 width=224) (actual time=0.002..9.453 rows=99619 loops=1)
-> Seq Scan on ga_session_20170131 ga_session_7 (cost=0.00..3488.79 rows=90579 width=225) (actual time=0.002..8.298 rows=90579 loops=1)
-> Seq Scan on ga_session_20170201 ga_session_8 (cost=0.00..3615.58 rows=93958 width=224) (actual time=0.002..9.199 rows=93958 loops=1)
-> Seq Scan on ga_session_20170202 ga_session_9 (cost=0.00..3286.56 rows=85256 width=224) (actual time=0.006..8.021 rows=85256 loops=1)
-> Seq Scan on ga_session_20170203 ga_session_10 (cost=0.00..3073.02 rows=79602 width=225) (actual time=0.002..7.727 rows=79602 loops=1)
-> Hash (cost=1.38..1.38 rows=38 width=38) (actual time=0.016..0.016 rows=38 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 11kB
-> Seq Scan on device (cost=0.00..1.38 rows=38 width=38) (actual time=0.002..0.004 rows=38 loops=1)
Planning time: 1.017 ms
Execution time: 1090.213 ms
然后,当使用该联接导致更多联接(此处未显示)时,这将导致错误的联接选择(嵌套循环)。
在ANALYSE
再次在分区上运行之前,实际上在分区上的行估算值也有误,因此在使用整个表时,查询计划程序似乎没有考虑基于索引的统计信息。
在构建查询计划时,有什么方法可以使查询计划者收集父表级别的统计信息或将分区的各个统计信息考虑在内?
2
如果您提供了实际表和索引定义(的相关部分),那么该问题对我来说会更好。您是如何实现分区的?有继承权吗?这样就没有“在整个表上创建索引”这样的东西了,每个索引只能跨越一个物理表,即一个分区。
—
Erwin Brandstetter
抱歉,我添加了信息。索引本身在每个表上(我很困惑,因为我们只调用一个函数在所有继承表上创建索引)。
—
Jan Katins