VACUUM ANALYZE为什么不清除所有死元组?


8

VACUUM ANALYZE VERBOSE在对一些较大的表进行重大DELETE/INSERT更改之后,我们对它们进行了“手动” 处理。这似乎没有问题,尽管有时表的VACUUM工作将运行几个小时(有关类似问题和推理,请参阅此帖子)。

在进行更多研究时,我发现即使在运行后,我们也有包含大量死元组的大表VACUUM。例如,以下是此响应中查询产生的一些统计信息。

-[ RECORD 50 ]--+---------------------------
relname         | example_a
last_vacuum     | 2014-09-23 01:43
last_autovacuum | 2014-08-01 01:19
n_tup           |    199,169,568
dead_tup        |    111,048,906
av_threshold    |     39,833,964
expect_av       | *
-[ RECORD 51 ]--+---------------------------
relname         | example_b
last_vacuum     | 2014-09-23 01:48
last_autovacuum | 2014-08-30 12:40
n_tup           |    216,596,624
dead_tup        |    117,224,220
av_threshold    |     43,319,375
expect_av       | *
-[ RECORD 52 ]--+---------------------------
relname         | example_c
last_vacuum     | 2014-09-23 01:55
last_autovacuum | 2014-09-23 18:25
n_tup           |    309,831,136
dead_tup        |    125,047,233
av_threshold    |     61,966,277
expect_av       | *

最后一个字段指出这些(和大多数表)将达到自动真空的阈值。但是,仅VACUUM ANALYZE VEBOSE在每个表上运行后,无效元组计数是否应该为0(或接近于0,而不是300M中的125M)?

文档指出:

VACUUM回收死元组占用的存储。

这是否意味着我们VACUUM无法正常工作?


更新

每个响应中的请求都是来自VERBOSE作业的一些日志:

INFO:  vacuuming "public.example_1"
INFO:  scanned index "idx_example_1_on_gp_id_and_dd_id" to remove 378386 row versions
DETAIL:  CPU 1.83s/3.42u sec elapsed 23.01 sec.
INFO:  scanned index "index_example_1_on_q_id" to remove 378386 row versions
DETAIL:  CPU 2.10s/3.91u sec elapsed 18.92 sec.
INFO:  "example_1": removed 378386 row versions in 7085 pages
DETAIL:  CPU 0.09s/0.05u sec elapsed 0.19 sec.
INFO:  index "idx_example_1_on_gp_id_and_dd_id" now contains 30347438 row versions in 291065 pages
DETAIL:  378386 index row versions were removed.
165587 index pages have been deleted, 164287 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  index "index_example_1_on_q_id" now contains 30347438 row versions in 333287 pages
DETAIL:  378386 index row versions were removed.
152773 index pages have been deleted, 152757 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  "example_1": found 1773 removable, 401984 nonremovable row versions in 14438 out of 1493006 pages
DETAIL:  0 dead row versions cannot be removed yet.
There were 10567 unused item pointers.
0 pages are entirely empty.
CPU 4.26s/7.51u sec elapsed 46.10 sec.
INFO:  vacuuming "pg_toast.pg_toast_17917"
INFO:  index "pg_toast_17917_index" now contains 0 row versions in 1 pages
DETAIL:  0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  "pg_toast_17917": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL:  0 dead row versions cannot be removed yet.
There were 0 unused item pointers.
0 pages are entirely empty.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  analyzing "public.example_1"
INFO:  "example_1": scanned 30000 of 1493006 pages, containing 611502 live rows and 0 dead rows; 30000 rows in sample, 40563141 estimated total rows

该表现在在统计数据中显示0个死元组。今天早上大多数桌子上的死元组要低得多,因此我们的VACUUM或自动真空都在工作。

我们确实有一些表什么也没有输出,但仍然显示无效的元组:

-[ RECORD 49 ]--+---------------------------
relname         | example_2
last_vacuum     | 2014-09-23 02:23
last_autovacuum | 2014-09-02 14:30
n_tup           |    117,914,944
dead_tup        |     34,507,388
av_threshold    |     23,583,039
expect_av       | *

我在日志中看到过几次,它们会一遍又一遍地检查索引。这似乎与长期运行的VACUUM作业相对应。知道为什么吗?这只是解决记录锁定问题吗(我认为在此作业运行期间不会发生任何写操作。)

INFO:  vacuuming "public.example_2"
...
INFO:  scanned index "index_example_2_on_gsg_id_and_dd_id" to remove 2795959 row versions
DETAIL:  CPU 3.88s/16.54u sec elapsed 23.09 sec.
INFO:  scanned index "index_example_2_on_q_id" to remove 2795959 row versions
DETAIL:  CPU 6.74s/21.13u sec elapsed 84.64 sec.
INFO:  "example_2": removed 2795959 row versions in 48214 pages
DETAIL:  CPU 0.71s/0.32u sec elapsed 33.65 sec.
INFO:  scanned index "index_example_2_on_gsg_id_and_dd_id" to remove 2591011 row versions
DETAIL:  CPU 2.84s/16.11u sec elapsed 19.28 sec.
INFO:  scanned index "index_example_2_on_q_id" to remove 2591011 row versions
DETAIL:  CPU 5.46s/22.70u sec elapsed 130.57 sec.
INFO:  "example_2": removed 2591011 row versions in 45539 pages
DETAIL:  CPU 0.67s/0.38u sec elapsed 15.16 sec.
INFO:  index "index_example_2_on_gsg_id_and_dd_id" now contains 123807784 row versions in 1560915 pages
DETAIL:  108836958 index row versions were removed.
1100790 index pages have been deleted, 718471 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.25 sec.
INFO:  index "index_example_2_on_q_id" now contains 123807784 row versions in 1886087 pages
DETAIL:  110336259 index row versions were removed.
1058063 index pages have been deleted, 266983 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.07 sec.
INFO:  "example_2": found 124808 removable, 1355901 nonremovable row versions in 2086343 out of 6966379 pages
DETAIL:  0 dead row versions cannot be removed yet.
There were 7858495 unused item pointers.
0 pages are entirely empty.
CPU 595.49s/2130.13u sec elapsed 5656.34 sec.
INFO:  vacuuming "pg_toast.pg_toast_18079"
INFO:  index "pg_toast_18079_index" now contains 0 row versions in 1 pages
DETAIL:  0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  "pg_toast_18079": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL:  0 dead row versions cannot be removed yet.
There were 0 unused item pointers.
0 pages are entirely empty.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  analyzing "public.example_2"
INFO:  "example_2": scanned 30000 of 6966379 pages, containing 528443 live rows and 522 dead rows; 30000 rows in sample, 152953760 estimated total rows

0 dead row versions cannot be removed yet.表示您没有长时间运行的事务,这些事务会阻止删除无效的元组。
Erwin Brandstetter

Answers:


10

VACUUM只能删除长死的死元组,即所有可能使用的死元组。如果您的交易寿命长,则它们可能会阻止删除最近失效的元组。

这是一个长期交易阻止删除的情况的示例:

INFO:  "pgbench_accounts": found 0 removable, 2999042 nonremovable row versions in 49181 out of 163935 pages
DETAIL:  2999000 dead row versions cannot be removed yet.

它不是真正的长期事务,而是长期快照。当然,长时间运行的select或insert语句可以做到这一点。对于高于读提交的隔离级别,整个事务将保留快照,直到快照关闭为止,因此,如果某些事务打开一个可重复的读事务,然后在不提交提交的情况下进行休假,那将是一个问题。挂起的预备事务也将(如果您不知道预备事务是什么,那么您可能不使用它们)。

您显示的示例并不表示有问题,但您还说到那时该问题已解决。如果这是一个反复出现的问题,则您可能应该开始记录VACUUM VERBOSE语句的输出,以便您可以找到涵盖该问题存在期间的信息。

多次遍历索引是由于您的maintenance_work_mem设置。在索引的每次传递中,它只能为每6个字节的内存删除一个元组,并且如果需要删除更多的元组,则需要进行多次传递。因此,增加maintenance_work_mem会有所帮助。


您能否提供一个“长期交易”的例子?您是要长时间运行数据库查询还是INSERT/ IMPORT?还是说比连接打开/关闭还要长的时间?
jwadsa​​ck 2014年

4

物理表的大小通常(通过从表末尾修剪可移动页面的机会除外)不会因运行VACUUM(或VACUUM ANALYZE)而减小。您需要运行VACUUM FULL以实际收缩表。

这是相关答案的引文,其中包含更多详细信息:

每个文档(实际上仅在报价下方几行):

普通VACUUM(无FULL)只是回收空间并使其可重复使用。该命令的形式可以与表的正常读取和写入并行进行,因为没有获得排他锁。但是,在大多数情况下,多余的空间不会返回给操作系统。

更多内容:

您将对pg_repack感兴趣,它可以与VACUUM FULL没有排他锁的情况相同。


1
抱歉,如果我的问题不清楚,但是我在问剩余的死元组。我知道VACUUM没有它FULL不会减小磁盘的大小,对此我很好。我之所以提到大型表,是因为我首先链接到该文章,指出经过调整的积极真空策略将是“胜利……如果有大型表的行从未删除或更新过”。我们的大桌子每天更换。
jwadsa​​ck 2014年
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.