返回固定行数后查询暂停


8

我有一个视图,它可以快速(几秒钟)运行多达41条记录(例如TOP 41),但要花44分钟或更长的时间才能显示44条或更多条记录,如果使用TOP 42或运行,则结果会中等TOP 43。具体来说,它将在几秒钟内返回前39条记录,然后暂停近三分钟,然后返回其余记录。查询TOP 44或时,此模式相同TOP 100

该视图最初是从基本视图派生的,仅在底部代码中添加了一个过滤器,即最后一个过滤器。如果我从基础链接子视图,或者用嵌入的基础代码编写子视图,似乎没有什么区别。基本视图仅在几秒钟内返回100条记录。我想我可以使子视图的运行速度与基础视图一样快,而不是慢50倍。有人看到过这种行为吗?关于原因或解决方案的任何猜测?

在我测试了所涉及的查询后的最后几个小时,这种行为一直是一致的,尽管在事情开始变慢之前返回的行数略有起伏。这不是新事物。我现在正在查看它,因为总运行时间是可以接受的(<2分钟),但是至少在几个月以来,我已经在相关的日志文件中看到这种暂停。

封锁

我从未见过查询被阻塞,即使数据库上没有其他活动(经sp_WhoIsActive验证),问题仍然存在。基本视图包括所有NOLOCK内容,这是值得的。

查询

这是子视图的简化版本,为简单起见,基本视图都已内联。它仍然显示出运行时间的跳跃,大约有40条记录。

SELECT TOP 100 PERCENT
    Map.SalesforceAccountID AS Id,
    CAST(C.CustomerID AS NVARCHAR(255)) AS Name,
    CASE WHEN C.StreetAddress = 'Unknown' THEN '' ELSE C.StreetAddress                 END AS BillingStreet,
    CASE WHEN C.City          = 'Unknown' THEN '' ELSE SUBSTRING(C.City,        1, 40) END AS BillingCity,
                                                       SUBSTRING(C.Region,      1, 20)     AS BillingState,
    CASE WHEN C.PostalCode    = 'Unknown' THEN '' ELSE SUBSTRING(C.PostalCode,  1, 20) END AS BillingPostalCode,
    CASE WHEN C.Country       = 'Unknown' THEN '' ELSE SUBSTRING(C.Country,     1, 40) END AS BillingCountry,
    CASE WHEN C.PhoneNumber   = 'Unknown' THEN '' ELSE C.PhoneNumber                   END AS Phone,
    CASE WHEN C.FaxNumber     = 'Unknown' THEN '' ELSE C.FaxNumber                     END AS Fax,
    TransC.WebsiteAddress AS Website,
    C.AccessKey AS AccessKey__c,
    CASE WHEN dbo.ValidateEMail(C.EMailAddress) = 1 THEN C.EMailAddress END,  -- Removing this UDF does not speed things
    TransC.EmailSubscriber
    -- A couple dozen additional TransC fields
FROM
    WarehouseCustomers AS C WITH (NOLOCK)
    INNER JOIN TransactionalCustomers AS TransC WITH (NOLOCK) ON C.CustomerID = TransC.CustomerID
    LEFT JOIN  Salesforce.AccountsMap AS Map WITH (NOLOCK) ON C.CustomerID = Map.CustomerID
WHERE
        C.DateMadeObsolete IS NULL
    AND C.EmailAddress NOT LIKE '%@volusion.%'
    AND C.AccessKey IN ('C', 'R')
    AND C.CustomerID NOT IN (243566)  -- Exclude specific test records
    AND EXISTS (SELECT * FROM Orders AS O WHERE C.CustomerID = O.CustomerID AND O.OrderDate >= '2010-06-28')  -- Only count customers who've placed a recent order
    AND Map.SalesforceAccountID IS NULL  -- Only count customers not already uploaded to Salesforce
-- Removing the ORDER BY clause does not speed things up
ORDER BY
    C.CustomerID DESC

Id IS NULL过滤器将丢弃由BaseView; 返回的大多数记录;没有TOP子句,它们分别返回1,100条记录和267K。

统计

运行时TOP 40

SQL Server parse and compile time:    CPU time = 234 ms, elapsed time = 247 ms.
SQL Server Execution Times:   CPU time = 0 ms,  elapsed time = 0 ms.
SQL Server Execution Times:   CPU time = 0 ms,  elapsed time = 0 ms.

(40 row(s) affected)
Table 'CustomersHistory'. Scan count 2, logical reads 39112, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Orders'. Scan count 1, logical reads 752, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'AccountsMap'. Scan count 1, logical reads 458, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times:   CPU time = 2199 ms,  elapsed time = 7644 ms.

运行时TOP 45

(45 row(s) affected)
Table 'CustomersHistory'. Scan count 2, logical reads 98268, physical reads 1, read-ahead reads 3, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Orders'. Scan count 1, logical reads 1788, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'AccountsMap'. Scan count 1, logical reads 2152, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times: CPU time = 41980 ms,  elapsed time = 177231 ms.

令我惊讶的是,由于实际输出中的这种微小差异,读取次数跃升了约3倍。

比较执行计划,除了返回的行数外,它们是相同的。与上述统计数据一样,TOP 45查询中早期步骤的实际行数显着增加,而不仅仅是增加了12.5%。

总的来说,它正在扫描Orders中的覆盖索引,从WarehouseCustomers中寻找相应的记录;将其循环连接到TransactionalCustomers(远程查询,确切的计划未知);并将其与AccountsMap的表扫描合并。远程查询是估计成本的94%。

杂项说明

早些时候,当我将视图的扩展内容作为独立查询执行时,它的运行速度非常快:100条记录需要13秒。我现在正在测试查询的简化版本,没有子查询,并且这个简单得多的查询要求三分钟才能返回40多个行,即使是作为独立查询运行时也是如此。

子视图包含大量读取(每个sp_WhoIsActive约为1M),但是在此计算机上(八个内核,32 GB RAM,95%专用SQL框)通常不是问题。

我删除并重新创建了两次视图,没有任何更改。

该数据不包含任何TEXT或BLOB字段。一个领域涉及UDF。删除它不会阻止暂停。

无论是在服务器上还是在1,400英里外的工作站上进行查询,时间都是相似的,因此延迟似乎是查询本身所固有的,而不是将结果发送给客户端。

注意:解决方案

修复最终变得很简单:用子句替换LEFT JOINto Map NOT EXISTS。这将导致查询计划中的一个微小差异,即在联接到Map表之后而不是之前联接到TransactionCustomers表(远程查询)。这可能意味着它仅从远程服务器请求所需的记录,这将减少传输的量约100倍。

通常我是第一个为之欢呼的人NOT EXISTS; 它通常比LEFT JOIN...WHERE ID IS NULL构造更快,并且更紧凑。在这种情况下,这很尴尬,因为问题查询是建立在现有视图上的,并且反联接所需的字段由基本视图公开,但它首先是从整数转换为文本。因此,为了获得不错的性能,我必须删除两层模式,而要拥有两个几乎完全相同的视图,第二个视图包含该NOT EXISTS子句。

谢谢大家对解决此问题的帮助!对于我的情况,这可能太具体了,无法为其他任何人提供帮助,但希望不会。如果没有别的,这就是NOT EXISTS比快一点的例子LEFT JOIN...WHERE ID IS NULL。但是,真正的教训可能是确保远程查询被尽可能有效地连接。该查询计划声称它占成本的2%,但并不总是能够准确估算。


评论不作进一步讨论;此对话已转移至聊天
保罗·怀特9

Answers:


4

可以尝试的一些事情:

  1. 检查索引

    • 所有JOIN关键字段都被索引了吗?如果您经常使用此视图,那么我会为该视图中的条件添加过滤索引。例如...

    • CREATE INDEX ix_CustomerId ON WarehouseCustomers(CustomerId, EmailAddress) WHERE DateMadeObsolete IS NULL AND AccessKey IN ('C', 'R') AND CustomerID NOT IN (243566)

  2. 更新统计

    • 过时的统计信息可能存在问题。如果可以摆动它,我会做一个FULLSCAN。如果行数很多,则数据可能会发生重大变化而不会触发自动重新计算。
  3. 清理查询

    • Map JOIN一个NOT EXISTS-您不需要该表中的任何数据,因为您只想要不匹配的记录

    • 删除ORDER BY。我知道评论说没关系,但是我很难相信。对于较小的结果集而言,这可能无关紧要,因为数据页已被缓存。


有趣的地方:过滤后的索引。该查询不会自动使用它,但我会测试一下是否带有强制提示。我已经更新了统计数据,今天晚些时候可以测试此数据和您的其他建议;在EOWD之后,我需要积压工作,以便可以对一组不错的数据进行测试。
所有行业的乔恩2012年

我尝试了这些调整的不同组合,关键似乎是与Map的反联接。作为LEFT JOIN...WHERE Id IS NULL,我得到了暂停;作为NOT EXISTS子句,运行时间为秒。我很惊讶,但我不能与结果争论!
所有行业的乔恩(Jon of All Trades)

2

改进1删除订单的子查询并将其转换为联接

FROM
WarehouseCustomers AS C WITH (NOLOCK)
INNER JOIN TransactionalCustomers AS TransC WITH (NOLOCK) 
                                                        ON C.CustomerID = TransC.CustomerID
LEFT JOIN  Salesforce.AccountsMap AS Map WITH (NOLOCK) 
                                                        ON C.CustomerID = Map.CustomerID
INNER Join Orders AS O 
                                                        ON C.CustomerID = O.CustomerID

 WHERE
    C.DateMadeObsolete IS NULL
    AND C.EmailAddress NOT LIKE '%@volusion.%'
    AND C.AccessKey IN ('C', 'R')
    AND C.CustomerID NOT IN (243566)
    AND O.OrderDate >= '2010-06-28'
    AND Map.SalesforceAccountID IS NULL

改进2-将TransactionalCustomers过滤的记录保留在本地临时表中

Select 
    CAST(C.CustomerID AS NVARCHAR(255)) AS Name,
    CASE WHEN C.StreetAddress = 'Unknown' THEN '' ELSE C.StreetAddress                 END AS BillingStreet,
    CASE WHEN C.City          = 'Unknown' THEN '' ELSE SUBSTRING(C.City,        1, 40) END AS BillingCity,
                                                       SUBSTRING(C.Region,      1, 20)     AS BillingState,
    CASE WHEN C.PostalCode    = 'Unknown' THEN '' ELSE SUBSTRING(C.PostalCode,  1, 20) END AS BillingPostalCode,
    CASE WHEN C.Country       = 'Unknown' THEN '' ELSE SUBSTRING(C.Country,     1, 40) END AS BillingCountry,
    CASE WHEN C.PhoneNumber   = 'Unknown' THEN '' ELSE C.PhoneNumber                   END AS Phone,
    CASE WHEN C.FaxNumber     = 'Unknown' THEN '' ELSE C.FaxNumber                     END AS Fax,
    C.AccessKey AS AccessKey__c
Into #Temp
From  WarehouseCustomers C
Where C.DateMadeObsolete IS NULL
        AND C.EmailAddress NOT LIKE '%@volusion.%'
        AND C.AccessKey IN ('C', 'R')
        AND C.CustomerID NOT IN (243566)

最终查询

FROM
#Temp AS C WITH (NOLOCK)
INNER JOIN TransactionalCustomers AS TransC WITH (NOLOCK) 
                                                            ON C.CustomerID = TransC.CustomerID
LEFT JOIN Salesforce.AccountsMap AS Map WITH (NOLOCK) 
                                                            ON C.CustomerID = Map.CustomerID
INNER Join Orders AS O 
                                                            ON C.CustomerID = O.CustomerID

WHERE
C.DateMadeObsolete IS NULL
AND C.EmailAddress NOT LIKE '%@volusion.%'
AND C.AccessKey IN ('C', 'R')
AND C.CustomerID NOT IN (243566)
AND O.OrderDate >= '2010-06-28'
AND Map.SalesforceAccountID IS NULL

第3点-我假设您在CustomerID,EmailAddress,OrderDate上有索引


1
回复:“改进” 1- 在这种情况下EXISTS通常比a更快JOIN,并且消除了潜在的重复。我认为这根本没有改善。
JNK 2012年

1
但是,问题是双重的-可能改变结果,并且除非两个表在联接中使用的字段上具有唯一的聚集索引,否则它的效率将低于EXISTS。子句并不总是坏的。
JNK 2012年

@PankajGarg:谢谢您的建议,不幸的是每个客户通常有多个订单,所以必须这样做EXISTS。同样,在视图中,尽管我提出了不带参数的虚拟TVF的想法,但是我无法缓存重用的客户数据。
所有行业的乔恩2012年
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.