1. 首页
  2. 问答
  3. MERGE查询在Management Studio 2008 Express中运行速度很慢

MERGE查询在Management Studio 2008 Express中运行速度很慢

2012-01-18 126 views
1

我有一个非常简单的MERGE查询，其中源表只有大约41MB，目标表大约为4GB。我基本上只是将行条目添加到较大的目标表（同时避免重复的主键条目）。在Management Studio中执行时，此查询运行速度非常慢。它已经运行了15分钟，并没有完成。只是想知道是否有任何加速执行的选择，或者如果我做错了什么。谢谢。MERGE查询在Management Studio 2008 Express中运行速度很慢

这里的查询：

<?xml version="1.0" encoding="utf-16"?> 
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.1" Build="10.50.1600.1" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan"> 
    <BatchSequence> 
    <Batch> 
     <Statements> 
     <StmtSimple StatementCompId="1" StatementEstRows="92529300000" StatementId="1" StatementOptmLevel="FULL" StatementSubTreeCost="45530500" StatementText="--sp_configure 'show advanced options', 1;&#xD;&#xA;--RECONFIGURE;&#xD;&#xA;--GO&#xD;&#xA;--sp_configure 'Ad Hoc Distributed Queries', 1;&#xD;&#xA;--RECONFIGURE;&#xD;&#xA;--GO&#xD;&#xA;MERGE XCManager.XC_DATA1 target&#xD;&#xA;USING XCManager.FrkFalls_FORMATTED source&#xD;&#xA;ON target.STATION_ID = source.STATION_ID&#xD;&#xA;WHEN NOT MATCHED BY TARGET THEN&#xD;&#xA; INSERT (STATION_ID, SENSORNAME, TIME_TAG, ORIG_VALUE, ED_VALUE, SOURCE)&#xD;&#xA; VALUES (source.STATION_ID, source.SENSORNAME, source.TIME_TAG, source.ORIG_VALUE, source.ED_VALUE, source.SOURCE);" StatementType="MERGE" QueryHash="0x0A5DB292FF222BCD" QueryPlanHash="0x1FBBCC92AE041A3E"> 
      <StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" /> 
      <QueryPlan CachedPlanSize="64" CompileTime="31" CompileCPU="15" CompileMemory="736"> 
      <RelOp AvgRowSize="9" EstimateCPU="92529.3" EstimateIO="7070610" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="92529300000" LogicalOp="Merge" NodeId="1" Parallel="false" PhysicalOp="Clustered Index Merge" EstimatedTotalSubtreeCost="45530500"> 
       <OutputList /> 
       <Update WithOrderedPrefetch="true" DMLRequestSort="true"> 
       <Object Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Index="[DAT1_PK]" Alias="[target]" IndexKind="Clustered" /> 
       <SetPredicate> 
        <ScalarOperator ScalarString="[XConnectDB_NHDES].[XCManager].[XC_DATA1].[SOURCE] as [target].[SOURCE] = RaiseIfNullUpdate([Expr1008]),[XConnectDB_NHDES].[XCManager].[XC_DATA1].[SENSORNAME] as [target].[SENSORNAME] = RaiseIfNullUpdate([Expr1009]),[XConnectDB_NHDES].[XCManager].[XC_DATA1].[ED_VALUE] as [target].[ED_VALUE] = [XConnectDB_NHDES].[XCManager].[FrkFalls_FORMATTED].[ED_VALUE] as [source].[ED_VALUE],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[ORIG_VALUE] as [target].[ORIG_VALUE] = RaiseIfNullUpdate([XConnectDB_NHDES].[XCManager].[FrkFalls_FORMATTED].[ORIG_VALUE] as [source].[ORIG_VALUE]),[XConnectDB_NHDES].[XCManager].[XC_DATA1].[TIME_TAG] as [target].[TIME_TAG] = RaiseIfNullUpdate([XConnectDB_NHDES].[XCManager].[FrkFalls_FORMATTED].[TIME_TAG] as [source].[TIME_TAG]),[XConnectDB_NHDES].[XCManager].[XC_DATA1].[STATION_ID] as [target].[STATION_ID] = RaiseIfNullUpdate([Expr1010]),[XConnectDB_NHDES].[XCManager].[XC_DATA1].[FLAG1] as [target].[FLAG1] = [Expr1011],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[FLAG2] as [target].[FLAG2] = [Expr1012],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[FLAG3] as [target].[FLAG3] = [Expr1013],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[FLAG4] as [target].[FLAG4] = [Expr1014],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[HIGH_HIGH_FLAG] as [target].[HIGH_HIGH_FLAG] = [Expr1015],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[HIGH_FLAG] as [target].[HIGH_FLAG] = [Expr1016],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[LOW_FLAG] as [target].[LOW_FLAG] = [Expr1017],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[LOW_LOW_FLAG] as [target].[LOW_LOW_FLAG] = [Expr1018],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[ROC_FLAG] as [target].[ROC_FLAG] = [Expr1019],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[NO_CHG_FLAG] as [target].[NO_CHG_FLAG] = [Expr1020],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[ALARM_FLAG] as [target].[ALARM_FLAG] = [Expr1021],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[STD_DEV_FLAG] as [target].[STD_DEV_FLAG] = [Expr1022],[XConnectDB_NHDES].[XCManager].[XC_DATA1].[AVG_FLAG] as [target].[AVG_FLAG] = [Expr1023]"> 
        <ScalarExpressionList> 
         <ScalarOperator> 
         <MultipleAssign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="SOURCE" /> 
          <ScalarOperator> 
           <Intrinsic FunctionName="RaiseIfNullUpdate"> 
           <ScalarOperator> 
            <Identifier> 
            <ColumnReference Column="Expr1008" /> 
            </Identifier> 
           </ScalarOperator> 
           </Intrinsic> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="SENSORNAME" /> 
          <ScalarOperator> 
           <Intrinsic FunctionName="RaiseIfNullUpdate"> 
           <ScalarOperator> 
            <Identifier> 
            <ColumnReference Column="Expr1009" /> 
            </Identifier> 
           </ScalarOperator> 
           </Intrinsic> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="ED_VALUE" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[FrkFalls_FORMATTED]" Alias="[source]" Column="ED_VALUE" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="ORIG_VALUE" /> 
          <ScalarOperator> 
           <Intrinsic FunctionName="RaiseIfNullUpdate"> 
           <ScalarOperator> 
            <Identifier> 
            <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[FrkFalls_FORMATTED]" Alias="[source]" Column="ORIG_VALUE" /> 
            </Identifier> 
           </ScalarOperator> 
           </Intrinsic> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="TIME_TAG" /> 
          <ScalarOperator> 
           <Intrinsic FunctionName="RaiseIfNullUpdate"> 
           <ScalarOperator> 
            <Identifier> 
            <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[FrkFalls_FORMATTED]" Alias="[source]" Column="TIME_TAG" /> 
            </Identifier> 
           </ScalarOperator> 
           </Intrinsic> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="STATION_ID" /> 
          <ScalarOperator> 
           <Intrinsic FunctionName="RaiseIfNullUpdate"> 
           <ScalarOperator> 
            <Identifier> 
            <ColumnReference Column="Expr1010" /> 
            </Identifier> 
           </ScalarOperator> 
           </Intrinsic> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="FLAG1" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1011" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="FLAG2" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1012" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="FLAG3" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1013" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="FLAG4" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1014" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="HIGH_HIGH_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1015" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="HIGH_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1016" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="LOW_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1017" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="LOW_LOW_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1018" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="ROC_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1019" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="NO_CHG_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1020" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="ALARM_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1021" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="STD_DEV_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1022" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
          <Assign> 
          <ColumnReference Database="[XConnectDB_NHDES]" Schema="[XCManager]" Table="[XC_DATA1]" Alias="[target]" Column="AVG_FLAG" /> 
          <ScalarOperator> 
           <Identifier> 
           <ColumnReference Column="Expr1023" /> 
           </Identifier> 
          </ScalarOperator> 
          </Assign> 
         </MultipleAssign> 
         </ScalarOperator> 
        </ScalarExpressionList> 
        </ScalarOperator> 
       </SetPredicate> 
       <ActionColumn> 
        <ColumnReference Column="Action1007" /> 
       </ActionColumn> 
       <RelOp AvgRowSize="130" EstimateCPU="9252.93" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="92529300000" LogicalOp="Compute Scalar" NodeId="3" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="38367300"> 
        <OutputList>

来源

2012-01-18 jaker

+0

是否有target.STATION_ID上的索引？ – 2012-01-18 19:45:32

+0

我的四个主键列有一个聚集索引。 STATION_ID，SENSORNAME，TIME_TAG和SOURCE。 – jaker 2012-01-18 19:48:24

+0

为什么你使用合并？为什么不直接插入，因为你没有指定更新或删除操作？你希望插入多少条记录？ – HLGEM 2012-01-18 19:56:35

回答

1

看起来你已经添加了新的索引是不是有帮助的STATION_ID有bewteeen两个不同的数据类型：估计执行XML的

MERGE [myschema].[targettable] target 
USING [myschema].[sourcetable] source 
ON target.STATION_ID = source.STATION_ID  
WHEN NOT MATCHED BY TARGET THEN  
    INSERT (STATION_ID, SENSORNAME, TIME_TAG, ORIG_VALUE, ED_VALUE, SOURCE)  
    VALUES (source.STATION_ID, source.SENSORNAME, source.TIME_TAG, source.ORIG_VALUE, source.ED_VALUE, source.SOURCE);

开头表，它需要做一个隐式转换到nvarchar(20)。

如果可能使这些一致。

的MERGE版本本质上是做类似

INSERT INTO FrkFalls_FORMATTED 
SELECT * 
FROM XC_DATA1 source 
LEFT JOIN FrkFalls_FORMATTED target 
ON target.STATION_ID = source.STATION_ID 
WHERE target.STATION_ID IS NULL

由于STATION_ID不是唯一的这可能意味着它最终加盟然后丢弃的中间记录无助于最终结果的整个负载。您上传的估算计划显示了多对多MERGE LEFT JOIN中的920亿行。这些数字可能是严重错误的，但您没有提供更丰富的实际执行计划。

Plan

你可以尝试放弃合并，改写为

INSERT INTO FrkFalls_FORMATTED 
SELECT * 
FROM XC_DATA1 source 
WHERE NOT EXISTS (SELECT * 
        FROM FrkFalls_FORMATTED target 
        WHERE target.STATION_ID = source.STATION_ID)

这可能是更有效的。

来源

2012-01-18 23:03:30

+0

也试过上面的查询，但得到了重复键错误：**消息2601，级别14，状态1，行1 不能在具有唯一索引'DAT1_PK'的对象'XCManager.XC_DATA1'中插入重复键行。 声明已被终止。** – jaker 2012-02-14 16:43:44

+0

啊！想想我拥有它。在索引'DAT1_PK'中关闭唯一。然后查询成功运行，并且只从源表插入新行。再次感谢大家的帮助！ – jaker 2012-02-14 16:52:02

+0

+1 INSERT而不是MERGE对我也很好。 – 2013-05-23 18:46:53

0

我猜你可能已经完成插入数据，但对于未来，我觉得这个查询会更快：

INSERT INTO Test.CoveredIndexTestMaster 
SELECT * FROM Test.CoveredIndexTestSubset 
EXCEPT 
SELECT * FROM Test.CoveredIndexTestMaster;

EXCEPT运算符返回左表中的所有行（Test.CoveredIndexTestSubset）那些不在正确的表（Test.CoveredIndexTestMaster）中，在这种情况下，它们将被插入到Test.CoveredIndexTestMaster中。 （这些表格当然有相同的结构。）

尽管我没有时间设置完全与您的情况重复的测试，但我可以告诉您这一点。 Test.CoveredIndexTestMaster有大约900K行，Test.CoveredIndexTestSubset有大约206K行。 Subset中有超过3K行不在Master中。

上面的查询需要14秒运行。

这两个表都没有索引。该表有2列。一个int和一个varchar。两个表中都有dublicate行。

所以，你的milage会有所不同，但我会有兴趣听到你的结果。

这是根据马丁的输入更新查询：

INSERT INTO Test.CoveredIndexTestMaster 
SELECT * 
FROM Test.CoveredIndexTestSubset 
WHERE Col1 IN (SELECT col1 FROM Test.CoveredIndexTestSubset 
        EXCEPT 
        SELECT col1 FROM Test.CoveredIndexTestMaster)

这花了20秒的运行。

来源

2012-01-18 23:08:13 Karl

+1

这不会做同样的事情。这比较了整行而不仅仅是'STATION_ID'值。 – 2012-01-18 23:09:29

+0

对。如果子集和主节点中存在重复的主键但具有不同的详细信息，则它将由除外运算符返回，并且插入失败。我更新了原始答案中的查询。 – Karl 2012-01-18 23:39:13

+0

我欣赏所有帮助家伙。我仍然有这个问题。当我尝试的第一个除了这种形式的查询：INSERT INTO XConnectDB_NHDES.XCManager.XC_DATA1 SELECT * FROM XConnectDB_NHDES.XCManager.FrkFalls_FORMATTED execpt SELECT * FROM XConnectDB_NHDES.XCManager.XC_DATA1为目标 其中target.STATION_ID = 'FFDN3' – jaker 2012-02-14 16:22:53

1

实际上我的原始解决方案关闭索引中的唯一性并不好。我试图导入的遗留数据充满了很多非唯一的行。这使我无法在导入后重新开启独特功能。

我最终不得不用一个临时表'q'做一个更奇特的查询来解决这个问题。此查询首先从源表中选择所有不同的行（基于字段TIME_TAG，STATION_ID和SENSORNAME），然后将它们插入到目标表中，只要它不是重复行（同样基于字段TIME_TAG，STATION_ID和SENSORNAME）。

;WITH q AS 
      (
      SELECT *, ROW_NUMBER() OVER (PARTITION BY TIME_TAG, STATION_ID, SENSORNAME ORDER BY TIME_TAG) rn 
      FROM XConnectDB_NHDES.XCManager.Lakeport_FORMATTED 
      ) 


    INSERT INTO XConnectDB_NHDES.XCManager.XC_DATA1 
    SELECT STATION_ID, SENSORNAME, TIME_TAG, SOURCE, ORIG_VALUE, ED_VALUE, FLAG1, FLAG2, FLAG3, FLAG4, HIGH_HIGH_FLAG, HIGH_FLAG, LOW_FLAG, LOW_LOW_FLAG, ROC_FLAG, NO_CHG_FLAG, ALARM_FLAG, STD_DEV_FLAG, AVG_FLAG 
    FROM q source 
    WHERE rn = 1 and 
    NOT EXISTS (SELECT * 
         FROM XConnectDB_NHDES.XCManager.XC_DATA1 target 
         WHERE target.TIME_TAG = source.TIME_TAG and 
         target.STATION_ID = source.STATION_ID and 
         target.SENSORNAME = source.SENSORNAME)

来源

2012-02-24 16:06:36 jaker

相关问题

 相关问题