SSIS - 将字符串分成多行

真的寻找一些建议。我有一个学校考勤数据的数据集，它以我的身份进入：student_id，school_id，term_start_date，attendance_marks。SSIS - 将字符串分成多行

这个数据的一个例子线将是：

1234, 1002, 2016-09-01, '/\##/L/\##BB/\/\/\/\/\'

现场的标记字符串是基本上每天两个会话，每个标记对应于各种不同的代码。基本上，我得到一个开始日期，然后必须制定出每一天的出勤标志......我知道这很恐怖，但这就是我如何从恐惧中获得数据。

我已经写了一个脚本对象来循环这个字符串，并输出每一天的行来加载到数据仓库。

在发布此脚本工作的代码之前，我应该说，它做我需要它做的事情......但它是痛苦的缓慢。我希望我可以借鉴这里的集体经验，看看是否有更高效的方法来实现这一目标？

public override void Input0_ProcessInputRow(Input0Buffer Row) 
{ 
    int studentID = Row.STUDDENTID; 
    int baseID = Row.SCHOOLID; 
    DateTime dateKey = Row.STARTDATE; 
    string marks = Row.MARKS.ToString(); 

    // TODO 
    // This is a bodge at the moment to make sure the mark string is always divisible by 2 
    // in the production release I'll have to handle this as an exception 
    if (marks.Length % 2 != 0) 
    { 
     marks = marks + '@'; 
    } 

    char[] c = marks.ToCharArray(); 

    for (int i = 0; i < c.Length; i += 2) 
    { 
     Output0Buffer.AddRow(); 
     Output0Buffer.baseID = baseID; 
     Output0Buffer.studentID = studentID; 
     Output0Buffer.dateKey = dateKey.AddDays(i/2).Year * 10000 + dateKey.AddDays(i/2).Month * 100 + dateKey.AddDays(i/2).Day; 
     Output0Buffer.markAM = c[i].ToString(); 
     Output0Buffer.markPM = c[i + 1].ToString(); 

     if (i == c.Length - 1) 
     { 
      base.FinishOutputs(); 
     } 
    } 
}

有没有更好的方法来做到这一点？我是否已经大规模地推翻了它，还是我只需要忍受它需要很长时间才能运行？提前致谢。

来源

2017-06-06 Jay P

您使用的是什么版本的SQL？ – iamdave

使用SSIS 2012加载到SQL 2012数据库中，从Oracle 12c数据库读取考勤数据。 SQL Server方面只在当前运行，因为这是一个备用服务器，供我测试/创建它。展望未来，我可以将它安装在任何版本的SQL Server上（我希望最新的一个）。 –

那你的登台区是SQL Server 2012吗？ – iamdave

这也许可以写在甲骨文的PL-SQL，虽然我与Oracle既不工作，也不建议你做你的直播系统内转换为提取

的一部分，如果您加载数据是成一个SQL Server分段环境，您可以使用脚本使用派生的计数表将Attendance字符串拆分为每行一个字符，然后将其返回到您的OLEDB Source组件中。这将有多个学生，学校和日工作（即：它是一个适当的集基础的解决方案），因此可以被用作一个通取，将返回整个数据集：

-- Create test data 
declare @d table(StudentID int,SchoolID int, AttDate date, Attendance nvarchar(500)); 
insert into @d values 
(1234, 1002, '20160901', '/\##/L/\##BB/\/\/\/\') 
,(1235, 1002, '20160901', '/\##/L/\##/\BB/\/\/\') 
,(1234, 1002, '20160902', '/\##/L/\##BB/\/\/\/\/\') 
,(1235, 1002, '20160902', '/\##/L/\/\BB/\/\##/\/\'); 

       -- Create a list of 10 rows 
with n(n) as (select n from (values(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) n(n)) 
       -- Cross join 6 times to reach 1,000,000 numbers if required. Filtered to longest Attendance string in the SELECT TOP() 
    ,t(t) as (select top (select max(len(Attendance)) from @d) row_number() over (order by (select null)) from n n1,n n2,n n3,n n4,n n5,n n6) 
select d.StudentID 
     ,d.SchoolID 
     ,d.AttDate 
     ,d.Attendance 
     ,t.t as AttCharNum 
     ,substring(d.Attendance,t.t,1) as AttChar -- Use SUBSTRING to retreive just the one character 
from @d d 
    join t -- Join to numbers only where they are actually required 
     on t.t <= len(d.Attendance) 
order by d.StudentID 
     ,d.AttDate 
     ,t.t;

输出：

+-----------+----------+------------+------------------------+------------+---------+ 
| StudentID | SchoolID | AttDate |  Attendance  | AttCharNum | AttChar | 
+-----------+----------+------------+------------------------+------------+---------+ 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   1 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   2 | \  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   3 | #  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   4 | #  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   5 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   6 | L  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   7 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   8 | \  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   9 | #  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   10 | #  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   11 | B  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   12 | B  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   13 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   14 | \  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   15 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   16 | \  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   17 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   18 | \  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   19 |/  | 
|  1234 |  1002 | 2016-09-01 | /\##/L/\##BB/\/\/\/\ |   20 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   1 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   2 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   3 | #  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   4 | #  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   5 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   6 | L  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   7 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   8 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   9 | #  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   10 | #  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   11 | B  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   12 | B  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   13 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   14 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   15 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   16 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   17 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   18 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   19 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   20 | \  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   21 |/  | 
|  1234 |  1002 | 2016-09-02 | /\##/L/\##BB/\/\/\/\/\ |   22 | \  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   1 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   2 | \  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   3 | #  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   4 | #  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   5 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   6 | L  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   7 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   8 | \  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   9 | #  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   10 | #  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   11 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   12 | \  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   13 | B  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   14 | B  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   15 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   16 | \  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   17 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   18 | \  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   19 |/  | 
|  1235 |  1002 | 2016-09-01 | /\##/L/\##/\BB/\/\/\ |   20 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   1 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   2 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   3 | #  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   4 | #  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   5 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   6 | L  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   7 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   8 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   9 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   10 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   11 | B  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   12 | B  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   13 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   14 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   15 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   16 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   17 | #  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   18 | #  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   19 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   20 | \  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   21 |/  | 
|  1235 |  1002 | 2016-09-02 | /\##/L/\/\BB/\/\##/\/\ |   22 | \  | 
+-----------+----------+------------+------------------------+------------+---------+

来源

2017-06-06 11:45:57 iamdave

太棒了，我会玩一玩，看看我能做些什么。我怀疑这是一个更好的方法，这将是我最后一次听C＃开发人员关于加载数据:) –

是的，这工作得更好，非常感谢。 –

@JayP不用担心，很高兴你有东西分类:) – iamdave

SSIS - 将字符串分成多行

回答

相关问题