在python中分割逗号分隔的字符串

Question

此问题已被询问和回答多次。一些例子：[1],[2]。但似乎并没有一些更一般的东西。我正在寻找的是一种用逗号分隔字符串的方式，这些逗号不在引号或分隔符对之内。例如：

s1 = 'obj<1, 2, 3>, x(4, 5), "msg, with comma"' 

应分成三个要素

['obj<1, 2, 3>', 'x(4, 5)', '"msg, with comma"'] 

的问题的列表，现在是这样可以变得更加复杂，因为我们可以考虑对<>和()。

s2 = 'obj<1, sub<6, 7>, 3>, x(4, y(8, 9), 5), "msg, with comma"' 

应该被分成：

['obj<1, sub<6, 7>, 3>', 'x(4, y(8, 9), 5)', '"msg, with comma"'] 

不使用正则表达式的幼稚的解决方案是通过寻找字符,<(解析字符串。如果找到<或(，那么我们开始计数奇偶校验。如果奇偶性为零，我们只能以逗号分割。举例说，我们要拆分s2，我们可以parity = 0开始，当我们到达s2[3]我们遇到<这将增加1奇偶校验时遇到>或)奇偶只会减少，当它遇到它会增加<或( 。虽然奇偶校验不是0，我们可以简单地忽略逗号而不做任何分割。

这里的问题是，有没有办法通过正则表达式快速？我真的正在研究这个solution，但是这看起来并没有涵盖我给出的例子。

一个更普遍的功能将是这样的：

def split_at(text, delimiter, exceptions): 
    """Split text at the specified delimiter if the delimiter is not 
    within the exceptions""" 

一些应用会是这样的：

split_at('obj<1, 2, 3>, x(4, 5), "msg, with comma"', ',', [('<', '>'), ('(', ')'), ('"', '"')] 

会的正则表达式能够处理这个还是有必要建立一个专门的解析器？

Answer 1

虽然它无法使用正则表达式，下面简单的代码将达到预期的效果：

def split_at(text, delimiter, opens='<([', closes='>)]', quotes='"\''): 
    result = [] 
    buff = "" 
    level = 0 
    is_quoted = False 

    for char in text: 
     if char in delimiter and level == 0 and not is_quoted: 
      result.append(buff) 
      buff = "" 
     else: 
      buff += char 

      if char in opens: 
       level += 1 
      if char in closes: 
       level -= 1 
      if char in quotes: 
       is_quoted = not is_quoted 

    if not buff == "": 
     result.append(buff) 

    return result 

在解释运行此：

>>> split_at('obj<1, 2, 3>, x(4, 5), "msg, with comma"', ',')                                 
#=>['obj<1, 2, 3>', ' x(4, 5)', ' "msg with comma"'] 

Answer 2

如果你有递归嵌套表达式，你可以在逗号分割并验证它们是否与pyparsing相匹配：

import pyparsing as pp 

def CommaSplit(txt): 
    ''' Replicate the function of str.split(',') but do not split on nested expressions or in quoted strings''' 
    com_lok=[] 
    comma = pp.Suppress(',') 
    # note the location of each comma outside an ignored expression: 
    comma.setParseAction(lambda s, lok, toks: com_lok.append(lok)) 
    ident = pp.Word(pp.alphas+"_", pp.alphanums+"_") # python identifier 
    ex1=(ident+pp.nestedExpr(opener='<', closer='>')) # Ignore everthing inside nested '< >' 
    ex2=(ident+pp.nestedExpr())      # Ignore everthing inside nested '()' 
    ex3=pp.Regex(r'("|\').*?\1')      # Ignore everything inside "'" or '"' 
    atom = ex1 | ex2 | ex3 | comma 
    expr = pp.OneOrMore(atom) + pp.ZeroOrMore(comma + atom) 
    try: 
     result=expr.parseString(txt) 
    except pp.ParseException: 
     return [txt] 
    else:  
     return [txt[st:end] for st,end in zip([0]+[e+1 for e in com_lok],com_lok+[len(txt)])]    


tests='''\ 
obj<1, 2, 3>, x(4, 5), "msg, with comma" 
nesteobj<1, sub<6, 7>, 3>, nestedx(4, y(8, 9), 5), "msg, with comma" 
nestedobj<1, sub<6, 7>, 3>, nestedx(4, y(8, 9), 5), 'msg, with comma', additional<1, sub<6, 7>, 3> 
bare_comma<1, sub(6, 7), 3>, x(4, y(8, 9), 5), , 'msg, with comma', obj<1, sub<6, 7>, 3> 
bad_close<1, sub<6, 7>, 3), x(4, y(8, 9), 5), 'msg, with comma', obj<1, sub<6, 7>, 3) 
''' 

for te in tests.splitlines(): 
    result=CommaSplit(te) 
    print(te,'==>\n\t',result) 

个

打印：

obj<1, 2, 3>, x(4, 5), "msg, with comma" ==> 
    ['obj<1, 2, 3>', ' x(4, 5)', ' "msg, with comma"'] 
nesteobj<1, sub<6, 7>, 3>, nestedx(4, y(8, 9), 5), "msg, with comma" ==> 
    ['nesteobj<1, sub<6, 7>, 3>', ' nestedx(4, y(8, 9), 5)', ' "msg, with comma"'] 
nestedobj<1, sub<6, 7>, 3>, nestedx(4, y(8, 9), 5), 'msg, with comma', additional<1, sub<6, 7>, 3> ==> 
    ['nestedobj<1, sub<6, 7>, 3>', ' nestedx(4, y(8, 9), 5)', " 'msg, with comma'", ' additional<1, sub<6, 7>, 3>'] 
bare_comma<1, sub(6, 7), 3>, x(4, y(8, 9), 5), , 'msg, with comma', obj<1, sub<6, 7>, 3> ==> 
    ['bare_comma<1, sub(6, 7), 3>', ' x(4, y(8, 9), 5)', ' ', " 'msg, with comma'", ' obj<1, sub<6, 7>, 3>'] 
bad_close<1, sub<6, 7>, 3), x(4, y(8, 9), 5), 'msg, with comma', obj<1, sub<6, 7>, 3) ==> 
    ["bad_close<1, sub<6, 7>, 3), x(4, y(8, 9), 5), 'msg, with comma', obj<1, sub<6, 7>, 3)"] 

当前的行为就像'(something does not split), b, "in quotes", c'.split',')包括保持前导空格和引号。从田间剥去报价和领先空间是微不足道的。

更改else下try到：使用迭代器和发电机

else: 
    rtr = [txt[st:end] for st,end in zip([0]+[e+1 for e in com_lok],com_lok+[len(txt)])] 
    if strip_fields: 
     rtr=[e.strip().strip('\'"') for e in rtr] 
    return rtr 

Answer 3

：

def tokenize(txt, delim=',', pairs={'"':'"', '<':'>', '(':')'}): 
    fst, snd = set(pairs.keys()), set(pairs.values()) 
    it = txt.__iter__() 

    def loop(): 
     from collections import defaultdict 
     cnt = defaultdict(int) 

     while True: 
      ch = it.__next__() 
      if ch == delim and not any (cnt[x] for x in snd): 
       return 
      elif ch in fst: 
       cnt[pairs[ch]] += 1 
      elif ch in snd: 
       cnt[ch] -= 1 
      yield ch 

    while it.__length_hint__(): 
     yield ''.join(loop()) 

，并

>>> txt = 'obj<1, sub<6, 7>, 3>,x(4, y(8, 9), 5),"msg, with comma"' 
>>> [x for x in tokenize(txt)] 
['obj<1, sub<6, 7>, 3>', 'x(4, y(8, 9), 5)', '"msg, with comma"']