2017-04-07 61 views
0

在Ansible 2.2中,我想遍历从S3中读取的大量文件。ansible包含数组切片

这里是我的role/tasks/main.yml

 
    - name: Simulate variable read from S3 
    set_fact: 
     list_of_files_to_import: [ 
     "a.tar.gz", 
     "b.tar.gz", 
     "c.tar.gz", 
     "d.tar.gz", 
     "e.tar.gz", 
     "f.tar.gz", 
     "g.tar.gz", 
     "h.tar.gz", 
     ... 
     "zz.tar.gz" 
     ] 

    - name: Process each file from S3 
    include: submodule.yml 
    with_items: list_of_files_to_import 

这里是role/tasks/submodule.yml

 
--- 
    - name: Restore TABLE {{ item }} 
    debug: var={{ item }} 

这崩溃,因为有太多的文件。

我发现,我可以切片的阵列,并在同一时间发送部分:

 
    - name: Process each file from S3 
    include: submodule.yml 
    with_items: "{{ list_of_files_to_import[0:5] }}" 

    - name: Process each file from S3 
    include: submodule.yml 
    with_items: "{{ list_of_files_to_import[5:10] }}" 

    - name: Process each file from S3 
    include: submodule.yml 
    with_items: "{{ list_of_files_to_import[10:15] }}" 

    - name: Process each file from S3 
    include: submodule.yml 
    with_items: "{{ list_of_files_to_import[15:20] }}" 

而是硬编码这些小块,我想尝试像

 
    - name: Process each file from S3 
    include: submodule.yml 
    with_items: "{{ list_of_files_to_import[{{start}}:{{end}}] }}" 

但我们cannot get variable-defined variable names

如何处理Ansible 2.2中的大量项目?

+0

列表有多大?崩溃的原因是什么?太多包括? –

+0

它是300个文件的顺序。我看到的唯一原因是“内存不足”。但是,是的,它似乎不能处理这么多包括。 –

+1

我不确定您的详情,但它看起来像问题[#16391](https://github.com/ansible/ansible/issues/16391)。如果是这样的话,这个问题应该在下一个Ansible版本中解决。 – nelsonda

回答

0

我最终用shell脚本解决了这个问题,反复地用一些--extra-vars来调用playbook来指定要处理的文件。

这仅适用于S3中的文件列表具有相似的文件名。基本上它循环遍历文件名并一次处理每一个文件。

 
#!/usr/bin/env bash 

# return 0(true) if the year/month combination is valid 
valid() { 
    yr=$1 
    mn=$2 
    # do not run on months after this month 
    if [ "$yr" -eq "2017" -a "$mn" -gt "$(date +%m)" ] 
    then 
    return 1 
    fi 

    return 0 
} 

# For every year from 2002 to this year 
for year in `seq 2002 $(date +%Y)` 
do 
    # for every zero-padded month, 01-12 
    for month in `seq -f "%02g" 01 12` 
    do 
     # for each type of item in inventory 
     for object in widgets doodads 
     do 
      if valid $year $month; 
      then 
      ansible-playbook playbook_name.yml --extra-vars "object=$object year=$year month=$month" 
      else 
      echo skipping invalid combo $object $year $month 
      fi 
     done 
    done 
done