2017-09-23 25 views
1

所以目前我试着去得到一个功能权限。基本上,我尝试做多(兼)线程需要的信息从一个JSON文件到程序,然后为每个线程应该使用每个JSON对象,然后用这些信息来执行代码。Python多线程/ Json和Python的流程组合?

我迄今所做的 - 这个代码仅适用于多进程这的确作品:

#Read json File 
with open('profileMulti.json', 'r', encoding='UTF-8') as json_data: 
    profiles_string = json.load(json_data) 

def get_individual_profiles(config): 
    top_layer = config.get('Profiles') 
    if top_layer: 
     top_level_keys = ['profile_{}'.format(i) for i in range(len(top_layer))] 
     print(top_level_keys) 
     return [(key, top_layer.get(key)) for key in top_level_keys] 
    return [] 

def stringify(key, next_layer): 
    return [ 
     ' '.join(key.capitalize().split('_')), 
     next_layer.get('Name'), 
     next_layer.get('Last_Name'), 
     next_layer.get('Email'), 
     next_layer.get('Phone') 
     #etc etc... 
     ] 

    config = profiles_string 
profiles = get_individual_profiles(config) 

pool = ThreadPool() 

# Launch a process for each item 
threads = [pool.apply_async(stringify, tuple(item)) for item in profiles] 


# get() the results as each finishes 
results = [res.get() for res in threads] 
print('threaded results:') 
for item in results: 
    print(item) 

输出:

threaded results: 
['Profile 0', 'Thrill', 'Ofit', '[email protected]', '123 412 123'] 

['Profile 1', 'Hellow', 'World', '[email protected]', '543 412 312'] 

这是伟大的。但问题是:

,每当我想等使用这些信息

为例

def checkoutNames(NameUrl, nameID): 
payload = { 
     "shared": { 
      "challenge": { 
       "Name": item["Name"], 
       "Last_Name": item["Last_Name"], 
       "Email": item["Email"], 
       "Phone": item["Phone"], 

是它不会承认这些属性,但我需要做的是,我需要由项目1 ..项目[2]等从以便从def stringify(key, next_layer):我不知道,现在叫他们怎么做,所以我并不需要做。

的另一个问题是,每当我们说的ETC,我在代码中使用项目1。然后它将只使用最后一个线程并跳过其余部分。所以,如果我做

print(item[1]) 

那么这会是唯一的输出是最后一个是你好

所以我需要的问题得到解决:

1.每一线程并发执行,并与信息

2.执行代码做一个修复,所以我并不需要使用项目1,而是使用项目['Name']。

所以现在的问题是,这是可能的,什么是这个想法?

编辑 - 这是我目前只有一个Json的轮廓具有和工作正常只用一个配置文件的代码。这是一个没有多处理

JSON文件

 { 
    "Profiles": { 
     "profile_0": { 
      "Url": "Myownwebsite.se", 
      "My-Note": "Helloworld", 
      "Email": "[email protected]" 
      "PersonNumber": "1234543", 
      "postal_code": "54123", 
      "given_name": "World", 
      "Last_name": "Hellow", 
      "street_address": "helloworld 123", 
      "city": "Stockholm", 
      "country": "Sweden", 
      "phone": "123456789", 
      "Color": "Red", 
      "house_number": "123", 
      "year": "2017" 
     }, 
     "profile_1": { 
      "Url": "Myasdwfaesite.se", 
      "My-Note": "aasfase", 
      "Email": "[email protected]" 
      "PersonNumber": "5634543", 
      "postal_code": "123445", 
      "given_name": "Balling", 
      "Last_name": "Calling", 
      "street_address": "qwertr 123", 
      "city": "London", 
      "country": "UK", 
      "phone": "65412331", 
      "Color": "Blue", 
      "house_number": "321", 
      "year": "2018" 
     } 

     #Profile_2 etc etc 
    } 
} 

代码

with open('profileMulti.json', 'r', encoding='UTF-8') as json_data: 
    config = json.load(json_data) 

NameUrl = config["Url"] 

myNote = config["My-Note"] 

def checkoutNames(NameUrl, nameID): 

#Request & other codes - Removed to recude the code 
#...... 
#...... 
    headers = { 
     'Referer': '', 
     'Content-Type': '' 
    } 
    payload = { 
     "shared": { 
      "challenge": { 
       "email": config["Email"], 
       "PersonNumber": config["PersonNumber"], 
       "postal_code": config["ZipCode"], 
       "given_name": config["Name"], 
       "Last_name": config["LastName"], 
       "street_address": config["Address"], 
       "postal_code": config["ZipCode"], 
       "city": config["City"], 
       "country": config["Country"], 
       "email": config["Email"], 
       "phone": config["Phone"], 
      } 

def checkoutNotes(NamesUrl, NamesPost): 

#Request & other codes - Removed to recude the code 
#...... 
#...... 

    headers = { 
     'Accept': 'application/json, text/javascript, /; q=0.01', 
     'Accept-Language': 'en-US,en;q=0.5', 
     'Accept-Encoding': 'gzip, deflate, br', 
     'Referer': NameUrl, 
     'Connection': 'keep-alive' 
    } 
    payloadInfo = { 
     "Information": { 
      "Color": config["Color"], 
      "house_number": config["houseNumber"], 
      "year": config["Year"] 
     } 
    }  
def wipe(): 
    os.system('cls' if os.name == 'nt' else 'clear') 

def main(): 
    time.sleep(1) 

    FindName(myNote) 

if _name_ == '_main_': 
    try: { 
     main() 
    } 
    except KeyboardInterrupt: 
     wipe() 

编辑

我只是想打印出什么雅罗斯拉夫代码提供输出

enter image description here

回答

1

你做出列表并且像字典一样工作。如果订单很重要 - 从集合中使用。顺便说一句,你可以加载你的JSON使用这样的事情:

from collections import OrderedDict 
.... # your previous code 
profiles_string = json.load(json_data, object_pairs_hook=OrderedDict) 

尝试使用此代码:

import json 
from collections import OrderedDict 
from multiprocessing.pool import ThreadPool 
import threading 
from time import sleep 
import random 

with open('profileMulti.json', 'r', encoding='UTF-8') as json_data: 
    config = json.loads(json_data, object_pairs_hook=OrderedDict) 


def stringify(key, next_layer): 
    sleep(random.random()) # work emulation 
    # `key` is profile name (profile_0, profile_1 etc) 
    # `next_layer` is profile payload as OrderedDict: 
    # { 
    # "Url": "Myasdwfaesite.se", 
    # "My-Note": "aasfase", 
    # "Email": "[email protected]", 
    # "PersonNumber": "5634543", 
    # "postal_code": "123445", 
    # "given_name": "Balling", 
    # "Last_name": "Calling", 
    # "street_address": "qwertr 123", 
    # "city": "London", 
    # "country": "UK", 
    # "phone": "65412331", 
    # "Color": "Blue", 
    # "house_number": "321", 
    # "year": "2018" 
    #} 

    # this print statement just for illustartion 
    print(threading.current_thread().name, key) 
    return { key: next_layer } # still ordered 

# how much workers should be started 
profiles_number = len(config.get('Profiles', {}).items()) 

pool = ThreadPool(profiles_number) 

# Launch a thread for each item and get() the results as each finishes 
# `starmap_async` returns result object when all tasks are finished, so 
# single get is called. Result object in thios case behave like list 
results = pool.starmap_async(stringify, config.get('Profiles', {}).items()).get() 

print('threaded results:') 
for i in results: 
    print(i, end='\n***\n') # result separator when printed 
+0

谢谢@Yaroslav!但是,当我尝试这一点。它不起作用。我得到的印刷品是[无,无]。什么可能导致这个问题?也许是因为'''print(item)'''没有任何东西?也如果我想从线程打印出来的Url。在这种情况下不会打印(配置['Url'])完成它? – WeInThis

+0

@WeInThis,对不起,双'返回'声明是错误的。尝试编辑的代码,应该没问题 –

+0

再次嘿!好吧,它的工作。但是我发现另一个问题。所以现在发生的事情是,它只是加载到一个线程中,并且在该线程内部是包括所有配置文件在内的所有对象的整个打印。我不认为它应该这样做。但也是。如果我想要抓取URL,情况会怎样。等'''Url =“从profile_0抓取url”并打印(Url)'''? – WeInThis