长期数据采集

Question

我想运行一个Python脚本，是为长期实验室数据采集（〜1个月）。下面的脚本应该从SQL数据库获取数据，并从DAQ设备获取数据并将其保存到文本文件中。然后实时绘制1小时的数据。实时绘图由于记忆错误而被注释掉了，但这不是我关心的问题。

运行了3-4天之间的代码给我下面的错误：

Trackback (most recent call last): 
file *** line 105 in <module> 
    deltay=float(float(tupline[3]/10-float(tupline2[3])/10) 
TypeError: 'NoneType' object has to attribute '__getitem__' 

我如何能得到这个脚本对于较长时间运行有什么想法？

import ctypes 
from UniversalLibrary import * 
import time 
import datetime 
from numpy import * 
from scipy import * 
import MySQLdb as mdb 
import matplotlib 
matplotlib.use('Agg') 
import matplotlib.pyplot as plt 
import random 



boardNum=0 
portNum=10 
inputVoltage = 3 



BoardNum = 0 
Gain = BIP5VOLTS 
Chan = 0 



spTimes =14*24*60*60  ##here you set the time 
timeDuration=spTimes 
padsToVisualize =[0,1,2,3,4] ##pads that will be measured and visualized 

#plt.ion() ##this command is necassary in order to have new points on the plot without erasing the old ones 

curTime=time.clock() 
movingWinDur=60 ##seconds 
print "curtime=",curTime 

str_one_line=[] 
straverage=[] 
while (curTime < timeDuration): 
    del str_one_line[:] 
    padCounter=0; 

    d=datetime.datetime.now() 
    str_one_line=[(d.strftime("%Y-%m-%d %H:%M:%S")),','] ##formatting the date -- this format is chosen based on the requirements for connecting to SQL database 

    while (padCounter <len(padsToVisualize)): 

     #fig1=plt.figure(padsToVisualize[padCounter]) ## making figures for plotting different pads 


     print "pad No.: ", str(padsToVisualize[padCounter]) 
     #l=cbAOut(BoardNum,Chan,Gain,3019) ## sets the voltage to 3.0 volts (the number 3019 is based on 12-bit resolution --> DAQ resolution) 
     #n=cbDConfigPort(BoardNum,portNum,1) ##the variable n serves aas a command line where the command is sent to the DAQ 
     #b=cbDOut(BoardNum,portNum,(63-padsToVisualize[padCounter])) #####based on Prof. Mayer's solution, the line sends the pad number that we are interested in to the DAQ to get the signals 
     curTime=time.clock() 
     d=datetime.datetime.now() 
     ######################################################################### 

     ########## this part will use the datetime to fetch the temperature from the SQL data base 
     ########## since the time stamps may not exactly match, the code uses a method of interpolation 
     freq = 5 
     time.sleep(1.8*freq) ## 1.8 is an arbitrary number ; to make sure there is data on sql database 
     searchfor=d ## the following lines find the temperature at time=searchfor 
     print searchfor 
     minussearchfor = searchfor-datetime.timedelta(0,freq/1.2) 
     STRminussearchfor = minussearchfor.strftime("%Y-%m-%d %H:%M:%S") 
     print "STRminussearchfor=", STRminussearchfor 
     print "minussearchfor= ", minussearchfor 
     plussearchfor =searchfor+datetime.timedelta(0,freq/1.2) 
     print "plussearchfor= ", plussearchfor 
     STRplussearchfor = plussearchfor.strftime("%Y-%m-%d %H:%M:%S") 
     print "STRplussearchfor=", STRplussearchfor 
     ##Database connection 
     db = mdb.connect("localhost", "root", "qwerty@12345", "bondtest") 
     cur = db.cursor() 
     cur.execute("SELECT * FROM bondtest_data WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime",{'date1':STRminussearchfor,'date2':STRplussearchfor}) 
##  con=pymssql.connect(host='LAB10-PC\SQLEXPRESS2008R2',user='sa',password='qwerty@12345') 
##  cur = con.cursor() 
##  cur.execute('SELECT * FROM OVEN11SQL.dbo.OvenLog1 WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime',{'date1':STRminussearchfor,'date2':STRplussearchfor}) 
     tupline1 = cur.fetchone() 
##  print 'between1= ',tupline1[1] 
     delta = tupline1[1]-searchfor 
##  print "delta = " ,delta 
     if (int(searchfor.strftime("%S"))-int(tupline1[1].strftime("%S"))>0): 
      delta = int(searchfor.strftime("%S"))-int(tupline1[1].strftime("%S")) 
     else: 
      delta = int(tupline1[1].strftime("%S"))-int(searchfor.strftime("%S")) 
##  print 'delta= ',delta 
     time1=tupline1[1]-datetime.timedelta(0,freq/1.2) 
     STRtime1=time1.strftime("%Y-%m-%d %H:%M:%S") 
     time2=tupline1[1]-datetime.timedelta(0,3*freq/1.2) 
     STRtime2=time2.strftime("%Y-%m-%d %H:%M:%S") 
##  time.sleep(2*freq) ##the program needs to wait for 3*frequency/2 so that the next point is available in SQL data base for interpolation 
     cur.execute('SELECT * FROM bondtest_data WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime',{'date1':STRtime2,'date2':STRtime1}) 
     tupline2 = cur.fetchone() ##next point is fetched in order to find the slope of the line for temperature change 
##  print 'between2= ', tupline2[1] 
     deltay=float(float(tupline1[3])/10-float(tupline2[3])/10) 
     deltax = int(tupline1[1].strftime("%S"))-int(tupline2[1].strftime("%S")) 
     deltax = freq 
##  print "deltay= ", deltay 
##  print "deltax= ", deltax 
     slope = deltay/deltax 
##  print 'slope= ', slope 

     ##in the following if statements depending on whether the temperature is increasing or decreasing the temperature for the desired point will be calculated 
     if (tupline2[3]<tupline1[3]): 
      tempsearchfor = float(tupline1[3])/10+delta*slope 
##   print '+delta*slope= ',delta*slope 
     elif (tupline2[3]>tupline1[3]): 
      tempsearchfor = float(tupline1[3])/10-delta*slope 
##   print '-delta*slope= ',delta*slope 
     else: 
      tempsearchfor = float(tupline1[3])/10 
##  print 'tempserachfor= ',tempsearchfor 

     ######################################################################### 

     strng = [str(int(padsToVisualize[padCounter])),',',(d.strftime("%Y-%m-%d %H:%M:%S")),',',str(round(curTime,4)),',',str(inputVoltage),','] 
     str_one_line.extend(strng)   ##str_one_line is the line that contains the values that will be written to the text file , the order is specified in the variables lists excel file 
     xyzCounter=Chan 
     EngUnits= array(range(50*1),dtype=float).reshape(50,1) ## constructing the array that will hold the 50 values for each set of signals and is later used for finding the average 
     average = array(range(3*1),dtype=float).reshape(3,1)  ## holds the average of the t50 points for x,y,z 
##  straverage=array(range(3*1),dtype=str).reshape(3,1) ##this array will hold the strings for x,y,z signal to be written into txtfile 
     del straverage[:] 
     while(xyzCounter<3): ##the way the oven is set up it has x,y,z outputs from channel 0,1,2 accordingly 
      #n=cbDConfigPort(BoardNum,portNum,1) 
      #b=cbDOut(BoardNum,portNum,(63-padsToVisualize[padCounter]))  #####based on Prof. Mayer's solution, the line sends the pad number that we are interested in to the DAQ to get the signals 
      a=0 
      time1=time.clock() 
      while (a<50): ## this while loop takes 50 measurements and writes the average of those points in to average array 

       #DataValue=cbAIn(BoardNum, xyzCounter, Gain) 
       #EngUnits[a,0]=float((cbToEngUnits(BoardNum,Gain,DataValue))/3/100*1000) 
       EngUnits[a,0] = random.uniform(0,0.5) 
       average[xyzCounter,0]=float(sum(EngUnits))/len(EngUnits) 
       a+=1 
      time2=time.clock() 
      timePerSample=(time2-time1)/50 
      print "time per sample= ",timePerSample 
      print "samples per second =", (1/timePerSample) ##measuring the frequency of the emasurements 
      tempstr=str(round((average[xyzCounter,0]),3)) ##in order to remove the two brackets temp refers to temporary 
      #tempstr=tempstr[1:-1] 
      straverage.append(tempstr) 
      xyzCounter+=1 


     #print average 
     temperaturearray=array(range(1*1),dtype=str).reshape(1,1) 
     temperaturearray=[str(tempsearchfor)] 
     three_sens_signals=array(range(1*5),dtype=str).reshape(1,5) 
     three_sens_signals=[str((straverage[0])),',',str((straverage[1])),',',str((straverage[2])),','] 
     str_one_line.extend(three_sens_signals) 
     str_one_line.extend(temperaturearray) 
     str_one_line.extend(',') 

     padCounter+=1 
     filename='log_simulation.txt' 
     f = open(filename,"a") 
     ## writing to file section 
    print "padcounter=",padCounter," str_one_line=", str_one_line 
    for item in str_one_line: 
     f.write("%s" % item) 
    f.write("\n") 
    f.close() 

    curTime=time.clock() 

Answer 1

总之，有很多需要加以改进/这段代码返工的事情（编辑所有进口*语句和使用的命名空间应该像你开始）。话虽如此（代码显然大多是按原样工作），但问题的发生是因为无论出于何种原因，存储在tupline或tupline2中的值都是None。 （无论何种原因的数据或表损坏），这会导致它返回“无”，而且在你的代码，这些变量是使用SQL语句

tupline = cur.fetchone() 
tupline2 = cur.fetchone() 

其中一个电话显然是不顺利运行分配。也许有人正在删除或替换表格？您可以添加一些语义来检查这种情况，报告并尝试继续。一些东西沿线

if tupline == None or tupline2 == None: 
    print "Why does tupline = None?" 
    continue 

你应该弄清sql .fetchone（）方法何时返回无。真的，你应该做一些更好的日志记录，但是这可能会让你进入下一个调试阶段。

Answer 2

正如Paul Seeb所述，错误信息发生是因为tupline或tupline2是None，当代码到达deltay=...行时。您的代码（成功）在分配后访问tupline元素，然后到达deltay行。所以问题必须与tupline2。

我的猜测是数据库不包含tupline2查询的匹配记录;这肯定是要检查的东西。我还注意到，您在tupline2查询之前发布了延迟注释，表明此查询不能可靠地返回记录。我不知道你的程序逻辑，但你也许能够只用tupline2查询后，像这样的线路解决该问题：

if tupline2 is None: 
    continue` 

，或者直到它成功

while True: 
    cur.execute(
     """SELECT * FROM bondtest_data 
      WHERE DateTime BETWEEN %(date1)s AND %(date2)s 
      ORDER BY DateTime""", 
     {'date1':STRtime2,'date2':STRtime1} 
    ) 
    tupline2 = cur.fetchone() 
    if tupline2 is None: 
     print "Failed to obtain a record for tupline2; trying again..." 
     time.sleep(<some duration>) 
    else: 
     break 

你可能会重试 tupline2查询

我还注意到，您的代码每次通过循环时都会创建一个新的数据库连接（db），并且永远不会明确地关闭它。我建议在主循环开始时移动db = mdb.connect(...)命令，或者在最后的fetchone()命令之后的某处添加db.close()。你的代码有可能达到某种连接限制，但这可能会引发一个不同的例外。当db被重新分配并且旧连接超出范围时，更可能您从某种自动关闭连接中受益。