2017-07-06 45 views
0

我正在清理我的生产数据集,我正在寻找一些无用的条目。Bigquery查找条目所属的表

SELECT count(pid_like) FROM TABLE_DATE_RANGE(DATASET.TABLE_PRODUCTION_, DATE_ADD(CURRENT_TIMESTAMP(), -30, 'DAY'),CURRENT_TIMESTAMP()) where c1 is null and c2 is null and c3 is null and c4 is null 

所以我想删除其中C1,C2,C3和C4空条目。

之前,我在最后30桌清洁为由:

# done for each last 30 tables 
DELETE FROM DATASET.TABLE_PRODUCTION_YYYYMMDD where c1 is null and c2 is null and c2 is null and c4 is null. 

但清洗后,一些无用的条目将会保留,但我不能对他们是什么台找到。

回答

2

试试这个发现哪些表行与空值:

#standardSQL 
SELECT 
    _TABLE_SUFFIX AS suffix, 
    COUNT(*) AS null_count 
FROM `DATASET.TABLE_PRODUCTION_*` 
WHERE _TABLE_SUFFIX BETWEEN 
    FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)) AND 
    FORMAT_DATE('%Y%m%d', CURRENT_DATE()) AND 
    c1 IS NULL AND 
    c2 IS NULL AND 
    c3 IS NULL AND 
    c4 IS NULL 
GROUP BY suffix 
HAVING null_count > 0 
ORDER BY null_count DESC; 

它将与空行和计数返回后缀(日期)。

+0

不错!你能指点我的相关文件吗? –

+1

也许这个? https://cloud.google.com/bigquery/docs/querying-wildcard-tables#filtering_selected_tables_using_table_suffix。这里是FORMAT_DATE,但您也可以在这里找到其他功能:https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#format_date –

+0

非常感谢:) –