而是再加载6行,然后将 字符串 传递给
json.loads():
with open(file) as f: for line in f: # slice the next 6 lines from the iterable, as a list. lines = [line] + list(itertools.islice(f, 6)) jfile = json.loads(''.join(lines)) # do something with jfile
json.load()不仅会吞噬文件中的下一个对象,而且
islice(f, 0, 7)只会读取前7行,而不是按7行块读取文件。
您可以在生成器中以大小N的块包装读取文件:
from itertools import islice, chaindef lines_per_n(f, n): for line in f: yield ''.join(chain([line], itertools.islice(f, n - 1)))
然后使用它来分块您的输入文件:
with open(file) as f: for chunk in lines_per_n(f, 7): jfile = json.loads(chunk) # do something with jfile
另外,如果您的块结果是可变长度的,请阅读直到有可以解析的内容:
with open(file) as f: for line in f: while True: try: jfile = json.loads(line) break except ValueError: # Not yet a complete JSON value line += next(f) # do something with jfile
欢迎分享,转载请注明来源:内存溢出
评论列表(0条)