Sometimes I am not sure how fast or how big a file based dataset grows and being stuck at 100% disk usage is really PITA. In comes gzip with a 30-40% compression ratio!
Use this snippet to pipe through gzip and write as usual:
# compress output with gzip import gzip with gzip.open('<your file>', 'wt') as f: f.write('<your data>')
Same procedure for reading from the compressed file:
# read compressed file import gzip with gzip.open('<your file>', 'rt') as f: text = f.read()
When using Pyton 3 you can change the mode from
xt if the file does not already exist and it will create it on the fly. Reading or writing to binary files through gzip is equally possible!