A quick script to ZSTD all your shredded tables

Friday 25 August, 2017 | By: Simon Rumble

Mike’s recent post about compressing Snowplow tables works great for atomic.events, with clients seeing compression down to 30% of the original size or so. But what about all your shredded tables?

For now you have to manually convert the output from igluctl while we wait for our pull request to make it into a release, from then on this will be automatic. There’s also a pull request in to change the default compression for atomic.events too.

Run the following code from the command line from the root of your schema definitions and it’ll automatically convert everything relevant to ZSTD. This was written by one of our staff.



We exist to make organisations better understand their businesses by enabling all decision makers in a company to work with the same version of the truth.

Social Links