Generate Redis protocol from csv using Awk
An example approach for reformatting csv data into Redis protocol, to be bulk loaded with the console client. Data is binned into 5 minute samples for time series analysis.
awk -f redis-proto.awk data.csv | redis-cli --pipe
redis-proto.awk:
BEGIN { FS="," }
{
bin=sprintf("%s%04d", strftime("%Y%m%d",$2), $1 - ($1 % 5))
fld=sprintf("%s:%s", $3, $4)
rate=int(($5) / 1000)
print "*4\r\n$7\r\nHINCRBY\r\n$12\r\n" bin "\r\n$" length(fld) "\r\n" fld "\r\n$" length(rate) "\r\n" rate "\r"
}
Written by Matt Keranen
Related protips
2 Responses
I think you're missing a \n
after rate "\r"
, but other than that, this is pretty cool!
over 1 year ago
·
IIRC that was because the final \n is added to the line automatically by the print statement. Redis protocol does not like extra carriage returns. I could probably use a printf and be more explicit.
over 1 year ago
·
Have a fresh tip? Share with Coderwall community!
Post
Post a tip
Best
#Redis
Authors
Sponsored by #native_company# — Learn More
#native_title#
#native_desc#