Streaming Large Data Responses with Rails
I recently needed to stream a large CSV from a reporting interface and the streaming docs in Rails are mostly geared towards streaming template rendering. This allows you to stream any response object that responds to #each
and can be used in any class that extends ActionController::Metal
class StreamingController < ApplicationController
def index
headers['X-Accel-Buffering'] = 'no' # Stop NGINX from buffering
headers["Cache-Control"] = "no-cache" # Stop downstream caching
headers["Transfer-Encoding"] = "chunked" # Chunked response header
headers.delete("Content-Length") # See one line above
# Anything that responds to #each can be used for a response_body
self.response_body = Enumerator.new do |lines|
# Uses ActiveRecord Batch API to find records efficently
Model.find_each do |record|
lines << record.to_csv
end
end
end
end
Written by Craig McNamara
Related protips
3 Responses
Thank you for this example. May I ask which webserver you are using? Because setting
headers["Transfer-Encoding"] = "chunked"
breaks the network request for me on Puma.
over 1 year ago
·
Is there no possibility to edit a comment? I just wanted to mention that I added a comment in the puma repository and referenced your example.
over 1 year ago
·
Hi Craig
I would like to understand what does your to_csv function doing here
over 1 year ago
·
Have a fresh tip? Share with Coderwall community!
Post
Post a tip
Best
#Rails
Authors
Sponsored by #native_company# — Learn More
#native_title#
#native_desc#