Last Updated: September 29, 2021
·
12.53K
· craigmcnamara

Streaming Large Data Responses with Rails

I recently needed to stream a large CSV from a reporting interface and the streaming docs in Rails are mostly geared towards streaming template rendering. This allows you to stream any response object that responds to #each and can be used in any class that extends ActionController::Metal

class StreamingController < ApplicationController
  def index
    headers['X-Accel-Buffering'] = 'no' # Stop NGINX from buffering
    headers["Cache-Control"] = "no-cache" # Stop downstream caching
    headers["Transfer-Encoding"] = "chunked" # Chunked response header
    headers.delete("Content-Length") # See one line above

    # Anything that responds to #each can be used for a response_body
    self.response_body = Enumerator.new do |lines|
      # Uses ActiveRecord Batch API to find records efficently
      Model.find_each do |record|
        lines << record.to_csv
      end
    end
  end
end

3 Responses
Add your response

Thank you for this example. May I ask which webserver you are using? Because setting

headers["Transfer-Encoding"] = "chunked"

breaks the network request for me on Puma.

over 1 year ago ·

Is there no possibility to edit a comment? I just wanted to mention that I added a comment in the puma repository and referenced your example.

over 1 year ago ·

Hi Craig
I would like to understand what does your to_csv function doing here

over 1 year ago ·