Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IOError: closed stream #467

Open
alagos opened this issue Jan 10, 2017 · 3 comments
Open

IOError: closed stream #467

alagos opened this issue Jan 10, 2017 · 3 comments

Comments

@alagos
Copy link

alagos commented Jan 10, 2017 •
edited
Loading

Recently I've added a new validation to my dragonfly model

validates_size_of :file, maximum: 10.megabytes

and it works great in my app, but since then, my specs started to fail:

     IOError:
       closed stream
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/dragonfly-1.1.1/lib/dragonfly/temp_object.rb:104:in `size'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/dragonfly-1.1.1/lib/dragonfly/temp_object.rb:104:in `size'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validations/length.rb:42:in `validate_each'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validator.rb:151:in `block in validate'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validator.rb:148:in `each'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validator.rb:148:in `validate'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:453:in `public_send'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:453:in `block in make_lambda'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:190:in `call'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:190:in `block in simple'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:502:in `call'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:502:in `block in call'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:502:in `each'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:502:in `call'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:88:in `run_callbacks'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validations.rb:395:in `run_validations!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validations/callbacks.rb:113:in `block in run_validations!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:84:in `run_callbacks'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validations/callbacks.rb:113:in `run_validations!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activemodel-4.2.3/lib/active_model/validations.rb:334:in `valid?'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/validations.rb:58:in `valid?'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/validations.rb:83:in `perform_validations'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/validations.rb:43:in `save!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/transactions.rb:291:in `block in save!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/transactions.rb:220:in `transaction'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/transactions.rb:348:in `with_transaction_returning_status'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activerecord-4.2.3/lib/active_record/transactions.rb:291:in `save!'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/configuration.rb:18:in `block in initialize'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/evaluation.rb:15:in `[]'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/evaluation.rb:15:in `create'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy/create.rb:12:in `block in result'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy/create.rb:9:in `tap'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy/create.rb:9:in `result'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/factory.rb:42:in `run'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/factory_runner.rb:29:in `block in run'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/activesupport-4.2.3/lib/active_support/notifications.rb:166:in `instrument'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/factory_runner.rb:28:in `run'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy_syntax_method_registrar.rb:20:in `block in define_singular_strategy_method'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy_syntax_method_registrar.rb:28:in `block (2 levels) in define_list_strategy_method'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy_syntax_method_registrar.rb:28:in `times'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy_syntax_method_registrar.rb:28:in `each'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy_syntax_method_registrar.rb:28:in `map'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/strategy_syntax_method_registrar.rb:28:in `block in define_list_strategy_method'
     # /home/alter/.rvm/gems/ruby-2.2.5@project/gems/factory_girl-4.7.0/lib/factory_girl/evaluator.rb:42:in `method_missing'
     # ./spec/factories/messages.rb:37:in `block (3 levels) in <top (required)>'

My model and factory has nothing special (I guess):

class Attachment < ActiveRecord::Base
  extend Dragonfly::Model::Validations

  belongs_to :message

  dragonfly_accessor :file

  validates_size_of :file, maximum: 10.megabytes
  validates_property :ext, of: :file, case_sensitive: false,
                           in: %w(pdf doc docx xls xlsx gif jpg jpeg png)
end

FactoryGirl.define do
  factory :message do
    from { create(:user) }
    to { create(:client) }
    subject 'Konnichiwa!'
    body '<b>Hello World</b>'
    order

    trait :with_attachment do
      attachments { create_list(:attachment, 1) }
    end
  end

  factory :attachment do
    file do
      ActionDispatch::Http::UploadedFile.new(
        filename: 'quokka.jpg', type: 'image/png',
        tempfile: File.new("#{Rails.root}/spec/fixtures/quokka.jpg")
      )
    end
  end
end

I checked this issue's comment, but I don't have that meta_request gem in my project. And I repeat, that attachment factory is working ok without the size validation.
Any ideas what it could be?
@alagos
Copy link
Author

alagos commented Nov 2, 2017 •
edited
Loading

I'm having this issue in production right now, so I had no other choice than dig deeper into it.

I've debugged with pry where the temp object is initialized to see what I'm receiving or why the test is different than a normal request, so I started with the successful case (uploading a file through a request):

From: /home/alter/.rvm/gems/ruby-2.2.5@productwatcher/gems/dragonfly-1.1.3/lib/dragonfly/temp_object.rb @ line 58 Dragonfly::TempObject#initialize:

    53:         @pathname = Pathname.new(obj.path)
    54:       elsif obj.is_a? Pathname
    55:         @pathname = obj
    56:       elsif obj.respond_to?(:tempfile)
    57:         binding.pry
 => 58:         @tempfile = obj.tempfile
    59:       elsif obj.respond_to?(:path) # e.g. Rack::Test::UploadedFile
    60:         @pathname = Pathname.new(obj.path)
    61:       else
    62:         raise ArgumentError, "#{self.class.name} must be initialized with a String, a Pathname, a File, a Tempfile, another TempObject, something that responds to .tempfile, or something that responds to .path - you gave #{obj.inspect}"
    63:       end

[5] pry(#<Dragonfly::TempObject>)> obj
=> #<ActionDispatch::Http::UploadedFile:0x007fe6b0a88538
 @content_type="image/jpeg",
 @headers="Content-Disposition: form-data; name=\"message[attachments_attributes][][file]\"; filename=\"17632406_1250875254965762_2239186526359357087_o.jpg\"\r\nContent-Type: image/jpeg\r\n",
 @original_filename="17632406_1250875254965762_2239186526359357087_o.jpg",
 @tempfile=#<File:/tmp/RackMultipart20171102-17032-1n6uh19.jpg>>
[6] pry(#<Dragonfly::TempObject>)> n

From: /home/alter/.rvm/gems/ruby-2.2.5@productwatcher/gems/dragonfly-1.1.3/lib/dragonfly/temp_object.rb @ line 65 Dragonfly::TempObject#initialize:

    60:         @pathname = Pathname.new(obj.path)
    61:       else
    62:         raise ArgumentError, "#{self.class.name} must be initialized with a String, a Pathname, a File, a Tempfile, another TempObject, something that responds to .tempfile, or something that responds to .path - you gave #{obj.inspect}"
    63:       end
    64: 
 => 65:       @tempfile.close if @tempfile
    66: 
    67:       # Name
    68:       @name = if name
    69:         name
    70:       elsif obj.respond_to?(:original_filename)

[6] pry(#<Dragonfly::TempObject>)> @tempfile
=> #<File:/tmp/RackMultipart20171102-17032-1n6uh19.jpg>
[7] pry(#<Dragonfly::TempObject>)> @tempfile.size
=> 103707

as you can see, there is no problem to get the tempfile size before this is being closed.

[8] pry(#<Dragonfly::TempObject>)> n

From: /home/alter/.rvm/gems/ruby-2.2.5@productwatcher/gems/dragonfly-1.1.3/lib/dragonfly/temp_object.rb @ line 68 Dragonfly::TempObject#initialize:

    63:       end
    64: 
    65:       @tempfile.close if @tempfile
    66: 
    67:       # Name
 => 68:       @name = if name
    69:         name
    70:       elsif obj.respond_to?(:original_filename)
    71:         obj.original_filename
    72:       elsif @pathname
    73:         @pathname.basename.to_s

[8] pry(#<Dragonfly::TempObject>)> @tempfile
=> #<File:/tmp/RackMultipart20171102-17032-1n6uh19.jpg (closed)>
[9] pry(#<Dragonfly::TempObject>)> @tempfile.size
=> 103707

after closing gives no problem either to get the size.
But now, this is the failing case, where the same process is through a spec.

From: /home/alter/.rvm/gems/ruby-2.2.5@productwatcher/gems/dragonfly-1.1.3/lib/dragonfly/temp_object.rb @ line 58 Dragonfly::TempObject#initialize:

    53:         @pathname = Pathname.new(obj.path)
    54:       elsif obj.is_a? Pathname
    55:         @pathname = obj
    56:       elsif obj.respond_to?(:tempfile)
    57:         binding.pry
 => 58:         @tempfile = obj.tempfile
    59:       elsif obj.respond_to?(:path) # e.g. Rack::Test::UploadedFile
    60:         @pathname = Pathname.new(obj.path)
    61:       else
    62:         raise ArgumentError, "#{self.class.name} must be initialized with a String, a Pathname, a File, a Tempfile, another TempObject, something that responds to .tempfile, or something that responds to .path - you gave #{obj.inspect}"
    63:       end

[1] pry(#<Dragonfly::TempObject>)> obj
=> #<ActionDispatch::Http::UploadedFile:0x0000000ca7b5c8
 @content_type="image/png",
 @headers=nil,
 @original_filename="quokka.jpg",
 @tempfile=#<File:/files/alter/workspace/productswatcher/spec/fixtures/quokka.jpg>>
[2] pry(#<Dragonfly::TempObject>)> n

From: /home/alter/.rvm/gems/ruby-2.2.5@productwatcher/gems/dragonfly-1.1.3/lib/dragonfly/temp_object.rb @ line 65 Dragonfly::TempObject#initialize:

    60:         @pathname = Pathname.new(obj.path)
    61:       else
    62:         raise ArgumentError, "#{self.class.name} must be initialized with a String, a Pathname, a File, a Tempfile, another TempObject, something that responds to .tempfile, or something that responds to .path - you gave #{obj.inspect}"
    63:       end
    64: 
 => 65:       @tempfile.close if @tempfile
    66: 
    67:       # Name
    68:       @name = if name
    69:         name
    70:       elsif obj.respond_to?(:original_filename)

[2] pry(#<Dragonfly::TempObject>)> @tempfile
=> #<File:/files/alter/workspace/productswatcher/spec/fixtures/quokka.jpg>
[3] pry(#<Dragonfly::TempObject>)> @tempfile.size
=> 4744

The file size is displayed correctly, but after this is closed, it starts to raise the exception:

[4] pry(#<Dragonfly::TempObject>)> n

From: /home/alter/.rvm/gems/ruby-2.2.5@productwatcher/gems/dragonfly-1.1.3/lib/dragonfly/temp_object.rb @ line 68 Dragonfly::TempObject#initialize:

    63:       end
    64: 
    65:       @tempfile.close if @tempfile
    66: 
    67:       # Name
 => 68:       @name = if name
    69:         name
    70:       elsif obj.respond_to?(:original_filename)
    71:         obj.original_filename
    72:       elsif @pathname
    73:         @pathname.basename.to_s

[4] pry(#<Dragonfly::TempObject>)> @tempfile
=> #<File:/files/alter/workspace/productswatcher/spec/fixtures/quokka.jpg (closed)>
[5] pry(#<Dragonfly::TempObject>)> @tempfile.size
IOError: closed stream
from (pry):5:in `size'

If I comment @tempfile.close if @tempfile in the gem, the spec works as expected, but I'm guessing this is far from being a final solution, less for production.

@markevans
Copy link
Owner

markevans commented Nov 6, 2017 •
edited
Loading

thanks for the detailed debug info. So just to clarify, this is just in tests right? (I noticed you said "I'm having this issue in production right now" - did you mean a CI server or something?)

Also I notice that you're initializing the ActionDispatch::Http::UploadedFile with a File object, not a Tempfile object - does it make a difference using a Tempfile? (as this is what is should be)

Also, to get the tests to pass it would probably be sufficient just to do (unless there's something I'm missing)

    factory :attachment do
      file Rails.root.join('spec/fixtures/quokka.jpg')
    end

@ertrzyiks
Copy link

ertrzyiks commented Aug 10, 2018 •
edited
Loading

We were using Rack::Test::UploadedFile instead of ActionDispatch::Http::UploadedFile but I believe it does not matter that much. The problem was that by the moment code reached size check, garbage collector cleaned the temporary file and we were left with a pointer to not existing file, hence closed stream error occurred.

Passing a file itself helped indeed

factory :attachment do
  file { Rails.root.join('spec/fixtures/quokka.jpg').open }
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@markevans @alagos @ertrzyiks