Effective Assets

Upload images and files directly to AWS S3 with a custom form input, then seamlessly organize and attach them to any ActiveRecord object.

A Rails Engine full solution for managing assets (images, files, videos, etc).

Attach one or more assets to any model with validations.

Upload direct to Amazon S3 implementation based on jQuery-File-Upload and image processing on a background process with CarrierWave and DelayedJob

Rails FormBuilder, Formtastic and SimpleForm inputs for displaying, managing, and uploading assets direct to S3.

Works with AWS public-read and authenticated-read for easy secured downloads.

Includes integration with ActiveAdmin

Rails 3.2.x and Rails4 support

Getting Started

Add to your Gemfile:

gem 'haml-rails'       # or try using gem 'hamlit-rails'
gem 'effective_assets'

Run the bundle command to install it:

bundle install

Then run the generator:

rails generate effective_assets:install

The generator will install an initializer which describes all configuration options and creates two database migrations, one for EffectiveAssets the other for DelayedJob.

If you want to tweak the table name (to use something other than the default assets and attachments), manually adjust both the configuration file and the migration now.

Then migrate the database:

rake db:migrate

If you intend to use the form helper method to display and upload assets, require the javascript in your application.js:

//= require effective_assets

and the stylesheet in your application.css:

*= require effective_assets

If you intend to use ActiveAdmin (optional):

Add to your ActiveAdmin.js file:

//= require effective_assets

And to your ActiveAdmin stylesheet

body.active_admin {
}
@import "active_admin/effective_assets";

If ActiveAdmin is installed, there will automatically be an 'Effective Assets' page.

Create/Configure an S3 Bucket

You will need an AWS IAM user with sufficient priviledges and a properly configured S3 bucket to use with effective_assets

Log into AWS Console

• Visit http://aws.amazon.com/console/
• Click 'Sign In to the Console' from the top-right and sign in with your AWS account.

Create an S3 Bucket

• Click Services -> S3 from the top-left menu
• Click Create Bucket
• Give the Bucket a name, and select the US East (N. Virgina) region
• Click Next, Next, Next
• Don't configure permissions yet, we still have to create a user
• Click Next

Configure CORS Permissions

• From the S3 All Buckets Screen (as above)

• Click the S3 bucket we just created

• Click the Permissions tab

• Click CORS configuration and enter the following:

<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
  </CORSRule>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
  </CORSRule>
</CORSConfiguration>

• Click Save

The Bucket is now set up and ready to accept uploads, but we still need a user that has permission to access S3

Create an IAM User and record its AWS Access Keys

• After logging in to your AWS console

• Click Services -> IAM from the top-left

• Select Users from the left-side menu

• Click Add user

• Give it a user name

• Check Yes Programmatic access, No AWS management console access

• Click Next: Permissions

• Click Create group

• Give it a name like 's3-full-access'

• Sroll down and select 'AmazonS3FullAccess'

• Click Create group

• Click Next: Review

• Click Create user

• Record the Access key ID and Secret access key. These are the two values required by the effective_assets.rb initializer

• Click Close

This user is now set up and ready to access the S3 Bucket previously created

Add S3 Access Keys

Add the name of your S3 Bucket, Access Key and Secret Access Key to the config/initializers/effective_assets.rb file.

config.aws_bucket = 'my-bucket'
config.aws_access_key_id = 'ABCDEFGHIJKLMNOP'
config.aws_secret_access_key = 'xmowueroewairo74pacja1/werjow'

You should now be able to upload to this bucket.

Usage

Model

Use the acts_as_asset_box mixin to define a set of 'boxes' all your assets are grouped into. A box is just a category, which can have any name.

If the box is declared using a singular word, such as :photo it will be set up as a has_one asset. When defined as a plural, such as :photos it implies a has_many assets.

The following will create 4 separate boxes of assets:

class User
  acts_as_asset_box :avatar, :photos, :videos, :mp3s
end

Calling user.avatar will return a single Effective::Asset. Calling user.photos will return an array of Effective::Assets.

Then to get the URL of an asset:

asset = user.avatar
  => an Effective::Asset

asset.url
  => "http://aws_bucket.s3.amazonaws.com/assets/1/my_avatar.png"

asset.url(:thumb)   # See image versions (below)
  => "http://aws_bucket.s3.amazonaws.com/assets/1/thumb_my_avatar.png"

asset.authenticated_url   # See image versions (below)
  => "http://aws_bucket.s3.amazonaws.com/assets/1/thumb_my_avatar.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-...""

asset.authenticated_url(:thumb, expire_in: 10.minutes)
  => "http://aws_bucket.s3.amazonaws.com/assets/1/thumb_my_avatar.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-...""

user.photos
  => [Effective::Asset<1>, Effective::Asset<2>] # An array of Effective::Asset objects

Validations

You can validate the presence and length of the assets with a simple syntax:

class User
  acts_as_asset_box :avatar => true, :photos => false, :videos => 2, :mp3s => 5..10
end

true means presence, false means no validations applied.

or a more involved syntax:

class User
  acts_as_asset_box avatar: [presence: true], photos: false, videos: [length: 2], mp3s: [length: 5..10]
end

The user in this example is only valid if exists an avatar, 2 videos, and 5..10 mp3s.

Procs are not supported.

Form Input

There is a standard rails form input:

= form_for @user do |f|
  = f.asset_box_input :pictures

A SimpleForm input:

= simple_form_for @user do |f|
  = f.input :pictures, :as => :asset_box

and a Formtastic input:

= semantic_form_for @user do |f|
  = f.input :pictures, :as => :asset_box

The :as => :asset_box will work interchangeably with SimpleForm or Formtastic, as long as only one of these gems is present in your application.

If you use both SimpleForm and Formtastic, you will need to call asset_box_input differently:

= simple_form_for @user do |f|
  = f.input :pictures, :as => :asset_box_simple_form

= semantic_form_for @user do |f|
  = f.input :pictures, :as => :asset_box_formtastic

Uploading & Attaching

Use the custom form input for uploading (direct to S3) and attaching assets to the pictures box.

= f.input :pictures, :as => :asset_box, :uploader => true

= f.input :pictures, :as => :asset_box, :limit => 2, :file_types => [:jpg, :gif, :png]

= f.input :pictures, :as => :asset_box, :dialog => true, :dialog_url => '/admin/effective_assets' # Use the attach dialog

= f.input :pictures, :as => :asset_box, :click_submit => true # Auto click submit button after file uploads

You may also upload secure (AWS: 'authenticated-read') assets with the same uploader:

= f.input :pictures, :as => :asset_box, :private => true

There is also a mechanism for collecting additional information from the upload form which will be set in the asset.extra field.

= semantic_form_for Product.new do |f|
  = f.input :photos, :as => :asset_box
  = f.semantic_fields_for :photos do |pf|
    = pf.input :field1, :as => :string
    = pf.input :field2, :as => :boolean

Here the semantic_fields_for will create some inputs with name

product[photos][field1]
product[photos][field2]

Any additional field like this will be passed to the Asset and populate the extra Hash attribute

Note: Passing :limit => 2 will have no effect on a singular asset_box, which by definition has a limit of 1.

We use the jQuery-File-Upload gem for direct-to-s3 uploading. The process is as follows:

• User sees the form and clicks Browse. Selects 1 or more files, then clicks Start Uploading.
• The server makes a post to the S3UploadsController#create action to initialize an asset, and get a unique ID
• The file is uploaded directly to its 'final' resting place on S3 via Javascript uploader at assets/:id/:filename
• A PUT is then made back to the S3UploadsController#update which updates the Effective::Asset object and sets up a task in DelayedJob to process the asset (for image resizing)
• An Effective::Attachment is created, which joins the Effective::Asset to the parent Object (User in our example) in the appropriate position.
• The DelayedJob task should be running and will handle any image resizing as defined by the AssetUploader.
• The asset will appear in the form, and the user may click&drag the asset around to set the position.

Strong Parameters

Make your controller aware of the acts_as_asset_box passed parameters:

def permitted_params
  params.require(:base_object).permit(EffectiveAssets.permitted_params)
end

The permitted parameters are:

:attachments_attributes => [:id, :asset_id, :attachable_type, :attachable_id, :position, :box, :_destroy]

Amazon S3 Public / Private

When an asset is uploaded, it is created with an aws_acl of either public-read or authenticated-read.

When in authenticated-read mode, calling @asset.url will return a URL that is valid for just 60 minutes.

This privacy level can be configured in the following 3 ways:

1. The app wide default is set in config/initializers/effective_assets.rb. All Effective::Asset objects will be created with this ACL unless specified below.

2. When you call acts_as_asset_box on a model, the :private or :public keys will define the behavior for just this asset box.

acts_as_asset_box :avatar => :private
acts_as_asset_box :avatar => :public

or, with validations:

acts_as_asset_box :avatar => [presence: true, private: true]
acts_as_asset_box :avatar => [presence: true, public: true]

All assets uploaded into this box will be created with this ACL unless overridden below.

3. Set the ACL on the form upload field

= f.input :avatar, :as => :asset_box, :private => true
= f.input :avatar, :as => :asset_box, :public => true

Has final say over the privacy setting when uploaded from this form.

Image Processing and Resizing

CarrierWave is used by this gem to perform image versioning.

All image processing is run asynchronously.

If there is a valid ActiveJob queue_adapter configured, it will be used. Otherwise we rely on the sucker_punch gem.

See the installer created at app/uploaders/asset_uploader.rb to configure image versions.

Use the process :record_info => :thumb directive to store image version dimensions and file sizes.

When this uploader file is changed, you must reprocess any existing Effective::Asset objects to recreate all image versions.

This one-liner downloads the original file from AWS S3, creates the image versions locally using imagemagick, then uploads each version to its final resting place back on AWS S3.

Effective::Asset.find(123).reprocess!
=> true

This can be done in batch using the built in rake script (see below).

Helpers

You can always get the URL directly

current_user.avatar.url(:thumb)

To display the asset as a link with an image (if its an image, or a mime-type appropriate icon if its not an image):

# Asset is the @user.fav_icon
# version is anything you set up in your uploaders/asset_uploader.rb as versions.  :thumb
# Options are passed through to a call to rails image_tag helper
effective_asset_image_tag(asset, version = nil, options = {})

To display the asset as a link with no image:

# Options are passed through to rails link_to helper
effective_asset_link_to(asset, version = nil, options = {})

Authorization

All authorization checks are handled via the config.authorization_method found in the config/initializers/ file.

It is intended for flow through to CanCan or Pundit, but that is not required.

This method is called by all controller actions with the appropriate action and resource

Action will be one of [:index, :show, :new, :create, :edit, :update, :destroy]

Resource will the appropriate Effective::Something ActiveRecord object or class

The authorization method is defined in the initializer file:

# As a Proc (with CanCan)
config.authorization_method = Proc.new { |controller, action, resource| authorize!(action, resource) }

# As a Custom Method
config.authorization_method = :my_authorization_method

and then in your application_controller.rb:

def my_authorization_method(action, resource)
  current_user.is?(:admin) || EffectivePunditPolicy.new(current_user, resource).send('#{action}?')
end

or disabled entirely:

config.authorization_method = false

If the method or proc returns false (user is not authorized) an Effective::AccessDenied exception will be raised

You can rescue from this exception by adding the following to your application_controller.rb:

rescue_from Effective::AccessDenied do |exception|
  respond_to do |format|
    format.html { render 'static_pages/access_denied', :status => 403 }
    format.any { render :text => 'Access Denied', :status => 403 }
  end
end

Permissions

To allow user uploads, using Cancan:

can [:create, :update, :destroy], Effective::Asset, :user_id => user.id

To allow a user to see the admin / active_admin area:

can :admin, :effective_assets

Rake Tasks

Use the following rake tasks to aid in batch processing a large number of (generally image) files.

Reprocess

If the app/uploaders/asset_uploader.rb file is changed, run the following rake task to reprocess all Effective::Asset objects and thereby recreate all image versions

rake effective_assets:reprocess           # All assets
rake effective_assets:reprocess[200]      # reprocess #200 and up
rake effective_assets:reprocess[1,200]    # reprocess #1..#200

This command enqueues a .reprocess! job for each Effective::Asset on the DelayedJob queue.

If a DelayedJob worker process is already running, the reprocessing will begin immediately, otherwise start one with

rake jobs:work

Check

Checks every Effective::Asset and all its versions for a working URL (200 http status code).

Any non-200 http responses are logged as an error.

This is a sanity-check task, that makes sure every url for every asset and version is going to work.

This is just single-threaded one process.

If you need to check a large number of urls, use multiple rake tasks and pass in ID ranges. Sorry.

rake effective_assets:check         # check that every version of every Effective::Asset is a valid http 200 OK url
rake effective_assets:check[200]    # check #200 and up
rake effective_assets:check[1,200]  # check #1..#200
rake effective:assets:check[1,200,:thumb]   # check #1..#200 only :thumb versions

License

MIT License. Copyright Code and Effect Inc.

Credits

This gem relies on:

CarrierWave (https://github.com/carrierwaveuploader/carrierwave)

sucker_punch (https://github.com/brandonhilkert/sucker_punch)

jQuery-File-Upload (https://github.com/blueimp/jQuery-File-Upload)

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request