Uploading images on Heroku--Saving them using AWS S3 and Fog

By: Lukasz Muzyka, On: 2014-08-22

This tutorial assumes you have already completed:

Users' avatars

Our images can now be loaded into our application. However, when you deploy it to Heroku:

bash


    $ git add . #add changes to git
    $ git commit -m "image upload with paperclip" #commit changes
    $ git push # push changes to bitbucket
    $ git push heroku master # push changes to heroku
    $ git heroku run rake db:migrate #update heroku database

If you try to upload the image on your heroku website now - it will work. However, the minute you restart your app (what happens also when you update it):


    $ heroku restart

You will see that all images are now gone. This is because Heroku doesn't allow file storage on it's servers. To deal with that problem, we will use external service to store images. In our example, we will use Amazon S3 service. In this tutorial we will cover:

Create an Amazon S3 account (requires credit card - but cost is super low)
Add AWS-SDK and Fog gems
Make it work on Heroku

Amazon S3 is not free but it's so cheap, that you won't even notice the bill. I have received my first bill from Amazon after 3 months and it was whooping 3 cents. Currently Amazon offers free usage of S3 for first year up to 5GB. Still, you need to use a credit card in order to register. That may suck if you don't have one. Quick tip for those in Shanghai - drop me a line via contact form and maybe I will be able to help you directly. Or try doing search on Google how to do it without a credit card, maybe by the time you read this there is a better way. If you find something useful, please share the information in the comment section.

Step 1: Create an Amazon S3 account

Head over to AWS wesbite and register if you don't have an Amazon account yet. If you do, simply login. After you're done go to your AWS Management Console and select S3.

You might have to click through some additional stuff to confirm that you understand terms etc.

Once you have your S3 account setup, open AWS management console and create two new "buckets".

Name them "rails-demo-production" and "rails-demo-dev" respectively. If the names are not available, use other names but make sure you keep using the same names in your app.

Now we need to get our security credentials to access the buckets. Click on "Security Credentials."

and click "Access Keys (Access Key ID and Secret Access Key)"

They should look somewhat like this:

Access Key: AKIAIRPKLJNSDO23425
Secret Access Key: 9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7

Make sure you keep them safe.

Step 2: Add AWS-SDK and Fog gems

Now, we will go back to our application and make all necessary changes to enable image upload to S3.

Gemfile


    gem 'aws-sdk', '~> 1.36.1'
    gem 'fog'

And bundle new gems:

bash


    $ bundle install

Step 2.2: Modify user.rb File

We need to tell our user model to use fog to upload the files to S3. We will take our original declaration:

app/models/user.rb


has_attached_file :avatar, :styles => { :medium => "300x300>", :thumb => "100x100#" },
    :default_url => "/images/:style/missing.png"

And modify it slightly:

app/models/user.rb


has_attached_file :avatar, :styles => { :medium => "300x300>", :thumb => "100x100#" },
    :default_url => "/images/:style/missing.png",
    :url  => ":s3_domain_url",
    :path => "public/avatars/:id/:style_:basename.:extension",
    :storage => :fog,
    :fog_credentials => {
        provider: 'AWS',
        aws_access_key_id: "AKIAIRPKLJNSDO23425",
        aws_secret_access_key: "9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7"
    },
    fog_directory: "rails-demo-env"

Step 2.3: Add gem Figaro

It is not the best practice to keep our sensitive variables hardcoded inside of our source code. We need to keep them somewhere securely. Since rails 4.1 we can use secrets.yml file but I find it confusing and not that useful comparing to gem Figaro. Figaro will help us to manage all secrets and update Heroku variables securely. Let's begin by adding it to our gem file:

Gemfile


    # existing code

    gem "figaro"

And bundle

bash


$ bundle install

Step 2.4: Run Installation

Now we need to run installation that will generate application.yml file for our variables:

bash


$ rails generate figaro:install

    create  config/application.yml
    append  .gitignore

Step 2.5: Replace AWS variables

This file will not be tracked by git as figaro automatically added it to our .gitingore file. Let's open it and add our variables there:

config/aplication.yml


    # Add application configuration variables here, as shown below.
    #
    # PUSHER_APP_ID: "2954"
    # PUSHER_KEY: 7381a978f7dd7f9a1117
    # PUSHER_SECRET: abdc3b896a0ffb85d373
    # STRIPE_API_KEY: EdAvEPVEC3LuaTg5Q3z6WbDVqZlcBQ8Z
    # STRIPE_PUBLIC_KEY: pk_BRgD57O8fHja9HxduJUszhef6jCyS

We can now replace this with our variables for AWS:

config/aplication.yml


SECRET_KEY_BASE: 60949ad338a3c9a578350ae917bb74372a40a45e5d93c80addaa13d4bf57a62352c644e861c24ce8a5c61fc193f7dbbbf39aadff80e17285f5ac5bf6f0db8200
AWS_ACCES_KEY_ID: AKIAIRPKLJNSDO23425
AWS_SECRET_ACCESS_KEY: 9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7

development:
  FOG_DIRECTORY: rails-demo-dev
production:
  FOG_DIRECTORY: rails-demo-production

Step 2.6: Replace Hardcoded Values

Now we can replace our hardcoded values with references to "environment variables handled by figaro"

app/models/user.rb


has_attached_file :avatar, :styles => { :medium => "300x300>", :thumb => "100x100#" },
    :default_url => "/images/:style/missing.png",
    :url  => ":s3_domain_url",
    :path => "public/avatars/:id/:style_:basename.:extension",
    :storage => :fog,
    :fog_credentials => {
        provider: 'AWS',
        aws_access_key_id: ENV["AWS_ACCESS_KEY_ID"],
        aws_secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"]
    },
    fog_directory: ENV["FOG_DIRECTORY"]

Step 2.7: Add password and login to Application

We can also move here our login and password we have used before to send emails form our environments configuration files:

config/environments/development.rb


config.action_mailer.smtp_settings = {
    address: "smtp.gmail.com",
    port: 587,
    authentication: "plain",
    enable_starttls_auto: true,
    user_name: ENV["EMAIL_LOGIN"],
    password: ENV["EMAIL_PASSWORD"]
}

config/environments/production.rb


config.action_mailer.smtp_settings = {
    address: "smtp.gmail.com",
    port: 587,
    authentication: "plain",
    enable_starttls_auto: true,
    user_name: ENV["EMAIL_LOGIN"],
    password: ENV["EMAIL_PASSWORD"]
}

And add password and login to our application.yml file:

config/application.yml


SECRET_KEY_BASE: 60949ad338a3c9a578350ae917bb74372a40a45e5d93c80addaa13d4bf57a62352c644e861c24ce8a5c61fc193f7dbbbf39aadff80e17285f5ac5bf6f0db8200
AWS_ACCES_KEY_ID: AKIAIRPKLJNSDO23425
AWS_SECRET_ACCESS_KEY: 9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7
EMAIL_LOGIN: email@gmail.com
EMAIL_PASSWORD: passwordhere

development:
  FOG_DIRECTORY: rails-demo-dev
production:
  FOG_DIRECTORY: rails-demo-production

And update your secrets.yml file:

config/secrets.yml


development:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

test:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

If you're running you application - this is the time to restart it. If you're not running it yet, go ahead and start it.

You can try it out now. Your application will start uploading images to amazon S3. Depending on where you live, this might be very slow process. Don't worry though, once we deploy our app to Heroku it will become a lot faster.

Step 3: Make it work on Heroku

We need a way to provide our secret codes to Heroku without sharing them inside code. This can be pretty cumbersome, but with Figaro setting up Heroku is a breeze. Just run this rake task inside your application:

bash


$ rake figaro:heroku

In case this fails - timeouts/crashes etc (and in China everything is possible), you can always submit the variables manually:

bash


    $ heroku config:set SECRET_KEY_BASE='1fe51548304cc75187d202dff0bd17a16999b815b1c326382afd2bf8a4323493f04a2a7c63485c022b724f05911170a65ad72ab30be87f40de8be937eaa8f6e2'
    $ heroku config:set AWS_ACCESS_KEY='AKIAIRPKLJNSDO23425'
    $ heroku config:set AWS_SECRET_ACCESS_KEY='9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7'
    $ heroku config:set FOG_DIRECTORY='rails-demo-production'
    $ heroku config:set EMAIL_LOGIN='you@gmail.com'
    $ heroku config:set EMAIL_PASSWORD='yourpassword'

Make sure you use the right values.

Step 3.2: Final Adjustment in production.rb file

We're almost ready to commit and push to remote repository and the server. But first, lets make some final adjustments. By default our production will not allow to access public directory from browser. To change that behavior (so we get our placeholder images working) we need to change one line in config/environments/production.rb

config/environments/production.rb


config.serve_static_assets = true

We're ready

bash


    $ git add .
    $ git commit -m "AWS and Fog added for user avatars"
    $ git push
    $ git push heroku master

If you login you will be able to mark this tutorial as finished to track your progress

Comments

On: 2014-08-28 Brunitob wrote:

I have a questios, where do you get:

SECRET_KEY_BASE: 60949ad338a3c9a578350ae917bb74372a40a45e5d93c80addaa13d4bf57a62352c644e861c24ce8a5c61fc193f7dbbbf39aadff80e17285f5ac5bf6f0db8200

On: 2015-02-14 Andrew Hamilton wrote:

run ' $: rake secret ' in your terminal to generate a key. ( http://edgeguides.rubyonrails.org/41release_notes.html#config-secrets-yml)

On: 2015-02-12 Anchieta Junior wrote:

Just in case someone has a problem with that, the new comand to run Figaro Gem installation is just "figaro install" and it's no longer "rails generate figaro:install".

On: 2015-02-14 Andrew Hamilton wrote:

Also worth noting that the rake task for Figaro setting up on Heroku is now ' $: figaro heroku:set -e production ' instead of ' $: rake figaro:heroku '. ( https://github.com/laserlemon/figaro#heroku-configuration )

On: 2015-05-13 Adam wrote:
i'm having a problem uploading my avatar to S3. i get this error : ArgumentError in Devise::RegistrationsController#update Missing required arguments: awsaccesskey_id

Extracted source (around line #244): unless missing.empty?
```
      raise ArgumentError, "Missing required arguments: #{missing.join(", ")}"
    end
    unless recognizes.empty?
```
i followed every step in the tutorial and i updated my AWSACCESKEYID and AWSSECRETACCESSKEY.

i think it's a fog gem problem. can someone help me please, i'm stuck ! thank you.

On: 2015-08-19 clement wrote:

I had the same problem, and I dont really understand why, but wrapping the ENV["AWSACCESSKEYID"] in strings (replacing by : "#{ENV["AWSACCESSKEYID"]}") did it.

Comment

You can login to comment