Uploading images on Heroku--Saving them using AWS S3 and Fog
By: Lukasz Muzyka, On:
This tutorial assumes you have already completed:
Our images can now be loaded into our application. However, when you deploy it to Heroku:
bash
$ git add . #add changes to git
$ git commit -m "image upload with paperclip" #commit changes
$ git push # push changes to bitbucket
$ git push heroku master # push changes to heroku
$ git heroku run rake db:migrate #update heroku database
If you try to upload the image on your heroku website now - it will work. However, the minute you restart your app (what happens also when you update it):
$ heroku restart
You will see that all images are now gone. This is because Heroku doesn't allow file storage on it's servers. To deal with that problem, we will use external service to store images. In our example, we will use Amazon S3 service. In this tutorial we will cover:
- Create an Amazon S3 account (requires credit card - but cost is super low)
- Add AWS-SDK and Fog gems
- Make it work on Heroku
Amazon S3 is not free but it's so cheap, that you won't even notice the bill. I have received my first bill from Amazon after 3 months and it was whooping 3 cents. Currently Amazon offers free usage of S3 for first year up to 5GB. Still, you need to use a credit card in order to register. That may suck if you don't have one. Quick tip for those in Shanghai - drop me a line via contact form and maybe I will be able to help you directly. Or try doing search on Google how to do it without a credit card, maybe by the time you read this there is a better way. If you find something useful, please share the information in the comment section.
Step 1: Create an Amazon S3 account
Head over to AWS wesbite and register if you don't have an Amazon account yet. If you do, simply login. After you're done go to your AWS Management Console and select S3.
You might have to click through some additional stuff to confirm that you understand terms etc.
Once you have your S3 account setup, open AWS management console and create two new "buckets".
Name them "rails-demo-production" and "rails-demo-dev" respectively. If the names are not available, use other names but make sure you keep using the same names in your app.
Now we need to get our security credentials to access the buckets. Click on "Security Credentials."
and click "Access Keys (Access Key ID and Secret Access Key)"
They should look somewhat like this:
- Access Key: AKIAIRPKLJNSDO23425
- Secret Access Key: 9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7
Make sure you keep them safe.
Step 2: Add AWS-SDK and Fog gems
Now, we will go back to our application and make all necessary changes to enable image upload to S3.
Gemfile
gem 'aws-sdk', '~> 1.36.1'
gem 'fog'
And bundle new gems:
bash
$ bundle install
Step 2.2: Modify user.rb File
We need to tell our user model to use fog to upload the files to S3. We will take our original declaration:
app/models/user.rb
has_attached_file :avatar, :styles => { :medium => "300x300>", :thumb => "100x100#" },
:default_url => "/images/:style/missing.png"
And modify it slightly:
app/models/user.rb
has_attached_file :avatar, :styles => { :medium => "300x300>", :thumb => "100x100#" },
:default_url => "/images/:style/missing.png",
:url => ":s3_domain_url",
:path => "public/avatars/:id/:style_:basename.:extension",
:storage => :fog,
:fog_credentials => {
provider: 'AWS',
aws_access_key_id: "AKIAIRPKLJNSDO23425",
aws_secret_access_key: "9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7"
},
fog_directory: "rails-demo-env"
Step 2.3: Add gem Figaro
It is not the best practice to keep our sensitive variables hardcoded inside of our source code. We need to keep them somewhere securely. Since rails 4.1 we can use secrets.yml file but I find it confusing and not that useful comparing to gem Figaro. Figaro will help us to manage all secrets and update Heroku variables securely. Let's begin by adding it to our gem file:
Gemfile
# existing code
gem "figaro"
And bundle
bash
$ bundle install
Step 2.4: Run Installation
Now we need to run installation that will generate application.yml
file for our variables:
bash
$ rails generate figaro:install
create config/application.yml
append .gitignore
Step 2.5: Replace AWS variables
This file will not be tracked by git as figaro automatically added it to our .gitingore file.
Let's open it and add our variables there:
config/aplication.yml
# Add application configuration variables here, as shown below.
#
# PUSHER_APP_ID: "2954"
# PUSHER_KEY: 7381a978f7dd7f9a1117
# PUSHER_SECRET: abdc3b896a0ffb85d373
# STRIPE_API_KEY: EdAvEPVEC3LuaTg5Q3z6WbDVqZlcBQ8Z
# STRIPE_PUBLIC_KEY: pk_BRgD57O8fHja9HxduJUszhef6jCyS
We can now replace this with our variables for AWS:
config/aplication.yml
SECRET_KEY_BASE: 60949ad338a3c9a578350ae917bb74372a40a45e5d93c80addaa13d4bf57a62352c644e861c24ce8a5c61fc193f7dbbbf39aadff80e17285f5ac5bf6f0db8200
AWS_ACCES_KEY_ID: AKIAIRPKLJNSDO23425
AWS_SECRET_ACCESS_KEY: 9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7
development:
FOG_DIRECTORY: rails-demo-dev
production:
FOG_DIRECTORY: rails-demo-production
Step 2.6: Replace Hardcoded Values
Now we can replace our hardcoded values with references to "environment variables handled by figaro"
app/models/user.rb
has_attached_file :avatar, :styles => { :medium => "300x300>", :thumb => "100x100#" },
:default_url => "/images/:style/missing.png",
:url => ":s3_domain_url",
:path => "public/avatars/:id/:style_:basename.:extension",
:storage => :fog,
:fog_credentials => {
provider: 'AWS',
aws_access_key_id: ENV["AWS_ACCESS_KEY_ID"],
aws_secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"]
},
fog_directory: ENV["FOG_DIRECTORY"]
Step 2.7: Add password and login to Application
We can also move here our login and password we have used before to send emails form our environments configuration files:
config/environments/development.rb
config.action_mailer.smtp_settings = {
address: "smtp.gmail.com",
port: 587,
authentication: "plain",
enable_starttls_auto: true,
user_name: ENV["EMAIL_LOGIN"],
password: ENV["EMAIL_PASSWORD"]
}
config/environments/production.rb
config.action_mailer.smtp_settings = {
address: "smtp.gmail.com",
port: 587,
authentication: "plain",
enable_starttls_auto: true,
user_name: ENV["EMAIL_LOGIN"],
password: ENV["EMAIL_PASSWORD"]
}
And add password and login to our application.yml
file:
config/application.yml
SECRET_KEY_BASE: 60949ad338a3c9a578350ae917bb74372a40a45e5d93c80addaa13d4bf57a62352c644e861c24ce8a5c61fc193f7dbbbf39aadff80e17285f5ac5bf6f0db8200
AWS_ACCES_KEY_ID: AKIAIRPKLJNSDO23425
AWS_SECRET_ACCESS_KEY: 9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7
EMAIL_LOGIN: email@gmail.com
EMAIL_PASSWORD: passwordhere
development:
FOG_DIRECTORY: rails-demo-dev
production:
FOG_DIRECTORY: rails-demo-production
And update your secrets.yml
file:
config/secrets.yml
development:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
test:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
If you're running you application - this is the time to restart it. If you're not running it yet, go ahead and start it.
You can try it out now. Your application will start uploading images to amazon S3. Depending on where you live, this might be very slow process. Don't worry though, once we deploy our app to Heroku it will become a lot faster.
Step 3: Make it work on Heroku
We need a way to provide our secret codes to Heroku without sharing them inside code. This can be pretty cumbersome, but with Figaro setting up Heroku is a breeze. Just run this rake task inside your application:
bash
$ rake figaro:heroku
In case this fails - timeouts/crashes etc (and in China everything is possible), you can always submit the variables manually:
bash
$ heroku config:set SECRET_KEY_BASE='1fe51548304cc75187d202dff0bd17a16999b815b1c326382afd2bf8a4323493f04a2a7c63485c022b724f05911170a65ad72ab30be87f40de8be937eaa8f6e2'
$ heroku config:set AWS_ACCESS_KEY='AKIAIRPKLJNSDO23425'
$ heroku config:set AWS_SECRET_ACCESS_KEY='9LXV/GWagu/CJ4DoMLKSmdasdaZeUyuax/Ei92hL7'
$ heroku config:set FOG_DIRECTORY='rails-demo-production'
$ heroku config:set EMAIL_LOGIN='you@gmail.com'
$ heroku config:set EMAIL_PASSWORD='yourpassword'
Make sure you use the right values.
Step 3.2: Final Adjustment in production.rb file
We're almost ready to commit and push to remote repository and the server. But first, lets make some final adjustments. By default our production will not allow to access public directory from browser. To change that behavior (so we get our placeholder images working) we need to change one line in config/environments/production.rb
config/environments/production.rb
config.serve_static_assets = true
We're ready
bash
$ git add .
$ git commit -m "AWS and Fog added for user avatars"
$ git push
$ git push heroku master
Comments
Comment
You can login to comment
On: Brunitob wrote:
I have a questios, where do you get:
On: Anchieta Junior wrote:
Just in case someone has a problem with that, the new comand to run Figaro Gem installation is just "figaro install" and it's no longer "rails generate figaro:install".
On: Adam wrote:
i'm having a problem uploading my avatar to S3. i get this error : ArgumentError in Devise::RegistrationsController#update Missing required arguments: awsaccesskey_id
Extracted source (around line #244): unless missing.empty?
i followed every step in the tutorial and i updated my AWSACCESKEYID and AWSSECRETACCESSKEY.
i think it's a fog gem problem. can someone help me please, i'm stuck ! thank you.