Published on

How to use AWS Rekognition using Ruby on Rails

Ever wanted to use a technology that can identify specific objects contained in any given image? Well look no further as AWS Rekognition fulfils that need.

Logo of AWS Rekognition

Figure 1: Logo of AWS Rekognition

AWS Rekognition is a tool that is part of the AWS Cloud Infrastructure ecosystem. It primarily serves to make it easy to add image and video analysis to any application using machine learning that requires zero knowledge of artificial intelligence.

AWS Rekognition can be used to recognizing objects, people, text and many more. One notable use case for Rekognition is to use it to identify dog breeds via a dog image.

Ok enough about AWS Rekognition, now let's learn how to implement it to a Ruby on Rails project!

Setup

To install Rails we will first need to install Ruby. This can be done using the Ruby Installer if you're using Windows. For any other operating systems please refer to this link: ruby-lang.org

You'll also need to install SQLite3 database, this can be done through the SQLite3 website.

Then you'll need to install Node.js backend JavaScript runtime environment and the Yarn package manager. You can install Node.js by visiting their website and Yarn can be installed through the Yarn website.

Finally, Rails can be installed using the gem install command, which is a part of RubyGems. To install Rails just run the following command:

gem install rails

In order to verify that everything was installed correctly, running the following command should display the version of Rails installed on your system:

rails --version

We are now finished setting up our Rails application, so on to development!⏩

Development

AWS Management Console Setup

First we need to obtain some credentials from the AWS Management Console by visiting here. For the sake of brevity that process won't be covered here so please refer to the following link specific instructions: https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html

After obtaining your AWS credentials make sure to save that .csv file in a secure location because we will need it later.

Application Development

Next open up your favorite coding IDE and create a new Rails project, here's a link on how to do so if you're unfamiliar with the process: https://guides.rubyonrails.org/getting_started.html

Once you've got a Rails project up and running, just cd into it and add the following code to the Gemfile.lock file:

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.7.4'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 5.2.5'
gem 'jquery-rails'
# Use mysql as the database for Active Record
gem 'mysql2', '>= 0.4.4', '< 0.6.0'
# Use Puma as the app server
gem 'puma', '~> 3.11'
# Use SCSS for stylesheets
gem 'sass-rails', '~> 5.0'
# Use Uglifier as compressor for JavaScript assets
gem 'uglifier', '>= 1.3.0'
gem 'rest-client'
gem 'aws-sdk-rekognition'
gem 'aws-sdk-s3'
gem 'aws-sdk', '~> 3.0', '>= 3.0.1'
gem 'paperclip'

# See https://github.com/rails/execjs#readme for more supported runtimes
# gem 'mini_racer', platforms: :ruby

# Use CoffeeScript for .coffee assets and views
gem 'coffee-rails', '~> 4.2'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.5'

gem 'therubyracer'

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.1.0', require: false
gem 'bootstrap', '~> 4.0.0'
gem 'devise'

gem 'figaro'

gem 'faye-websocket', '~> 0.10.4'

gem 'open-uri'

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
end

group :development do
  # Access an interactive console on exception pages or by calling 'console' anywhere in the code.
  gem 'web-console', '>= 3.3.0'
  gem 'listen', '>= 3.0.5', '< 3.2'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'spring'
  gem 'spring-watcher-listen', '~> 2.0.0'
end

group :test do
  # Adds support for Capybara system testing and selenium driver
  gem 'capybara', '>= 2.15'
  gem 'selenium-webdriver'
  # Easy installation and use of chromedriver to run system tests with Chrome
  gem 'chromedriver-helper'
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

Then run the following command to install all the dependencies listed in the Gemfile:

bundler install

Next in order to create an untracked file that can hold our credentials we need to run the following command to get Figaro to create the config/application.yml file:

bundle exec figaro install

Now if you navigate to the config folder we should see an application.yml file. In it, let's add the following code:

aws_client_id: <YOUR_AWS_CLIENT_ID>
aws_client_secret: <YOUR_AWS_CLIENT_SECRET>
aws_region: <YOUR_AWS_REKOGNITION_REGION>

You'll find your AWS Client Id and Client Secret in the .csv credentials file that was downloaded to your local machine. The AWS region for your AWS Rekognition project can be found on the top right of the AWS Rekognition project page. Just make the region name is one of the names displayed on the right side of this screenshot:

AWS Region names list

Figure 2: AWS Region names list

Once our config/application.yml file is populated with our AWS credentials, we can create a Home controller in our Rails application by running the following command:

rails generate controller home

After running the above command, there should be an app/controllers/home_controller.rb file created. Let's add the following code to it:


require 'net/https'
require 'uri'
require 'json'
require 'base64'
require 'rest_client'
require 'aws-sdk'

class HomeController < ApplicationController


    def index
    end

    def create
        uploaded_file = params[:image_file_data]
        salt = (Time.now.to_f * 1000).to_s
        file_name = "#{salt}-#{uploaded_file.original_filename}"
        File.open(Rails.root.join('public', 'uploads', file_name), 'wb') do |file|
            file.write(uploaded_file.read)

            credentials = Aws::Credentials.new(ENV['aws_client_id'], ENV['aws_client_secret'])
            client = Aws::Rekognition::Client.new region: ENV['aws_region'], credentials: credentials
            photo = file_name

            path = File.expand_path("public/uploads/#{photo}") # expand path relative to the current directory
            file = File.read(path)
            if(File.zero?(path))
                @result = {}
                return
            end
            attrs = {
            image: {
            bytes: file
            },
            max_labels: 10
            }

            @response = client.detect_labels(attrs)
            render "index"
        end
    end

end

Basically the create() method will extract image file data from the template first. Then, the connection to AWS will be established using the credentials stored in the /config/application.yml file. After this, the uploaded image file will be stored in the local file system's {root_folder_path}/public/uploads/{image_name} path. Finally, this locally stored file will be read by the AWS Rekognition client library to which a response will be issued and printed to the console in an organized manner thanks to the @response.labels.each do |label| loop.

Next we'll add a simple image file upload form to the Home controller's index template. So locate the app/views/home folder, create an index.html.erb file and add the following content:

<%= form_tag "/home/upload_picture", method: :post, multipart: true, class: 'mt-5' do %>
    <%= file_field_tag :image_file_data %>
    <%= submit_tag 'Upload', class: 'btn btn-primary' %>
<% end %>

<% if @response %>
    <% @response.labels.each do |label| %>
        <p>------------------</p>
        <p><b>Label:</b>      <%= label.name %></p>
        <p><b>Confidence:</b> <%= label.confidence %></p>
    <% end %>

<% end %>

This template file basically renders a simple image upload form. Then once a response from AWS Rekognition is received, the information contained within it is rendered in the for loop below the form. The each label that AWS Rekognition provides in its response contains a name and a confidence level. The name is a best guess of what the image is predominately displaying and the confidence level displays the numerical confidence of the degree to which the deep learning model is confident that the label name matches the image.

With the Home controller completed, we now have to specify some routes in the config/routes.rb file like so:

Rails.application.routes.draw do
  # For details on the DSL available within this file, see http://guides.rubyonrails.org/routing.html
  root to: "home#index"

  post '/home/upload_picture', to: 'home#create'
end

This routes.rb file will define the two routes used for this simple application. The first route is to render the home/index.html.erb template file and the second route is to send a POST request to the home#create action upon submitting the form in the home/index.html.erb page.

We are now ready to run this program so let us do so by running:

bin/rails server

Once the server is up and running navigate to localhost:3000.

Finally, when we can select an image from our local system and upload it by clicking the 'Upload' button, we can see a list of labels along with their numerical confidence level.

Upload Image form

Figure 3: Upload Image form

Conclusion

Congratulations! You now know how to implement AWS Rekognition in Ruby on Rails applications. If you need access to the source code for this application you can access it by visiting it's GitHub link.

Well that's it for this post! Thanks for following along in this article and if you have any questions or concerns please feel free to post a comment in this post and I will get back to you when I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn, subscribe to my YouTube channel.