Published 2023-04-14.
Last modified 2023-04-27.
Time to read: 8 minutes.
ruby collection.
This article continues my search for a database framework to use with Ruby Sinatra.
Although this article is not about Ruby on Rails, it is mentioned many times. I have used the Rails abbreviation for Ruby on Rails for convenience.
Rails and Ruby Sinatra share many architectural features and conventions.
Outline
This article is structured as follows:
- Briefly introduces Active Record.
-
Shows how Active Record integrates with
rake, the command-line interface to the Ruby build system. -
Introduces a wrapper project for Ruby Sinatra called
sinatra-activerecord. - Demonstrates typical Active Record usage in a minimal Ruby project, including integrating build system tasks and database migrations.
- Turns the Ruby project into a Sinatra webapp.
The code developed throughout this article is provided in a GitHub project called
min-sin.
This article stops once min-sin is fully explained.
The resulting webapp serves web pages, but it does not include CRUD operations.
You are welcome to use min-sin as the template for your next Sinatra-ActiveRecord project.
Rake, the Ruby Build System
The name rake is a contraction of Ruby make.
Make
is the original build system for UNIX systems.
Although make can work with any computer language,
rake is focused on building Ruby projects.
Rake has some novel features – primarily the command-line interface for build tasks.
Built-in rake tasks
include file operations, publishing sites via FTP/SSH, and running tests.
Rake looks for project-specific
task definitions
in files called Rakefile or rakefile.
Rails also allows rake tasks to be defined in files called lib/tasks/whatever.rake
This is the help message for rake.
$ rake -h rake [-f rakefile] {options} targets...
Options are ... --backtrace=[OUT] Enable full backtrace. OUT can be stderr (default) or stdout. --comments Show commented tasks only --job-stats [LEVEL] Display job statistics. LEVEL=history displays a complete job list --rules Trace the rules resolution. --suppress-backtrace PATTERN Suppress backtrace lines matching regexp PATTERN. Ignored if --trace is on. -A, --all Show all tasks, even uncommented ones (in combination with -T or -D) -B, --build-all Build all prerequisites, including those which are up-to-date. -C, --directory [DIRECTORY] Change to DIRECTORY before doing anything. -D, --describe [PATTERN] Describe the tasks (matching optional PATTERN), then exit. -e, --execute CODE Execute some Ruby code and exit. -E, --execute-continue CODE Execute some Ruby code, then continue with normal task processing. -f, --rakefile [FILENAME] Use FILENAME as the rakefile to search for. -G, --no-system, --nosystem Use standard project Rakefile search paths, ignore system wide rakefiles. -g, --system Using system wide (global) rakefiles (usually '~/.rake/*.rake'). -I, --libdir LIBDIR Include LIBDIR in the search path for required modules. -j, --jobs [NUMBER] Specifies the maximum number of tasks to execute in parallel. (default is number of CPU cores + 4) -m, --multitask Treat all tasks as multitasks. -n, --dry-run Do a dry run without executing actions. -N, --no-search, --nosearch Do not search parent directories for the Rakefile. -P, --prereqs Display the tasks and dependencies, then exit. -p, --execute-print CODE Execute some Ruby code, print the result, then exit. -q, --quiet Do not log messages to standard output. -r, --require MODULE Require MODULE before executing rakefile. -R, --rakelibdir RAKELIBDIR, Auto-import any .rake files in RAKELIBDIR. (default is 'rakelib') --rakelib -s, --silent Like --quiet, but also suppresses the 'in directory' announcement. -t, --trace=[OUT] Turn on invoke/execute tracing, enable full backtrace. OUT can be stderr (default) or stdout. -T, --tasks [PATTERN] Display the tasks (matching optional PATTERN) with descriptions, then exit. -AT combination displays all of tasks contained no description. -v, --verbose Log message to standard output. -V, --version Display the program version. -W, --where [PATTERN] Describe the tasks (matching optional PATTERN), then exit. -X, --no-deprecation-warnings Disable the deprecation warnings. -h, -H, --help Display this help message.
Active Record provides additional
rake tasks.
These tasks allow you to continuously refine your application's database and associated code.
Active Record rake task definitions include tasks that support data migrations.
Adding Active Record Rake Tasks to Ruby Projects
This article will demonstrate and explain how to include Active Record rake tasks into a Ruby project.
Later, this article turns the Ruby project into a Sinatra webapp.
Before rake tasks can be used, however,
additional dependencies must be installed and configured.
Read on, and I will reveal the simple magic behind the curtain!
Gems for Active Record Rake Tasks
Only two files are required to define a Ruby project that demonstrates this:
Gemfile and Rakefile.
Provided that the following 2 gems have been included in your project, like this:
source 'https://rubygems.org' gem 'sinatra-activerecord' gem 'rake', require: false
... then all you have to do is write a one-line Rakefile
to include Active Record tasks into your Ruby project:
require 'sinatra/activerecord/rake'
With just those two files in place,
a new Ruby project is defined that contains Active Record rake tasks, through its
transitive dependency, Active Record.
Let’s install our minimal project’s dependencies so we can examine the Active Record rake
tasks provided by sinatra-activerecord:
$ bundle Fetching gem metadata from https://rubygems.org/.......... Resolving dependencies... Using concurrent-ruby 1.2.2 Using minitest 5.18.0 Using i18n 1.12.0 Using bundler 2.4.6 Fetching rack 2.2.7 Using tzinfo 2.0.6 Using ruby2_keywords 0.0.5 Using tilt 2.1.0 Using activesupport 7.0.4.3 Using mustermann 3.0.0 Using activemodel 7.0.4.3 Using activerecord 7.0.4.3 Installing rack 2.2.7 Using rack-protection 3.0.6 Using sinatra 3.0.6 Using sinatra-activerecord 2.0.26 Bundle complete! 2 Gemfile dependencies, 15 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed.
Bundle Exec
We will use the bundle exec
command a lot for the remainder of this article.
Here is the help information:
$ bundle exec -h BUNDLE-EXEC(1) BUNDLE-EXEC(1)
NAME bundle-exec - Execute a command in the context of the bundle
SYNOPSIS bundle exec [--keep-file-descriptors] command
DESCRIPTION This command executes the command, making all gems specified in the [Gemfile(5)][Gemfile(5)] available to require in Ruby programs.
Essentially, if you would normally have run something like rspec spec/my_spec.rb, and you want to use the gems specified in the [Gemfile(5)][Gemfile(5)] and installed via bundle in‐ stall(1) bundle-install.1.html, you should run bundle exec rspec spec/my_spec.rb.
Note that bundle exec does not require that an executable is available on your shell´s $PATH.
OPTIONS --keep-file-descriptors Exec in Ruby 2.0 began discarding non-standard file descriptors. When this flag is passed, exec will re‐ vert to the 1.9 behaviour of passing all file descrip‐ tors to the new process.
BUNDLE INSTALL --BINSTUBS If you use the --binstubs flag in bundle install(1) bun‐ dle-install.1.html, Bundler will automatically create a di‐ rectory (which defaults to app_root/bin) containing all of the executables available from gems in the bundle.
After using --binstubs, bin/rspec spec/my_spec.rb is identi‐ cal to bundle exec rspec spec/my_spec.rb.
ENVIRONMENT MODIFICATIONS bundle exec makes a number of changes to the shell environ‐ ment, then executes the command you specify in full.
• make sure that it´s still possible to shell out to bundle from inside a command invoked by bundle exec (using $BUN‐ DLE_BIN_PATH)
• put the directory containing executables (like rails, rspec, rackup) for your bundle on $PATH
• make sure that if bundler is invoked in the subshell, it uses the same Gemfile (by setting BUNDLE_GEMFILE)
• add -rbundler/setup to $RUBYOPT, which makes sure that Ruby programs invoked in the subshell can see the gems in the bundle
It also modifies Rubygems:
• disallow loading additional gems not in the bundle
• modify the gem method to be a no-op if a gem matching the requirements is in the bundle, and to raise a Gem::Load‐ Error if it´s not
• Define Gem.refresh to be a no-op, since the source index is always frozen when using bundler, and to prevent gems from the system leaking into the environment
• Override Gem.bin_path to use the gems in the bundle, mak‐ ing system executables work
• Add all gems in the bundle into Gem.loaded_specs
Finally, bundle exec also implicitly modifies Gemfile.lock if the lockfile and the Gemfile do not match. Bundler needs the Gemfile to determine things such as a gem´s groups, autore‐ quire, and platforms, etc., and that information isn´t stored in the lockfile. The Gemfile and lockfile must be synced in order to bundle exec successfully, so bundle exec updates the lockfile beforehand.
Loading By default, when attempting to bundle exec to a file with a ruby shebang, Bundler will Kernel.load that file instead of using Kernel.exec. For the vast majority of cases, this is a performance improvement. In a rare few cases, this could cause some subtle side-effects (such as dependence on the ex‐ act contents of $0 or __FILE__) and the optimization can be disabled by enabling the disable_exec_load setting.
Shelling out Any Ruby code that opens a subshell (like system, backticks, or %x{}) will automatically use the current Bundler environ‐ ment. If you need to shell out to a Ruby command that is not part of your current bundle, use the with_clean_env method with a block. Any subshells created inside the block will be given the environment present before Bundler was activated. For example, Homebrew commands run Ruby, but don´t work in‐ side a bundle:
Bundler.with_clean_env do `brew install wget` end
Using with_clean_env is also necessary if you are shelling out to a different bundle. Any Bundler commands run in a sub‐ shell will inherit the current Gemfile, so commands that need to run in the context of a different bundle also need to use with_clean_env.
Bundler.with_clean_env do Dir.chdir "/other/bundler/project" do `bundle exec ./script` end end
Bundler provides convenience helpers that wrap system and exec, and they can be used like this:
Bundler.clean_system(´brew install wget´) Bundler.clean_exec(´brew install wget´)
RUBYGEMS PLUGINS At present, the Rubygems plugin system requires all files named rubygems_plugin.rb on the load path of any installed gem when any Ruby code requires rubygems.rb. This includes executables installed into the system, like rails, rackup, and rspec.
Since Rubygems plugins can contain arbitrary Ruby code, they commonly end up activating themselves or their dependencies.
For instance, the gemcutter 0.5 gem depended on json_pure. If you had that version of gemcutter installed (even if you also had a newer version without this problem), Rubygems would ac‐ tivate gemcutter 0.5 and json_pure <latest>.
If your Gemfile(5) also contained json_pure (or a gem with a dependency on json_pure), the latest version on your system might conflict with the version in your Gemfile(5), or the snapshot version in your Gemfile.lock.
If this happens, bundler will say:
You have already activated json_pure 1.4.6 but your Gemfile requires json_pure 1.4.3. Consider using bundle exec.
In this situation, you almost certainly want to remove the underlying gem with the problematic gem plugin. In general, the authors of these plugins (in this case, the gemcutter gem) have released newer versions that are more careful in their plugins.
You can find a list of all the gems containing gem plugins by running
ruby -e "puts Gem.find_files(´rubygems_plugin.rb´)"
At the very least, you should remove all but the newest ver‐ sion of each gem plugin, and also remove all gem plugins that you aren´t using (gem uninstall gem_name).
October 2022 BUNDLE-EXEC(1)
$ bundle exec rake -T rake db:create # Creates the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:create:all to create all databases) rake db:create_migration # Create a migration (parameters: NAME, VERSION) rake db:drop # Drops the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:drop:all to drop all databases) rake db:encryption:init # Generate a set of keys for configuring Active Record encryption in a given environment rake db:environment:set # Set the environment value for the database rake db:fixtures:load # Loads fixtures into the current environment's database rake db:migrate # Migrate the database (options: VERSION=x, VERBOSE=false, SCOPE=blog) rake db:migrate:down # Runs the "down" for a given migration VERSION rake db:migrate:redo # Rolls back the database one migration and re-migrates up (options: STEP=x, VERSION=x) rake db:migrate:status # Display status of migrations rake db:migrate:up # Runs the "up" for a given migration VERSION rake db:prepare # Runs setup if database does not exist, or runs migrations if it does rake db:reset # Drops and recreates all databases from their schema for the current environment and loads the seeds rake db:rollback # Rolls the schema back to the previous version (specify steps w/ STEP=n) rake db:schema:cache:clear # Clears a db/schema_cache.yml file rake db:schema:cache:dump # Creates a db/schema_cache.yml file rake db:schema:dump # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on ENV['SCHEMA_FORMAT'] or config.active_...) rake db:schema:load # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on ENV['SCHEMA_FORMAT'] or config.active_re...) rake db:seed # Loads the seed data from db/seeds.rb rake db:seed:replant # Truncates tables of each database for current environment and loads the seeds rake db:setup # Creates all databases, loads all schemas, and initializes with the seed data (use db:reset to also drop all databases first) rake db:version # Retrieves the current schema version number
All of the above tasks must be run by prefacing them with bundle exec.
If this becomes tiresome, define an alias for rake to bundle exec rake
as follows:
$ echo 'rake="bundle exec rake"' >> ~/.bash_aliases $ source ~/.bash_aliases
If you define an alias as shown above, then when you type a command like:
$ rake db:some_task_name
... the bash alias will expand your command line so the following is executed:
$ bundle exec rake db:some_task_name
Database Definition
Before the Active Record tasks can be used,
database parameters must be specified.
Typically this is done by creating
config/database.yml.
The following specifies sqlite for development and testing,
and PostgreSQL for production:
default: &default adapter: sqlite3 timeout: 5000 development: <<: *default database: db/development.sqlite3 test: <<: *default database: db/test.sqlite3 production: adapter: postgresql encoding: unicode pool: 5 host: <%= ENV['DATABASE_HOST'] || 'db' %> database: <%= ENV['DATABASE_NAME'] || 'sinatra' %> username: <%= ENV['DATABASE_USER'] || 'sinatra' %> password: <%= ENV['DATABASE_PASSWORD'] || 'sinatra' %>
The above database parameters needs two more gems in Gemfile,
one for each type of database:
gem 'sqlite3' gem 'pg'
The pg gem requires either libpq to be installed in the OS,
or a PostgreSQL client package must be installed, for example one of these, depending on your OS:
$ yes | sudo apt install libpq-dev $ sudo yum install postgresql-devel $ sudo zypper in postgresql-devel $ sudo pacman -S postgresql-libs
You need to install the newly added gems before you can do anything further with this project:
$ bundle Fetching gem metadata from https://rubygems.org/......... Resolving dependencies... Using rake 13.0.6 Using concurrent-ruby 1.2.2 Using minitest 5.18.0 Using i18n 1.12.0 Using tzinfo 2.0.6 Using pg 1.5.1 Using rack 2.2.7 Using tilt 2.1.0 Using sqlite3 1.6.2 (x86_64-linux) Using bundler 2.4.6 Using ruby2_keywords 0.0.5 Using activesupport 7.0.4.3 Using mustermann 3.0.0 Using activemodel 7.0.4.3 Using rack-protection 3.0.6 Using activerecord 7.0.4.3 Using sinatra 3.0.6 Using sinatra-activerecord 2.0.26 Bundle complete! 4 Gemfile dependencies, 18 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed.
Additional configuration is possible for Active Record.
Additional Rakefile Import Needed
Add the following to Rakefile:
require 'sinatra/activerecord'
If the above is not present in Rakefile,
the following error will appear each time you attempt to run an Active Record rake task:
ActiveRecord::AdapterNotSpecified: The `development` database is not
configured for the `development` environment.
At present, Rakefile is the only file in this project that contains Ruby code.
Active Record has all the logic for creating CRUD models,
and the sinatra-activerecord Gem integrates it with Sinatra.
$ unset DATABASE_URL
The environment variables
RAILS_ENV, RACK_ENV and APP_ENV are all similar
and can be used interchangeably much of the time.
Their purpose is to define the mode in which to run the application.
Common choices are: development, production, and test,
but you can define other values.
The default value is development.
APP_ENV- Sinatra-specific
RACK_ENV- Recognized by all
rackapplications by default RAILS_ENV- Rails-specific
I recommend that you adopt one of the following conventions:
-
For development, unset all of them:
Shell
$ unset RAILS_ENV RACK_ENV APP_ENV -
Unset all of these environment variables and only set the one that you intend to use:
Shell
$ unset RAILS_ENV RACK_ENV APP_ENV $ export APP_ENV=production
-
Set all of them to the same value.
Shell
$ export APP_ENV=production $ export RACK_ENV=$APP_ENV $ export RAILS_ENV=$APP_ENV
For Further Reading
Now that you know how Active Record is distinct from Rails,
you might find this article helpful:
Active Record Basics,
by Rails Guides.
This documentation does not emphasize that the Rails build system is actually rake,
and that the rails command simply forwards build-related command lines to rake.
You can use the min-sin
Ruby Sinatra project that we just recreated
to learn about Active Record from the Rails Guide without using Rails
by substituting bundle exec rake every time you see bin/rails.
Thus, when the Active Record documentation says:
$ bin/rails db:migrate
You should write instead:
$ bundle exec rake db:migrate
Which Tasks To Use?
There are so many rake tasks!
It seems overwhelming.
Happily, you only need to know a few of them most of the time.
Provided that you followed along and installed all the dependencies, the following tasks should all be operational. This article merely discusses them, but does not use them to add CRUD functionality to the webapp. Please experiment!
db:setup
Normally, the db:setup task is the first Active Record rake task to run.
This task creates the database, loads the schema, and initializes the schema with any available seed data.
$ bundle exec rake db:setup
db:create_migration
The db:create_migration task accepts two optional parameters
(NAME and VERSION),
and uses them to create a new migration.
Run the db:create_migration task like this:
$ bundle exec rake db:create_migration NAME=create_users_table
db:migrate
Once you have created a migration,
the db:migrate task creates the corresponding database table(s).
$ bundle exec rake db:migrate
db:drop and db:reset
The db:drop task drops the database.
The db:reset task drops the database and sets it up again.
Use this task as follows:
$ bundle exec rake db:reset
This is functionally equivalent to:
$ bundle exec rake db:drop db:setup
The above shows how two tasks can be specified at once; they are executed in sequence.
db:rollback
If you commit the change introduced by a migration and later discover a problem with the migration,
then you cannot just edit the migration and rerun it.
This is because rake only runs migrations once,
so nothing happens when you attempt to run the db:migrate task again.
You must first use db:rollback task to roll back the most recent migration
before editing it and rerunning the db:migrate task.
$ bundle exec rake db:rollback
bundle exec rake db:rollback can be invoked more than once;
it deletes the most recent migration each time it is used.
The following example rolls back the two most recent migrations,
then attempts the most recent migration again.
$ bundle exec rake db:rollback db:rollback $ # Make changes to the project's problem migration $ bundle exec rake db:migrate
Make This Project Into A Sinatra App
At present, the min-sin project is not actually a Sinatra webapp.
Yes, you can create data migrations and data models,
but the min-sin project cannot yet serve web pages.
This article was structured this way so you could realize the contribution that Active Record makes
to a project, separate from the contribution that Sinatra makes.
To turn the min-sin project into a Sinatra webapp that can serve web pages,
we just need to make two small files: config.ru and app.rb.
To understand why the following instructions work,
you need to know that Sinatra (and Rails) comply with the
rack specification.
Rack applications are normally launched with the
rackup command.
By default, the rackup command loads and runs a file called config.ru
in the top-level directory of a rack-compliant webapp.
The config.ru file has no relationship with the config/
directory we saw earlier.
The similar names can be confusing.
As we saw earlier,
the config/ directory defines this project’s databases for Active Record.
In contrast, config.ru is for launching the Sinatra webapp;
this works because Sinatra is rack-compliant.
To be precise, Rack middleware
is launched by running config.ru.
This is the rackup help information:
$ rackup -h Usage: rackup [ruby options] [rack options] [rackup config]
Ruby options: -e, --eval LINE evaluate a LINE of code -d, --debug set debugging flags (set $DEBUG to true) -w, --warn turn warnings on for your script -q, --quiet turn off logging -I, --include PATH specify $LOAD_PATH (may be used more than once) -r, --require LIBRARY require the library, before executing your script
Rack options: -b BUILDER_LINE, evaluate a BUILDER_LINE of code as a builder script --builder -s, --server SERVER serve using SERVER (thin/puma/webrick) -o, --host HOST listen on HOST (default: localhost) -p, --port PORT use PORT (default: 9292) -O NAME[=VALUE], pass VALUE to the server as option NAME. If no VALUE, sets it to true. Run '/home/mslinn/.rbenv/versions/3.1.0/bin/rackup -s SERVER -h' to get a list of options for SERVER --option -E, --env ENVIRONMENT use ENVIRONMENT for defaults (default: development) -D, --daemonize run daemonized in the background -P, --pid FILE file to store PID
Profiling options: --heap HEAPFILE Build the application, then dump the heap to HEAPFILE --profile PROFILE Dump CPU or Memory profile to PROFILE (defaults to a tempfile) --profile-mode MODE Profile mode (cpu|wall|object)
Common options: -h, -?, --help Show this message --version Show version
If you place config.ru in the top-level directory of a Sinatra/rack project,
when we run the rackup command to launch the webapp,
config.ru will be found without requiring any options.
require_relative 'app' run Sinatra::Application
As you can see above, config.ru requires app.rb;
this is the entry point for the logic that we want to provide in the Sinatra webapp.
Next, config.ru starts the Sinatra web server.
app.rb is similarly simple;
it can either define a classic Sinatra webapp
or a modular Sinatra webapp.
I find modular Sinatra webapps easier to maintain,
so the following is the smallest possible modular Sinatra webapp.
require "sinatra/base"
class MyApp < Sinatra::Base
get '/' do
"A very classy "hello" to you!!"
end
end
Run The Webapp
Now we can run our modular Sinatra webapp:
$ rackup 2023-04-27 09:20:35 -0400 Thin web server (v1.8.2 codename Ruby Razor) 2023-04-27 09:20:35 -0400 Maximum connections set to 1024 2023-04-27 09:20:35 -0400 Listening on localhost:9292, CTRL+C to stop
Point your web browser to localhost:9292
and you should see:
A very classy "hello" to you!
Go Forth and Migrate
This article removed the mystery of setting up Active Record with Sinatra.
You can use the min-sin Sinatra Active Record project as a starting point.
Now you should be able to stumble towards making a functional CRUD app without getting confused by the Rails documentation when you encounter it.
About the Author
I, Mike Slinn, have been working with Ruby for a long time now. Back in 2005, I was the product marketing manager at CodeGear (the company was formerly known as Borland) for their 3rd Rail IDE. 3rd Rail supported Ruby and Ruby on Rails at launch.
In 2006, I co-chaired the Silicon Valley Ruby Conference on behalf of the SD Forum in Silicon Valley. As you can see, I have the t-shirt. I was the sole chairman of the 2007 Silicon Valley Ruby Conference.
Several court cases have come my way over the years in my capacity as a software expert witness. The court cases featured questions about IP misappropriation for Ruby on Rails programs. You can read about my experience as a software expert if that interests you.
I currently enjoy writing Jekyll plugins in Ruby for this website and others, as well as Ruby utilities.