rafekettler.com

My thoughts on programming and technology

CORS and Rack Middleware

08.09.2012 at 09:00 AM in Javascript, Ruby, Rack, Web development, Software | View Comments

I was facing an issue at work today. My Javascript needed to talk to a server. That'd be okay if that server weren't on another domain. Unfortunately, things weren't as simple as just using JQuery.

The Problem and the Solution

I've met a decent number of people (mostly those with little experience with HTTP outside of web programming) who have been taught that cross domain requests from JS were not possible. Some more initiated folks are aware of the protocol that lets us turn on cross domain requests, but think that it requires a lot of effort. I, on the other hand, know that CORS (Cross Origin Resource Sharing) is incredibly easy (at a very basic level, it's one line in an Apache config file). I didn't want to take the easy approach, though. First, our resources were semi-private, so allowing all domains (that is, sending a header Access-Control-Allow-Origin : *) was out of the question. Second, we have people running development servers as well as testing and staging servers, so changing an Apache config file was not very scalable or convenient. This was particularly true because any non-standard ports had to be spelled out in the header.

I decided to solve my problem with Rack middleware. We're using Rails, so I considered doing the following:

class ApplicationController < ActionController::Base
  protect_from_forgery

  before_filter :allow_cors

  def allow_cors
    response.headers['Access-Control-Allow-Origin'] = ALLOWED_DOMAINS
  end

  # ...
end

This seemed strange, though. I wasn't comfortable with the amount of overhead that this might incur, particularly since it would be called for every request. I also thought that something this low-level and unrelated to the application logic didn't really belong in my controllers. So I decided to build some Rack middleware instead.

Rack Middleware

Rack is fairly poorly documented, unfortunately. The API and conventions for writing middleware are no exception to this, and there is very little formal documentation on how to do anything nontrivial with Rack middleware. Fortunately, there is a Github repo, rack-contrib, that is full of real world Rack middlewares. I read the (very readable) source for a few of those examples and knew exactly what to do. This is what I came up with:

First, lib/cors_middleware.rb:

# Add Access-Control-Allow-Origin headers to every request
class CorsMiddleware
  def initialize(app, config_file)
    @@allowed_domains ||= YAML.load_file(config_file)
    @app = app
  end

  def call(env)
    status, headers, body = @app.call(env)
    # Check our list of patterns and see if any match our Origin header.
    # If so, set Access-Control-Allow-Origin to the request's Origin
    origin = env['HTTP_ORIGIN']
    if origin && @@allowed_domains.any? { |pattern| File.fnmatch?(pattern, origin) }
      headers['Access-Control-Allow-Origin'] = origin
    end
    [status, headers, body]
  end
end

Then, config/cors.yml:

# Allowed domains for CORS. Shell style globbing is supported.
 - http://localhost:*

And config/application.rb:

require 'rails/all'
require_relative '../lib/cors_middleware'

# ...

config.middleware.use CorsMiddleware, "#{Rails.root}/config/cors.yml"

I restarted the server and everything worked magnificently.

A Few Final Notes

Some final notes because I found these things to be poorly documented on the internet:

  • The header Access-Control-Allow-Origin does not support any form of globbing except a single wildcard (to allow all domains). This is very unfortunate, but can be solved with some simple code (like my File.fnmatch calls earlier). Remember, * is the only valid value with a star in it.
  • Rack HTTP request headers are kept in the env hash but are all mangled so that everything is all caps and hyphens become underscores. For example, the header Content-Type is env['HTTP_CONTENT_TYPE'].
  • CORS is incredibly easy to implement and everyone exposing public APIs that don't require authentication for some or all of the data should enable it
  • Rack is a very powerful and efficient tool for doing low-level things with HTTP in any Ruby web app (particularly in Rails where everything seems to have a good bit of overhead)
  • The lack of examples in IETF/W3C's RFCs is incredibly frustating. Yes, I can read Backus-Naur form but I'd rather not when I could just have the behavior I want modeled, particularly when your BNF is loaded with random escape sequences to the point where I can't tell if it is intentional or not.
Read and Post Comments

Design for Developers

07.10.2012 at 05:00 PM in Web development, Software | View Comments

I have no idea what I'm doing. Really.

Disclaimer: some of the things that I suggest in this post will make people who know better cringe. I'm not a designer and this post isn't intended for designers; it's intended for developers with little design aptitude and time who still want to produce decent-looking websites. Also, as a matter of warning to more design-savvy developers: this post is intended for the most design-oblivious of them all (specifically, people like me). Read ahead with caution.

If you're like me, then you build a fair number of websites. If you're really like me, most of them look like complete and utter garbage. However, you're still a discerning consumer. You can tell why a site like tumblr looks good and a site like craigslist does not. The answer to that specific questions might just be 15 years of improvements in web technology, but in a more general sense the difference is that a lot of effort clearly went into the design of tumblr while craigslist is almost entirely unstyled.

As a discerning consumer of web design and as a producer of web sites, I take a lot of pride in my work and I'm always concerned about its quality. I know how to produce a good site from a technological perspective -- I can make sites that load fast, sip server resources, and scale well -- but I don't know much about making a site that's graphically appealing. Fortunately, quite a few people do and they've taken the time to share their work in a format that developers can use.

Now, to the meat of the post: my goal here is to share some tools I've come across and used extensively to create websites that don't look terrible or antiquated. Without further adieu, here's the list:

Twitter Bootstrap (don't judge me just yet)

This was a bold first pick. Bootstrap is controversial and a lot of designers hate it. I'm the first to admit: Bootstrap has made it too easy to put absolutely no effort into the design of your site, which has led to an army of sites that look like exact clones of its sample site:

Looks familiar, doesn't it?

I'm here to tell you that, with some customization and willpower, Bootstrap can give you a lot of scaffolding to produce a modern-looking site without having to muck around with too much CSS. Here are a few things about Bootstrap that the developer in me loves:

  • It prevents me from reinventing the wheel (I'm sorry, but no one should have to implement a CSS grid layout, ever. It's been done a million times before)
  • It gives you modern-looking inputs and form controls that look more or less the same across different OSes and browsers
  • It works on the major browsers
  • It comes with a lot of sizing stuff built-in for text, form controls, buttons, etc.
  • It provides a nice, clean typographical look that's a serious upgrade to anything's default
  • It's very well documented and maintained

Bootstrap is the JQuery of CSS for me. It does a lot of stuff that I'd just end up writing on my own (except probably in a less-correct and portable way). It's the lazy (or productive, depending on how you want to spin it) programmer's best friend.

Now, a few caveats:

  • Please, for the love of God, do not use their fixed, black nav bar with light text. I can't even remember if it ever looked good, but today it's a cliché that will make pretty much anyone in the know lose respect for you. It's just incredibly overused.
  • Don't use their default colors (more on how to pick these later). If I see another site with black nav on a white background with dark text and blue buttons (and lots of grey wells) I will kill a pandawhale.

In short, Bootstrap is great for prototyping and rapid development, and is a rock-solid foundation for the front-end design of a site. It is so overused, though, that one must be extremely deft when making use of it if they do not wish to anger the design gods.

Adobe Kuler

I'm not actually sure what Kuler is intended to be. I think it's a color-theme sharing site for designers. I do certainly know what I use it for: I use it to pick colors that go together and make my sites look distinctive.

I'll be honest: I don't know the least thing about color theory. I don't even know if knowing something about color theory would help me do what Kuler does for me. All I know is that when I see a group of colors together, I can tell if they look good. In that sense, Kuler plays to my strengths, by showing some vibrant and distinctive sets of colors that go well together and letting me pick. When I've chosen something that looks decent to me, I take each color's hex code and make a variable for it in my SASS stylesheet. Then, I chug away building a site that looks distinctive.

Note: this is how you should pick colors to customize your Bootstrap site.

Subtle Patterns

One of the habits of good-looking websites that I've noticed (perhaps only after being introduced to Subtle Patterns) is that a lot of them have subtle textures as background images. I have no sensible description for why this is, it just makes sites look better. Subtle Patterns is a collection of unobtrusive textures that you can use as background images for your site to immediately up its design panache.

My workflow for Subtle Patterns is typically to look at every pattern I think might work and try it on the site. Once I've found something that looks good, I stick to it.

Google Web Fonts

Web Fonts is basically a gallery of free fonts designed for the web. My process with this is similar to my Subtle Patterns process: go through fonts, find ones that look suitable, try them out, and pick which one's best.

Conclusion

Use these tools with judgement and grace. Don't think you can replace a professional designer in any case -- there are some situations where you'll have to bite the bullet and hire someone. But, when you're just looking to make a decent-looking minimum viable product or building a personal project that you don't want to be ashamed of, these tools are perfect for you.

Read and Post Comments

Why I Learned to Love Make

07.07.2011 at 12:00 PM in Software | View Comments

I'll be honest -- I never really took the time to learn any formal build system until recently. I hadn't worked on many major projects and at my job, there were only a few people working on any project, so there was no reason to learn anything fancy like Make or Maven. I just wrote shell scripts and that worked fine. But now that I've looked into Make, it doesn't seem like I'll ever go back to plain shell scripts.

In this post, I'm going to take a look at a few things that make Make infinitely better than the primitive way of doing things -- shell scripts or manually, and why Make has improved my life.

My initial apprehensions

I had a long list of reasons why I'd never tried learning Make before. Here's a fairly comprehensive version:

  • I figured my projects were too small to require a specialized build system or dependency tracking
  • I though shell scripts were just okay
  • I like writing bash scripts (Make is actually very similar to writing bash scripts, but with some domain-specific abstraction)
  • I'd heard a lot of disrespect for Make as a system and a language
  • I didn't need to work on any existing Makefiles

Nevertheless, I gave it a try when I had some free time, and it was a cathartic experience. Outlined throughout the rest of the article are a few reasons why I'll never go back to writing scripts in places where Make is king.

Argument parsing

Most of the time, when I first write a shell script to build a project, it starts off as a very simple, linear script. But then, I need to do something else (e.g. generate just one file, or generate a different distribution format, or make a patch), and the script starts to branch. Now, I have to start writing branches on the script to take different actions with different verbs or commands (e.g. installer, executable, and so on). I also have to introduce some additional complexity due to error handling -- I want to gracefully handle cases when the given verb is not a recognized command for the script. Basically, it gets very complicated fast.

In a general-purpose language like Python, there's libraries to abstract over this and make things easier (e.g. argparse or optparse). But bash has no such functionality built in. Enter Make: Make does all of the argument parsing for you. You just have to define targets and error reporting and branching is handled for you. Here's a comparison (note that this makefile is not valid -- I had a tough time formatting tabs, so I used 8 spaces instead):

# Command should be either update or installer
command=$1

if [ $command = 'installer' ]; then
    # do some stuff
elif [ $command = 'update' ]; then
    # do some stuff
else
    echo "Command $command not recognized."
fi
installer: dependency1 dependency2 # And so on with the dependencies
        # Some stuff

update: dependency1 dependency2
        # Some other stuff

You tell me which one is better. I say having the framework already laid is miles better.

Modularity

Almost all build shell scripts are modular -- e.g. what they do happens in discrete steps. At the most basic level, these three steps are setup, build, and cleanup. The build step is often comprised of several components as well. In a shell script, either you perform all three actions or none at all (unless you use the command branching strategy I outlined above, but we've already discussed why that's bad). However, Make is modular by nature -- because a Makefile, when used as intended, is comprised of small targets that combine to make larger, more general targets, it's incredibly simple to generate atomic portions of a build with no fussing around with your shell script. You simply have to specify the target that you want to build.

Make is smarter than you or your shell scripts

On a large project, builds can take a long time. It's important to only build what you need to build, lest you waste valuable time and resources. Again, unless a shell script uses a branching strategy or a complex scheme to check when files were last modified (at which point you'd be writing a Make by hand, which is obviously unnecessary), you can't prevent a build shell script from doing unnecessary work. However, because Make understands the dependencies between your files and checks modified times for files to determine if any particular target needs to be rebuilt, it prevents any needless rebuilding of unchanged targets (with a well written Makefile, of course). Make is smarter and more general than you or your shell script, so you should take advantage of the potential time and resource savings.

DRY

Make, although it probably predates DRY (Don't Repeat Yourself), epitomizes it. You're encouraged given plenty of tools to generalize targets so that you can build all similar targets with one target. You're encouraged to use variables, so that if a command or an option changes (e.g. you switch compilers, or want to use a different version of an interpreter), the changes only have to be made in one place. In fact, I'd go as far to say that in a well-written, general Makefile, any single change should only require modifications in one place.

Ease of use and standards

It's also important to note that Make is something of a standard in software development, particularly among the C and Unix communities. Pick a few major C projects (e.g. Python, Ruby, OpenSSH), and see how many use Makefiles for building. That means that when people need to change your build procedure, they'll know where to look.

Additionally, Make is a standard for users. Most somewhat-savvy users know that in order to install most software from source, you should run make and then sudo make install. That convenience is tremendous boon to your user base.

I'm not advocating conformism for conformity's sake, but conforming to standards dramatically increases your software's ease of use for both users and developers. Happy users become contributors and happy developers contribute more, and that's a good thing.

Automation

Obviously, you can automate running of shell scripts as well, but there's a lot more that could go wrong than just automating a run of Make.

Conclusion and Disclaimer

This post is not an argument for everyone to use Make. It's an argument for everyone to use a formal build system of some form with particular emphasis on the benefits of Make. Most of the stuff I praised Make for can also be said for Rake, Ant, Maven, or any other popular build system. My goal is to convert people away from shell scripts/manual building for any given project in favor of something that saves time and abstracts over a lot of the difficulties of writing shell scripts.

Read and Post Comments