Charlton's Blog

What This Is

This section contains a number of subheadings where I can archive and organize the various things I’m studying/working on from a month-to-month basis. It’s essentially a long list of tersely formatted status updates that I collect and maintain along with some commentary.

Reading…

Books:

• 1Q84 - Haruki Murakami. (978-0-307-59331-3) (fin.)
• ANSI Common Lisp - Paul Graham. (0133708756) (in p.)

I’m a huge fan of metaprogramming, and this is something we heavily take advantage of at Packfiles. We’re a Ruby shop, yet the same principles Paulg discusses on Lisp apply to our approach to desigining powerful DSLs that drive inherently complicated, multi-step processes.

Some collected excerpts from ANSI Common Lisp that have really resonated with me:

Almost any program can benefit from having the language tailored to suit its needs, but the more complex the program, the more valuable bottom-up programming becomes. A bottom-up program can be written as a series of layers, each one acting as a sort of programming language for the one above.

Bottom-up programming leads naturally to extensible software. If you take the principle of bottom-up programming all the way to the topmost layer of your program, then that layer becomes a programming language for the user.

Working bottom-up is also the best way to get reusable software. The essence of writing reusable software is to separate the general from the specific, and bottom-up programming inherently creates such a separation. Instead of devoting all your effort to writing a single, monolithic application, you devote part of your effort to building a language, and part to writing a (proportionately smaller) application on top of it. What’s specific to this application will be concentrated in the topmost layer. The layers beneath will form a language for writing applications like this one—and what could be more reusable than a programming language?

Bigger abstractions and an interactive environment can change the way organizations develop software. The phrase rapid prototyping describes a kind of programming that began with Lisp: in Lisp, you can often write a prototype in less time than it would take to write the spec for one. What’s more, such a prototype can be so abstract that it makes a better spec than one written in English.

As programming environments grow in power, and languages become more abstract, the Lisp style of programming is gradually replacing the old plan-and-implement model.

In the old model, bugs are never supposed to happen. Thorough specifications, painstakingly worked out in advance, are supposed to ensure that programs work perfectly. Sounds good in theory. Unfortunately, the specifications are both written and implemented by humans. The result, in practice, is that the plan-and-implement method does not work very well.

This is just what the new model of programming does assume. Instead of hoping that people won’t make mistakes, it tries to make the cost of mistakes very low. The cost of a mistake is the time required to correct it. With powerful languages and good programming environments, this cost can be greatly reduced. Programming style can then depend less on planning and more on exploration.

Planning is a necessary evil. It is a response to risk: the more dangerous an undertaking, the more important it is to plan ahead. Powerful tools decrease risk, and so decrease the need for planning. The design of your program can then benefit from what is probably the most useful source of information available: the experience of implementing it.

Online:

• David Heinemeier Hansson - The open source gift exchange
• Aaron Patterson - Fast Tokenizers with StringScanner

Listening…

• See last.fm.

Packfiles 📦

I’m excited to announce that Packfiles is officially an Embarc Collective member company!

Joining Florida’s leading nonprofit startup hub marks a significant milestone for both Packfiles and myself personally. It’s the start of something wonderful.

We ❤️ Embarc

Developing…

I’m pleased to announce Packfiles’ first open source gem, gunwale.

Gunwale is a simple gem to help you import CSVs into your ActiveRecord models using SmarterCSV and activerecord-import. It’s a fork of the now-discontinued row_boat gem, now fully updated and overhauled for use with the latest versions of Rails.

Features of Gunwale include:

• Configurable mappings between the columns of your CSV and your database,
• Easily preprocess CSV rows before they’re imported,
• Define your own custom value converters for normalization and preprocessing,
• Flexible custom error handling (handle_failed_row, handle_failed_rows),
• Easily pass options along to SmarterCSV and activerecord-import,
• Configurable logic for duplicate/conflicting records (on_duplicate_key_ignore, on_duplicate_key_update, etc)

What’s more, Gunwale is kind to your database:

• Imports are wrapped in a single database transaction, up to a configurable chunk_size (i.e., you’ll execute n/chunk_size inserts, rather than n inserts),
• By default, failed transactions are rolled back on error conditions (helping your database stay healthy!)

Check out the API for more in-depth details.

Here’s an example:

class ImportProduct < RowBoat::Base
  # required
  def import_into
    Product # The ActiveRecord class we want to import records into.
  end

  # required
  def column_mapping
    {
      # `:prdct_name` is the downcased and symbolized version
      # of our column header, while `:name` is the attribute
      # of our model we want to receive `:prdct_name`'s value
      prdct_name: :name,
      dllr_amnt: :price_in_cents,
      desc: :description
    }
  end

  # optional
  def value_converters
    {
      # Allows us to change values we want to import
      # before we import them
      price_in_cents: -> (value) { value * 1000 }
    }
  end

  # optional
  def preprocess_row(row)
    if row[:name] && row[:description] && row[:price]
      row
    else
      nil # return nil to skip a row
    end
    # we could also remove some attributes or do any
    # other kind of work we want with the given row.
  end

  #optional
  def options
    {
      # These are additional configurations that
      # are generally passed through to SmarterCSV
      # and activerecord-import
      validate: false, # this defaults to `true`
      wrap_in_transaction: false # this defaults to `true`
    }
  end
end