Ugly Stool Rotating Header Image

January, 2012:

ruby Sinatra and SSL

I am experimenting with the Google JSON API by writing a back end service.  My intent is to re-use the available client libraries to communicate with my service.  I need to test my implementation, and the mechanics of it all to understand how it works.  I used an existing simple Google service to see how the client would interact with it.

I wanted to mock the Google service quickly, so I started with Sinatra.  I implemented HTTPS support just because – not strictly necessary for this experiment.  I had to hack up the Google service document schema to redirect all URIs from www.googleapis.com to localhost, and I made a few edits to the Google ruby client to override the default port.

The code snippet below creates a self-signed certificate on every start.  I never remember the correct OpenSSL incantation to create a self-signed certificate.  I can find the answer by asking the oracle, but it seems to take a while.  I did not want to have to create another certificate ever again.  By default the auto-generated certificate is good for one year.  (I never expect the service to run for that long.)

Sinatra is configured to response with Hello World! when issuing an HTTP GET request to http://localhost:4567/.  All other HTTP GET requests simply respond with URI requested.  HTTP POST requests echo the data back to the client in the payload.

require 'sinatra/base'
require 'webrick'
require 'webrick/https'
require 'openssl'
name = "/C=US/ST=SomeState/L=SomeCity/O=Organization/OU=Unit/CN=localhost"
ca   = OpenSSL::X509::Name.parse(name)
key = OpenSSL::PKey::RSA.new(1024)
crt = OpenSSL::X509::Certificate.new
crt.version = 2
crt.serial  = 1
crt.subject = ca
crt.issuer = ca
crt.public_key = key.public_key
crt.not_before = Time.now
crt.not_after  = Time.now + 1 * 365 * 24 * 60 * 60 # 1 year
webrick_options = {
    :Port               => 4567,
    :SSLEnable          => true,
    :SSLVerifyClient    => OpenSSL::SSL::VERIFY_NONE,
    :SSLCertificate     => crt,
    :SSLPrivateKey      => key,
    :SSLCertName        => [[ "CN", WEBrick::Utils::getservername ]],
}
def agg_lines(data)
    data.each_line.collect{|x| "  >> #{x}" }.join
end
class MyServer < Sinatra::Base
    get '/' do
        "Hello World!\n"
    end
    get '*' do |x|
        "GET of URI #{x}\n"
    end
    post "*" do |x|
        request.body.rewind
        "POST to URI #{x}\n#{agg_lines(request.body.read)}\n"
    end
end
server = ::Rack::Handler::WEBrick
trap(:INT) do
    server.shutdown
end
server.run(MyServer, webrick_options)

I pieced this script together by crawling StackOverflow posts, and reading documentation.

In Search of rsync for Windows

I like almost everything about rsync.  The efficiency employed when transferring files, the mind-numbing command line options for just about every situation, the ability to work over a pipe, and network synchronization.  The biggest fail is the lack of native Windows support.  Syncing files is not solely relegated to UNIX, so anytime I have two directories to synchronize I shake my fists at the heavens,  “O rsync, rsync, wherefore art thou rsync!”

There are alternatives to rsync on Windows, and I use some of them with various levels of success.  I actually think the tools are fine, and work quite well.  The problem is me, I want to use these tools as if they were actually rsync, and they are not.

Sync Toy

Microsoft provides the free program SyncToy.  It can synchronize two directories, and works well.  It is not quite rsync because…

  1. Must define the directory pairs upfront through a UI.
  2. Leaves debris on the file system that contains records of the data moved between the two directories.
  3. The real power of this tool feels obscured by the UI. 

The latest version of SyncToy (2.1) greatly improves upon previous versions.  I setup jobs, and run them every now and again.

Robocopy

Robocopy is a command line utility to copy files.  It should be available in the default OS install, so if you have Windows it is there.  Robocopy supports mirroring two directories.

Create a backup up My Pictures – save all of the images (*.jpg fies) to my portable USB drive E:.

cmd> robocopy “My Pictures” E:\backup\pictures /MIR *.jpg

Create a backup up My Pictures to my NAS device.  Create a record so if the transfer if interrupted it can be automatically resumed.

cmd> robocopy “My Pictures” \\nas\backup\pictures /MIR /Z *.jpg

Robocopy has lots more options too.  One can do multi-threaded copy, bandwidth throttling, just create directory tree and file names (no data), etc.  Well worth a look for the CLI inclined.

Duplicati

Duplicati is a reimplementation of duplicity for Windows.  They are both great backup utilities, but not exactly what I want.  I mention them because Duplicati’s author created an rsync library for C#.  This library would be a great starting point for a Windows rsync implementation.

There are other rsync library available such as librsync, and rsync.net.  I like the idea of Duplicati’s because it is attached to an actively maintained and working application.  Start reading the source too see the implementation.

In the end it was me, not rsync.  I think the tooling on Windows is more than sufficient to do what I want – mirror two directories.  I leave the door open for experimentation.  I may write my own synchronization tool to bring a few more of the rsync-like abilites to Windows natively.  SyncToy is built upon a publicly available API, and Duplicati provides an OSS .NET rsync library too, so who knows.

Page optimized by WP Minify WordPress Plugin