Ugly Stool Rotating Header Image

WCF REST Client for SmugMug – 3 of n

The last post covered my failed attempts (to me) to use an XML schema to generate the XML parser.  Instead of continuing down the path of an auto-generated parser, I chose a slightly different approach.  I pushed all communication activities into WCF until the input was .NET methods, and the output was .NET types.  The following graphic better illustrates how WCF was used.

wcf-post-01-a

The magic of the green WCF strip is done with the IClientMessageFormatter interface.  Obra implements this interface, and specifically overrides the DeserializeReply  method to pre-process SmugMug’s XML responses.  DeserializeReply is used to strip the outer <rsp/> element, and the <method/> element from the XML response.

This XML data…

<rsp stat="ok">
  <method>smugmug.images.get</method>
  <Images>
    <Image FileName="image-1.jpg" Format="JPG" id="1001">
      <Album Key="FxWoe" id="1000"/>
    </Image>
  </Images>
</rsp>

…is converted to this XML data.

<Images>
  <Image FileName="image-1.jpg" Format="JPG" id="1001">
    <Album Key="FxWoe" id="1000"/>
  </Image>
</Images>

The pre-processed XML data contains only the relevant type metadata.  All of the extraneous information has been removed.  Writing a .NET class to represent these data are simple.  A class to represent the <Images/> element is show below.  This class is automatically picked up by the WCF XML deserailizer due to the XmlRoot attribute, and because it is located in the same .NET assembly as the WCF client.

[XmlRoot("Images")]
public class ImagesContract
{
    private readonly List<ImageContract> image =
        new List<ImageContract>();
    [XmlElement("Image")]
    public List<ImageContract> Images
    {
        get { return image; }
    }
}

Transforming the XML data to an instance of a class is done by the default implementation of IClientMessageFormatter.  Obra implements the IClientMessageFormatter interface, but it does so only to pre-process the response.  Once the response is pre-processed, Obra passes the response to the default implementation of DeserializeReply.

Obra’s implementation of DeserializeReply also checks for SmugMug error responses.  An example response is shown below.  If an error is encountered, a exception is throw with the information contained in the response.

<rsp stat="fail">
  <err code="17" msg="invalid method"/>
</rsp>

I have shown a better (at least alternative) solution for consuming a REST (POX) web service using WCF.  If you are parsing XML in your client you can do much better.  WCF exposes hooks that obviate the need for it, and WCF makes it trivial to communicate with a web service in a more type-safe manner.  Error conditions can be caught and dealt with earlier, which should make client code more straight-laced.  WCF makes it easy to push communication code away from your client.

In the next post I will cover how Obra uses WCF to send data to SmugMug.

WCF REST Client for SmugMug – 2 of n

In the last post I gave a brief overview of the SmugMug API, and what I wanted WCF to do.  To summarize, I wanted WCF to provide two things: strongly typed objects, and exceptions.

My first attempt to avoid XML parsing was to create an XML schema.  I had hoped to use an XML schema from SmugMug, but unfortunately I could not find one.  Furthermore, the SmugMug API wiki does not provide enough detail to create an XML schema.  SmugMug does a good, but not a great job of documenting the XML responses for all of the API methods.  (Granted, this is a tedious and laborious task, so it is understandable.)

The easiest way I found to generate an XML schema was to get example XML data directly from SmugMug by hand-crafting URI’s and using cURL.

The skeleton of my first attempt at an XML schema is shown below.

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:complexType name="Albums"/>
  <xs:complexType name="Images"/>
  <xs:complexType name="ImageExif"/>
  <xs:complexType name="Response">
    <xs:sequence>
      <xs:element name="method" type="xs:string"/>
      <xs:choice>
        <xs:element name="Albums" type="Albums"/>
        <xs:element name="Images" type="Images"/>
        <xs:element name="Image" type="ImageExif"/>
      </xs:choice>
    </xs:sequence>
  </xs:complexType>
  <xs:element name="rsp" type="Response"/>
</xs:schema>

SmugMug wraps all objects in a response (<rsp/>) element, but only one object is returned in a response, where an object is Albums, Images, or Image.  This is modeled in the schema with a choice element.  The xsd.exe tool does not generate desirable code based on the XML schema shown above.  The generated code is shown below.

[XmlElementAttribute("Albums", typeof(Albums)]
[XmlElementAttribute("Image", typeof(ImageExif)]
[XmlElementAttribute("Images", typeof(Images)]
public object Item {
    get {
        return this.itemField;
    }
    set {
        this.itemField = value;
    }
}

The type information for Albums, Images, and Image has been condensed to a C# object ((System.Object) because it is the only type common to Albums, Images, an Image.  (I potentially could have solved this issue using derivation in the schema if there was commonality across all of SmugMug types I cared about, but I had not bothered to investigate.)

Instead of using a choice element to model the fact that only one object is returned, I could have used a mixture of minOccurs and maxOccurs.  An example schema is shown below.

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:complexType name="Albums"/>
  <xs:complexType name="Images"/>
  <xs:complexType name="ImageExif"/>
  <xs:complexType name="Response">
    <xs:sequence>
      <xs:element name="method" type="xs:string"/>
      <xs:element minOccurs="0" maxOccurs="1" name="Albums" type="Albums"/>
      <xs:element minOccurs="0" maxOccurs="1" name="Images" type="Images"/>
      <xs:element minOccurs="0" maxOccurs="1" name="Image" type="ImageExif"/>
    </xs:sequence>
  </xs:complexType>
  <xs:element name="rsp" type="Response"/>
</xs:schema>

The schema duplicates the behavior of the choice element, but forces the xsd.exe tool to generate type information for the SmugMug objects.  The generated code is shown below.

[XmlElementAttribute]
public Albums Albums {
    get {
        return this.albumsField;
    }
    set {
        this.albumsField = value;
    }
}
[XmlElementAttribute]
public Images Images {
    get {
        return this.imagesField;
    }
    set {
        this.imagesField = value;
    }
}
[XmlElementAttribute]
public ImageExif Image {
    get {
        return this.imageField;
    }
    set {
        this.imageField = value;
    }
}

This code is closer to what I want.  The type is explicit – no casting required.  The objection is the fact that there are potentially many dead types to choose from.  If I am fetching Albums, why should the code ever have to consider anything but the list of Albums.  The calling code has to keep track of the method invoked, and then switch based on the SmugMug method to determine what property to interrogate.  Potentially, the client code devolves into the following.

Response response = this.InvokeSmugMugMethod();
switch (response.method)
{
    case "smugmug.albums.get":
        // do something with albums
        break;
    case "smugmug.images.get";
        // do something with images
}

In my next post I will describe the solution I used to solve these issues.

WCF REST Client for SmugMug – 1 of n

I created an open source project called Obra.  Obra is a PowerShell drive for SmugMug, which basically means you can treat SmugMug as if it was your C:.  You can execute directory listing, rename files, delete files, and look at file metadata.  In the case of SmugMug metadata means the album’s description, image dimensions, EXIF, etc.

I wrote Obra to learn about technologies I have not really used like WCF and PowerShell.  I had previously written a read-only ruby client for SmugMug, but it stalled when I started to implement write-support.  I did not want to lose people’s pictures, and I couldn’t figure out a good way to test my client without re-implementing the SmugMug REST service.  (This was me, not ruby.)  Fast-forward a couple of years, and I was now employed as a C# developer and learning about Nunit and NMock.  I wanted to incorporate these tools into the project as well.

When I started Obra I could not find much material about writing WCF REST clients.  Excluding REST, most of the material seemed biased towards WCF servers, not clients.  Hopefully, this series of posts rectifies that.

The SmugMug API supports REST, JSON, serialized PHP, and provides an XML-RPC service.  In SmugMug’s case REST really means POX, which is the interface Obra uses.  (I assume SmugMug did not re-invent the wheel when they designed their API, and rather based it on the Flickr API; they are very similar.) 

Client requests are HTTP GET’s (with the exception of uploading, which is a POST).  A request encodes the method and parameters in the URI.  For example, the following URI calls a method to get the albums for the session with ID 1ac.

http://api.smugmug.com/hack/rest/1.2.0/?method=smugmug.albums.get&SessionID=1ac

Responses all follow the same basic form.  The root element is <rsp/>, the first child node is <method/> and contains the name of the method called, and the second child node is the response element.  In the case of the method smugmug.albums.get the node is <Albums/>.

<rsp stat="ok">
   <method>smugmug.albums.get</method>
   <albums>
      <album id="0" title="2007-01-01" key="smgms">
         <category id="41" name="Airplanes" />
      </album>
   </albums>
</rsp>

Other methods return different elements for the second child node, such as <Categories/>, <Images/>, and <Login/>.

If the method call resulted in an error an error response is sent instead.  An example error response is shown below.

<rsp stat="fail">
  <err code="17" msg="invalid method" />
</rsp>

I wanted to wrap up this POX API in a tidy package.  I did not want any of the library code to parse XML.  I wanted the code to only operate on strongly typed objects, i.e. Album, Image, etc.  If a method call to SmugMug resulted in an error I wanted an exception to be thrown – avoid checking flags.  More importantly, I did not want to have to write any of this.  I rightly assumed that I code use WCF (and .NET) to handle all of the heavy lifting.  I’ll cover how in the following posts.

Unit Testing PowerShell Snapin

I am writing a PSDrive for PowerShell, and finally figured out how to unit test it without having to explicitly installing the assembly.

The expected flow for PowerShell plugins is the following.

  1. Write a PowerShell plugin.
  2. Register the plugin using installutil.exe.
  3. Add the PSSnapIn from your PowerShell session using Add-PSSnapIn.
  4. Use your PowerShell plugin for glory and profit.

As part of my unit tests I do not want to exec installutil, nor do I want to pollute the existing PowerShell snapins with the one under test, i.e. have the snapin installed twice.  If I am running a unit test for a snapin that is already installed I do not want to use the installed version.  I want the test version to temporarily remove the installed version, and use the snapin under test for the unit tests.  Unfortunately, there does not appear to be any public API to do this, so I used Reflector to figure out how to do it via a backdoor mechanism.

A PowerShell snapin is represented in code as a PSSnapInInfo class.  The list of snapins can be viewed with the Get-PSSnapin cmdlet.  The information contained in a PSSnapInInfo instance can be viewed with the following PowerShell snippet.

$snapin = (Get-PSSnapin)[0]
$snapin | Format-List *

When a snapin is registered with installutil a registry key is created under the key HKLM\Software\Microsoft\PowerShell\1\PowerShellSnapIns.  These registry keys are read by PowerShell at load time to populate the list of available PowerShell snapins.

I do not want unit tests to modify the registry, but I do want to get a hold of the list populated by the registry keys.  A little Reflector magic is necessary, but it eventually leads to four important classes: PSSnapInInfo, MshConsoleInfo, RunspaceConfigurationForSingleShell, RunspaceConfiguration.

PSSnapInInfo holds information about PowerShell snapins.  An instance of this class must be created about the snapin under test.

MshConsoleInfo contains the list of snapins available to a PowerShell session.  The list of snapins can be accessed by the SnapIns property.

RunspaceConfiguration is an abstract base class.  A concrete implementation of this class is RunspaceConfigurationForSingleShell, which contains an instance of MshConsoleInfo.  MshConsoleInfo can be accessed by the ConsoleInfo property.

The solution to avoiding installutil.exe, and programmatically inserting a snapin for the purpose of unit testing is to create a PSSnapInInfo instance, and add it to MshConsoleInfo.  None of these classes are publicly available, so reflection is used to access them.

Step 1, create an instance of PSSnapInInfo.

private static PSSnapInInfo GetPSSnapInInfo(
    string snapInName,
    string absoluteAssemblyPath,
    List types,
    List formats)
{
    Assembly assembly = Assembly.LoadFile(absoluteAssemblyPath, null);
    ConstructorInfo constructorInfo = typeof(PSSnapInInfo).GetConstructor(
        BindingFlags.NonPublic | BindingFlags.Instance,
        null,
        new Type[] {
            typeof(string)/*name*/,
            typeof(bool)/*isDefault*/,
            typeof(string)/*applicationBase*/,
            typeof(string)/*applicationName*/,
            typeof(string)/*moduleName*/,
            typeof(Version)/*psVersion*/,
            typeof(Version)/*version*/,
            typeof(Collection)/*types*/,
            typeof(Collection)/*formats*/,
            typeof(string)/*descriptionFallback*/,
            typeof(string)/*vendorFallback*/,
            typeof(string)/*customPSSnapInType*/},
        null);
    Collection myTypes = new Collection(types);
    Collection myFormats = new Collection(formats);
    object[] parameters = new object[] {
        snapInName/*name*/,
        true/*isDefault*/,
        System.IO.Path.GetDirectoryName(absoluteAssemblyPath)/*applicationBase*/,
        assembly.FullName/*assemblyName*/,
        absoluteAssemblyPath/*moduleName*/,
        new Version(2, 0)/*psVersion*/,
        assembly.GetName().Version/*verison*/,
        myTypes/*types*/,
        myFormats/*formats*/,
        String.Format("{0} SnapIn for Unit Teting", snapInName)/*descriptionFallback*/,
        "nUnit"/*vendorFallback*/,
        null/*customPSSnapInType*/};
    PSSnapInInfo info = (PSSnapInInfo)constructorInfo.Invoke(
        BindingFlags.NonPublic | BindingFlags.Instance,
        null,
        parameters,
        CultureInfo.CurrentCulture);
    return info;
}

Step 2, get the MshConsoleInfo instance from RunspaceConfiguration.

private object GetMshConsoleInfo(
        RunspaceConfiguration configuration)
{
    object consoleInfo = configuration.GetType().GetProperties(BindingFlags.NonPublic | BindingFlags.Instance).First(
        x => x.Name == "ConsoleInfo").GetValue(configuration, null);
    return consoleInfo;
}

Step 3, add PSSnapInInfo for my snapin under test to the list of available snapins.

private void AddPSSnapInToRunspaceConfiguration(
    PSSnapInInfo snapInInfo,
    RunspaceConfiguration configuration)
{
    object consoleInfo = this.GetMshConsoleInfo(configuration);
    var properties = consoleInfo.GetType().GetProperties(BindingFlags.NonPublic | BindingFlags.Instance);
    Collection snapIns = (Collection)consoleInfo.GetType().GetProperties(BindingFlags.NonPublic | BindingFlags.Instance).First(
        x => x.Name == "PSSnapIns").GetValue(consoleInfo, null);
    snapIns.Add(snapInInfo);
}

Step 4, load the PSSnapInInfo.

private void LoadPSSnapIn(
    PSSnapInInfo snapInInfo,
    RunspaceConfiguration configuration)
{
    MethodInfo methodInfo = configuration.GetType().GetMethod(
        "LoadPSSnapIn",
        BindingFlags.NonPublic | BindingFlags.Instance,
        null,
        new Type[] { typeof(PSSnapInInfo) },
        null);
    methodInfo.Invoke(configuration, new object[] { snapInInfo });
}

Viola!

To Help or Not to Help

I use command line utilities frequently, and my fingers tend to be faster than my brain.  My fingers have already decided on the options to enter before I remember what the options are.  This works great for find or grep, but I always forget the correct options for cut, sort, or uniq. 

I get the correct options by reading the help message output.  My fingers race ahead, and enter the help switch with the least number of keystrokes:

  $ uniq –h

uniq: invalid option — h
Try `uniq –help’ for more information.

Uh-oh, I got it wrong.  Thankfully, the message tells me exactly what I did wrong, but I am left wondering why it couldn’t just display the help message in the first place.  The output told me I should have typed (it seems rather smugly), so why not throw a guy a bone.

Media Information with PowerShell

This trick only works if you have Microsoft Expression Encoder installed, and PowerShell.  Load the Encoder assembly, construct a new MediaItem, and pass the first file in the directory ((dir)[0].FullName) as the constructor argument.

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Expression.Encoder")
$media = New-Object -TypeName Microsoft.Expression.Encoder.MediaItem (dir)[0].FullName

After creating the object, you can look at all the properties easily, or look at specific ones.

$media
$media.FileType
$media.MaxBitRate
$media.VideoSize

Swing Set Ground Cover

You are now the proud owner of a swing set, congratulations.  Before your land creatures can begin to scale the walls of Babylon, you will need to get ground cover.  When the kiddies are launching from 10 ft in the air they will require a little cushion to stave off internal bleeding.  Our pediatrician recommended either bark mulch, pea stone, or rubber mulch.  The company we purchased the swing set from recommend between three and six inches of ground cover.

Bark Mulch:

This is the cheapest solution at about $20 a cubic yard.  It will have to replaced in the future because the mulch will decompose.  There may be an issue with creepy crawlies as well.

Pea Stone:

This is the medium cost solution at about $40 a cubic yard.  Do not confuse pea stone with pea gravel.  Pea stone needs to be contained or else it will go everywhere and be difficult to remove.  If the pea stone is not delivered directly into the swing set border have a tarp ready.  Dump the pea stone onto the tarp and then move it into the swing set border.  If (when) the pea stone leaks onto the lawn a shop vac can be used to suck it up.

Rubber Mulch:

This is the most expensive solution at over a $100 a cubic yard.  No one in our area does bulk delivery.  Individual bags must be purchased, but do not purchase them from a big box store, instead try a swing set builder.  The bags are bigger and cheaper.  Rubber mulch is available in a variety of colors if that’s your thing.

Border:

Plan for a border at least three feet from the sides, and seven feet from any swings.  I used pressure treated posts turned on their sides.  The posts are dirt cheap $3 to $4 at a big box store.  To hold the posts in place I drilled two or three 3/8” holes through post, and drove 18” of 3/8” rebar through the holes.  (I got fancy and mitered the post ends together.)  With the rebar the posts are not going anywhere, and will keep the ground cover contained to the swing set’s domain.

Convert a multi-page TIFF to a PDF

I needed a hard copy of some legal documents from my last home sale.  The broker that managed said deal provided all the documents scanned in a single TIFF.  The oracle did not provide much guidance unless I wanted to install spyware or spend money for spyware.  I had the idea to use ImageMagick to possibly extract the images, and then I would create a PDF of each image, and then stitch the PDFs together to later print out.  As it turns out, ImageMagick already does everything I need.

$ convert –rotate 90 docs.tiff[18-20] docs.pdf

The above command extracts pages 19, 20, and 21 from docs.tiff, rotates the pages 90 degrees, and converts the output to PDF.

Radio Advertisements

I spend the majority of my time listening to satellite radio, and I highly recommend it.  Unfortunately, I do not have the service in every vehicle.  When I take the other car I listen terrestrial radio, which is full of commercials and advertisements.  I did not realize how bad it was until I started listening to service that has almost none.  It is downright insidious.  I am listening to the music and focused on it – the lyrics, beat, guitar, etc.  I start to get lost in the music and my mind begins to wander.  Eventually, I re-calibrate because I hit a dog (1 point) or an old lady (7 points), and BAMM I am listening to a commercial.  The worse part is I have been stuck listening to one for the past three minutes, and did not even notice.  Straight from War Pigs to buy our latest thermally insulated windows.

Windows 7 on a Thinkpad R52

Windows 7 does not ship with the wireless NIC drivers for the Intel Pro/Wireless 2915 ABG.  The work around is to download the drivers for WinXP directly from Intel.  After installing the drivers force a Windows Update, and you will get the latest chipset, display, sound, and NIC drivers.  Viola! 

Page optimized by WP Minify WordPress Plugin