Monday, April 30, 2012

F#, Clojure and Message Queues on Tibco EMS

It looks like I will be getting much more hands on with Tibco EMS. Since the Tibco EMS system in use will have connections from both .NET platforms and Java platforms, I wanted to write some scripts to run some engineering tests on Tibco EMS. I decided to simulate .NET side connections with F# and Java side connections with Clojure. Taking the sample code from Tibco installation, I created the following F# script that sends messages to a queue from the sample C# code:

#r @"C:\tibco\ems\6.3\bin\TIBCO.EMS.dll"

open System
open TIBCO.EMS

let serverUrl = "tcp://localhost:7222"
let producer = "producer"
let consumer = "consumer"
let password = "testpwd"
let queueName = "testQueue"


let getQueueTextMessages serverUrl  userid password queueName messageProcessor =
    async {
        let connection = (userid,password)
                         |> (new QueueConnectionFactory(serverUrl)).CreateQueueConnection
        let session = connection.CreateQueueSession(false,Session.AUTO_ACKNOWLEDGE)
        let queue = session.CreateQueue(queueName)
        let receiver =  session.CreateReceiver(queue)
        connection.Start()
        printf "Queue connection established!"
        while true do
            try
                receiver.Receive() |> messageProcessor
            with _ ->  ()
    }


let sendQueueTextMessages serverUrl  userid password queueName messages =
    let connection = (userid,password)
                     |> (new QueueConnectionFactory(serverUrl)).CreateQueueConnection
    let session = connection.CreateQueueSession(false,Session.AUTO_ACKNOWLEDGE)
    let queue = session.CreateQueue(queueName)
    let sender = session.CreateSender(queue)
    connection.Start()

    messages
    |> Seq.iter (fun item -> session.CreateTextMessage(Text=item)
                             |> sender.Send)
                             
    connection.Close()



// Just dump message to console for now
let myMessageProcessor (msg:Message) =
    msg.ToString() |> printf "%s\n"


let consumeMessageAsync = getQueueTextMessages "tcp://localhost:7222" "consumer" "testpwd"
let produceMessages queueName messages = sendQueueTextMessages "tcp://localhost:7222" "producer" "testpwd" queueName messages 

// Start message consumer asynchronously
Async.Start(consumeMessageAsync "testQueue" myMessageProcessor)


// Send messages to the Tibco EMS   
[ "Aslund"; "Barrayar"; "Beta Colony"; "Cetaganda"; "Escobar"; "Komarr"; "Marilac"; "Pol"; "Sergyar"; "Vervain"]
|> produceMessages "testQueue"

The queue consumer is implemented asynchronously so it won't block executing subsequent statements. To test Tibco JMS from Java, here is the equivalent Clojure code:

(import '(java.util Enumeration)
        '(com.tibco.tibjms TibjmsQueueConnectionFactory)
        '(javax.jms Message JMSException  Session
                    Queue QueueBrowser 
                    QueueConnection QueueReceiver 
                    QueueSession QueueSender))
                  
(def serverUrl "tcp://localhost:7222")
(def producer "producer")
(def consumer "consumer")
(def password "testpwd")
(def queueName "testQueue")

; Consume Queue Text messages asynchronously
(defn get-queue-text-messages [server-url user password queue-name process-message]
    (future
        (with-open [connection (-> (TibjmsQueueConnectionFactory. server-url)
                                   (.createQueueConnection user password))]
            (let [session (.createQueueSession connection false Session/AUTO_ACKNOWLEDGE)
                  queue (.createQueue session  queue-name)]
                (with-open [receiver (.createReceiver session queue)]              
                    (.start connection)
                    (loop []                       
                        (process-message (.receive receiver))
                        (recur)))))))
                   
; Send multiple Text messages
(defn send-queue-text-messages [server-url user password queue-name messages]
    (with-open [connection (-> (TibjmsQueueConnectionFactory. server-url)
                               (.createQueueConnection user password))]
        (let [session (.createQueueSession connection false Session/AUTO_ACKNOWLEDGE)
              queue (.createQueue session  queue-name)
              sender (.createSender session queue)]
            (.start connection)
            (doseq [item messages]
                (let [message (.createTextMessage session)]
                    (.setText message item)
                    (.send sender message))))))


; Create function aliases with connection information embedded                    
(defn consume-messages [queue-name message-processor]
    (get-queue-text-messages  serverUrl producer password queue-name message-processor))

(defn produce-messages [queue-name messages]
    (send-queue-text-messages  serverUrl producer password queue-name messages))

; Just dump messages to console for now
(defn my-message-processor [message]
    (println (.toString message)))

    
; Start consuming messages asynchronously
(consume-messages "testQueue" my-message-processor)                            

; Send messages to queue
(def my-messages '("alpha" "beta" "gamma" "delta"
                   "epsilon" "zeta" "eta" "theta"
                   "iota" "kappa" "lambda" "mu" "nu"
                   "xi", "omicron" "pi" "rho"
                   "signma" "tau" "upsilon" "phi",
                   "chi" "psi" "omega"))                    

(produce-messages  "testQueue"  my-messages)    

With these scripts, I can easily swap in different message generators and message processors as needed for any testing purposes. When I fired up both these scripts up to the part where queue consumers are running in both F# and Clojure version and then send the messages, I could see that Tibco EMS send half the messages to my F# script and the other half to my Clojure script. Since both of these scripts run in REPL environment, I can easily adjust my level of testing as I get results.

Monday, April 16, 2012

F# and SharePoint 2010 Object Hierarchy/Properties

I had the opportunity to take a SharePoint 2010 class recently. In the class, the labs were mostly a cut and paste affair due to time limitations. Those lab exercises only helps me to become familiar with working in Visual Studio and seeing some of the SharePoint API, but does not really help me engage actively in thinking about what I was actually doing. After the class, I decided to write a F# equivalent of the lab exercises to help me get a deeper understanding and to learn. The class did impress on me that the tooling for SharePoint 2010 development is so much more superior in C# such that I've decided to build only the logic code in F# and retain the C# SharePoint project. This blog post describes my effort in getting that first class exercise working with F# code.

I ended up creating an empty SharePoint 2010 C# project that references a F# library project. I added two application pages to the SharePoint 2010 project, the first being FarmHierarchy.aspx. In FarmHierarchy.aspx, I added the following markup between the opening and closing tags of the <asp:Content> element that has an ID of Main:

<h2>My Farm</h2>
<asp:TreeView ID="farmHierarchyViewer" runat="server"
    ShowLines="true" EnableViewState="true"></asp:TreeView>

The second application page I created was PropertyChanger.aspx. I added the following markup between the opening and closing tags of the <asp:Content> elsement that has an ID of Main:

    <h2>Properties:</h2>
    <asp:Label ID="objectName" runat="server" Text=""></asp:Label><br/><br/>
    <asp:Panel ID="webProperties" runat="server" Visible="false" BorderColor="Orange" BorderStyle="Dashed" BorderWidth="1">
        <asp:Label ID="WebLabel" runat="server" Text="Web Title"></asp:Label>
        <br/>
        <asp:TextBox ID="webTitle" runat="server" EnableViewState="true"></asp:TextBox>
        &nbsp;
        <asp:Button ID="webTitleUpdate" runat="server" Text="Update"/>
        &nbsp;
        <asp:Button ID="webCancel" runat="server" Text="Cancel" />
    </asp:Panel>
    <asp:Panel ID="listProperties" runat="server" Visible="false" BorderColor="Orange" BorderStyle="Dashed" BorderWidth="1">
        <asp:Label ID="ListLabel" runat="server" Text="List Properties"></asp:Label>
        <br/>
        <asp:CheckBox ID="listVersioning" runat="server" EnableViewState="true" Text="Enable Versioning" />
        <br/>
        <asp:CheckBox ID="listContentTypes" runat="server" EnableViewState="true" Text="Enable Content Types" />
        &nbsp;
        <asp:Button ID="listPropertiesUpdate" runat="server" Text="Update" />
        &nbsp;
        <asp:Button ID="listCancel" runat="server" Text="Cancel"/>
    </asp:Panel>

The F# code would iterate through the services, Web applications, site collections, and lists in the SharePoint farm, with the details added to nodes in the TreeView control:


module FsLab01

open System
open System.Web.UI.WebControls
open Microsoft.SharePoint
open Microsoft.SharePoint.Administration
open System.Web.UI

// Convenience methods
let nullfunc _ = ()

// Copied from Clojure
let cond clauses =
    let (_,func) = clauses |> Seq.find (fun (pred,func) -> pred)
    func()


let navListUrl url (id:Guid) =
    sprintf "%s/_layouts/lab01/PropertyChanger.aspx?type=list&objectID=%s" url <| id.ToString()


let navWebUrl url (id:Guid) =
    sprintf "%s/_layouts/lab01/PropertyChanger.aspx?type=web&objectID=%s" url <| id.ToString()

// Recursively add SPWeb & SPList objects
let rec addWeb (web:SPWeb) (parentNode:TreeNode) =

    let node = new TreeNode(web.Title,null,null,navWebUrl web.Url web.ID, "_self")
    node |> parentNode.ChildNodes.Add

    [0..(web.Lists.Count-1)]
    |> Seq.map (fun i -> web.Lists.[i])
    |> Seq.iter (fun item -> 
                      new TreeNode(item.Title,null,null,navListUrl web.Url item.ID,"_self")
                      |> node.ChildNodes.Add)
        
    [0..(web.Webs.Count-1)]
    |> Seq.map (fun i -> web.Webs.[i])
    |> Seq.iter (fun item -> try addWeb item node finally item.Dispose())


// Main function to be called by FarmHierarchy.aspx code to build TreeView control
let loadViewer (viewer:TreeView) (farm:SPFarm) =

    let processSite (webappnode:TreeNode) (site:SPSite) =
        site.CatchAccessDeniedException <- false
        try
            let node = new TreeNode(Text=site.Url)
            node |> webappnode.ChildNodes.Add

            addWeb site.RootWeb node

        finally
            site.CatchAccessDeniedException <- false
            

    let processWebApp (svcNode:TreeNode) (webapp:SPWebApplication) = 
        let node = new TreeNode(Text=webapp.DisplayName )
        node |> svcNode.ChildNodes.Add

        cond [(not webapp.IsAdministrationWebApplication,
               fun _ -> webapp.Sites |> Seq.iter (processSite node));
              (true,nullfunc)]
            
                

    let processService (svc:SPService) =
        let label = sprintf "FarmService (Type=%s; Status=%s)" (svc.TypeName) (svc.Status.ToString())
        let node = new TreeNode(Text=label)
        node |> viewer.Nodes.Add
        match svc with
        | :? SPWebService as websvc -> websvc.WebApplications |> Seq.iter (processWebApp node)
        | _ -> ()

    
    viewer.Nodes.Clear()
    farm.Services |> Seq.iter processService
    viewer.ExpandAll()

As I was creating loadViewer method, I was bothered by the if-else-then clauses in the code. I had a previous blog that talked about this issue when it struck me that what I really wanted was something similar to the cond macro in Clojure. Hence the convenience function called cond in the above F# code.

With the F# code written and packaged as a library, I can now use the above F# function in FarmHierarchy.aspx as follows:

        protected void Page_Load(object sender, EventArgs e)
        {
            SPFarm thisFarm = SPFarm.Local;
            FsLab01.loadViewer(farmHierarchyViewer, thisFarm);
        }

The second part of this lab was to create code to manipulate properties of SharePoint SPWeb or SPList objects. The F# code that manipulates the SharePoint object properties and the UI are as follow:



let changeProperty (page:Page) (objectName:Label) (webtitle:TextBox) 
                   (listpanel:Panel) (webpanel:Panel) 
                   (listVersioning:CheckBox) (listContentTypes:CheckBox) 
                   (webUpdateBtn:Button) (listUpdateBtn:Button)
                   (webCancelBtn:Button) (listCancelBtn:Button)   =    


    let homeurl  baseurl = sprintf "%s/_layouts/lab01/FarmHierarchy.aspx" baseurl

    let wrapupdates (web:SPWeb) action =
        web.AllowUnsafeUpdates <- true
        action()
        web.AllowUnsafeUpdates <- false
        

    let hidepanels () =
        listpanel.Visible <- false
        webpanel.Visible <- false

    let cancel (web:SPWeb) =
        page.Response.Redirect(homeurl web.Url)
    
        
    let checkNull item = 
        cond [(item=null,  fun _ -> objectName.Text <- "Malformed URL"
                                    hidepanels()
                                    "");
              (item<>null, fun _ -> item.ToString())]


    try
        let objectType = checkNull page.Request.["type"]
        let objectId = checkNull page.Request.["objectID"]

        match objectType with
        | "web" -> 
            listpanel.Visible <- false
            webpanel.Visible <- true
            use web = SPContext.Current.Site.OpenWeb(new Guid(objectId))

            // Hook up the events
            let myupdates _ =
                web.Title <- webtitle.Text
                web.Update()

            webUpdateBtn.Click.Add(
                fun _ ->
                    try 
                        wrapupdates web myupdates

                        page.Response.Redirect(homeurl web.Url)
                    with ex ->
                        objectName.Text <- ex.Message
                        hidepanels()
                )

            webCancelBtn.Click.Add(fun _ -> cancel web)

            objectName.Text <- sprintf "Web: %s" web.Title

            cond [(not page.IsPostBack,
                   fun () -> webtitle.Text <- web.Title);
                   (true,nullfunc)]

        | "list" -> 
            listpanel.Visible <- true
            webpanel.Visible <- false
            let web = SPContext.Current.Web
            let splist = web.Lists.[new Guid(objectId)]


            let myupdates _ =
                splist.EnableVersioning <- listVersioning.Checked
                splist.ContentTypesEnabled <- listContentTypes.Checked
                splist.Update()

            listUpdateBtn.Click.Add(
                fun _ ->
                    try 
                        wrapupdates web myupdates
                        page.Response.Redirect(homeurl web.Url)
                    with ex ->
                        objectName.Text <- ex.Message
                        hidepanels())

            listCancelBtn.Click.Add(fun _ -> cancel web)

            cond [(not page.IsPostBack,
                   fun _ -> listVersioning.Checked   <- splist.EnableVersioning
                            listContentTypes.Checked <- splist.ContentTypesEnabled)]
        | _ -> ()


    with  ex-> objectName.Text <- ex.Message

I needed to sign my F# code in order it to work in SharePoint. Since the C# SharePoint 2010 already created the key, all I had to do is to add --keyfile:<Location to keyfile>\key.snk to the F# build setting's "Other Flags" properties. In addition, before I can build and deploy, I needed to add the F# library dll to the C# SharePoint project's package. I can do so by clicking on Package in the SharePoint project and then click on the Advanced button.

I can now call this F# function from PropertyChanger.aspx code as follows:

            FsLab01.changeProperty(this.Page, objectName, webTitle,
                                   listProperties, webProperties, 
                                   listVersioning, listContentTypes,
                                   webTitleUpdate, listPropertiesUpdate,
                                   webCancel,listCancel);

Here's how what FarmHierarchy.aspx would look like :

Here's how what PropertyChanger.aspx would look like if I wanted to modify a SPWeb object:

Here's how what PropertyChanger.aspx would look like if I wanted to modify a SPList object: