Packaging and re-using SharePoint designer workflows (Part 2)

Purpose of this article

The purpose of this article is to explain how to create SPD workflows which are portable using an approach as described in an previous post. I’ve written an small BETA tool to export, import and attach the workflows. The GUID replacement is not the most elegant approach, but is faster than rewriting Microsoft’s SPD activities. Anyway I wanted explain how it should work already.

UPDATE: The tool is live, more on this here: http://agiledirect.wordpress.com/2008/09/11/beta-spd-workflow-export-utility-on-codeplex/ 

Steps for getting it working

The best approach for working with the export workflows tool is as follows;

Preparation

1. Design and create the lists you want to use for you workflow.
2. Export these lists for packaging as you are used to when packaging lists into features, you should already know how to do so.
3. Create an empty site (blank site for instance) to validate and instantiate the lists in using your created feature. This blank site will be your workspace for creating and exporting the SharePoint Designer workflow.

Actual workflow activities

1. Open op SharePoint Designer and create the workflows for that lists in that instantiated site.
2. Use the workflow export utility to export the workflows and generate the workflow feature XML
3. Package the feature in to an WSP (using WSPBuilder for instance), add the SPDToolkit.Workflows.Foundation.DLL to the GAC directory of your WSP solution.
4. Activate the lists feature on an (empty) site and activate the workflow feature. The workflow should now be added to the site and associated with the lists.

Preparation

Just for sample purposes I’ve created the very simple and silly “Beer Orders” and “Approved Beer Orders” lists in a site using the SharePoint UI. Both lists containing nothing more than just column with the default name ‘Title’. The idea of these list is that when user orders an beer with the Title “Heineken” it will immediately go to the approved orders (others will probably never be handled ) . Maybe I will extend the sample later on to let it make more sense.

Now, since we want to use this lists in our workflow, and re-use them on multiple sites, it is important that we package the lists as an feature so we can activate the feature on multiple sites. Activating the feature will instantiate the lists on that particular site. 

So, first we need to convert our UI generated list into a way that we can package it using solution packaging. There are quite some tools out there that can export lists to an list definition XML. You can use whatever tooling you want to create the list definition. I’ve used some self written logic to export the list definition and create the list instance feature. 

The lists are both packaged in an feature cleverly called “Beer Lists Feature”

Since we have packaged the lists, we need to test whether the lists are correctly provisioned. We do this by creating an blank site and activating our “Beer Lists Feature”. When this is correctly provisioned we will use this site for creating our SPD workflows which we will export using our utility.

I’ve activated the “Beer Lists Feature” feature and instantiated the lists in an “Blank Site” which I created at http://litwaredemo:15000/BeerWorkspace/.

Creating the workflow

Now to create the SPD workflow we open up the site at http://litwaredemo:15000/BeerWorkspace/. We can use the new workflow option in SharePoint designer to create the workflow. The result looks a bit like this;

To explain what it does. The workflow checks whether an item is added to “Beer Orders” with the title “Heineken” (Case sensitive). If this is the case it will add an item to “Approved Beer Orders” for processing. 

Exporting the workflow

We will now export our created workflow using the export workflow utility. This utility will export the workflow files and create an analyze XML file to which we can later recreate the workflow.

To export the workflow to a feature we use the following syntax:

SPDToolkit.Workflows.Utility.exe -path C:\Feature\ -name AgileDirect.ExampleSolution.Workflows -title Beer workflow -description Simple beer driven workflow A -url http://litwaredemo:15000/BeerWorkspace/

The arguments should be quite self-explanatory.

This will result in an feature like this;

Packaging it up

Now that we have the feature we need to package it. I use WSP builder, and the solution looks a bit like this in the end. You can see that its nothing more than copying the resulting feature in to the FEATURES folder of the WSP structure.

Now what we need to do is add our custom feature receiver to the GAC, so when the feature gets activated it will execute our custom feature receiver and add and the workflow to the site. To do this add it to the GAC folder of the WSPBuilder folder structure.

Now the feature needs to packaged into an WSP and an solution needs to be build, installed and deployed to an web-application. Again, I used WSPBuilder to do so.

Activating the workflows

To activate the workflows, create an empty site.I called mine, http://litwaredemo:15000/BeerWorkspaceTest/ Activate the “Beer Lists Feature” and the “Beer workflow” feature next. Please note that activating could take quite some time (up to 20 seconds for the first time). This should activate and enable the workflows on that site.

Testing the workflows

Add an item to the list “Beer Orders” with the name  “Heineken” and you will see that the workflow gets triggered and an message gets added to the history, and an item get put in the “Approved Beer Orders”

And you can see the item has appeared on the other list “Approved Beer Orders” as well.

 

Make it easier for your clients using feature stapling

Don’t forget that you can feature staple the features together so that the user only needs to activate one feature.

What’s next?

Well, next I will upload the utility, and I will create an screen-cast to explain how to work with the tool and export workflows. If this is still a bit too complicated to comprehend.

Its out now: Check the post http://agiledirect.wordpress.com/2008/09/11/beta-spd-workflow-export-utility-on-codeplex/

A couple of features on my wish list.

• Create a CodePlex place for it, so I can upload the code and you guys can download and work with the tool.
• Add support for versioning of workflows
• More advanced support for workflows
• Fix some bugs
• Use the object model API instead of the webservice (avoids the long wait when activating)

I’m back from Hawaii

Back from Hawaii, need to get a couple of things in order and then I will post part deux (2) on SPD workflows!

Packaging and re-using SharePoint designer workflows (Part 1)

Purpose of this article

The purpose of this article is to explain some of the inner workings of SharePoint designer workflows. It will also explain why SharePoint designer workflows are not portable by default. Additionally, in the upcoming blog there will be a tool available that allows for basic packaging of  SharePoint designer workflows to features, thus enabling SPD workflows to become a part of your solution packaging process.

SharePoint Designer workflow creation process explained

When you create a workflow in SPD. There is a lot of server-client interaction involved cleverly hidden from the end user. In a nutshell, this is what happens behind the scenes when a user creates a workflow in SharePoint designer.

1. User initiates the creation of a new workflow by choosing ‘New Workflow’ from the SPD menu.
2. SharePoint designer queries the remote server and downloads any workflow activities registered in authorized types (in proxy form) from the server using web services. SPD opens the workflow editor, and allows the user to create the workflow using the available activities on the server.
3. Upon save, SPD checks whether there is a Workflows library available in the site. If not it will create one. Additionally, SPD creates a folder in the Workflows library baring the same name of the workflow to store the files into.
4. SPD serializes the workflow to its XOML (WFName.xoml) equivalent, if any rules logic is used; it will also serialize this information to a (WFName.xoml.rules file).
5. SPD uploads the files into the folder using RPC.
6. After uploading the files it will validate the XOML for any errors by calling a web service method on the server passing the workflow XOML.
7. If the validation is passed it will create a configuration file that states which version of the workflow should (the actual version number of the xoml file) be used and to which list the workflow is associated. It will also specify workflow initiation conditions such as start on item creation, update or delete, and whether it should allow users to start workflow manually. This configuration file is uploaded to the server.
8. The configuration file is processed and the workflow is associated to the list by the server by SPD calling a method on an web service passing the URL and version of the configuration file.

Anatomy of an SharePoint Designer workflow

When you create and save a workflow in SharePoint designer, it stores a couple of files in a folder baring the same name of the workflow in the workflows library as mentioned in the process above.

 

Each workflow folder consists out of the following files;

The workflow XOML file (My Workflow On Test A.xoml)

The workflow XOML file contains the serialized workflow markup. This is the serialized XML/XOML version of the workflow you’ve edited in SharePoint designer. It basically contains a serialized activity graph of all the workflow activities used in your workflow.

   

Additional rules file (My Workflow On Test A.xoml.rules)

if you have used rules logic in your workflow; Sharepoint will create a rules file in the folder as well. This file will contain all custom rules logic generated by the rules engine in the SPD workflow editor.

A workflow configuration file (My Workflow On Test A.xoml.config)
The workflow configuration file stores information on which version of workflow and rules file it should use upon workflow association. It will also contain initiation information such whether the workflow should be started upon creation of an item in the associated list, or when an item is updated and so on. Furthermore, the configuration file will have an id of a list to which it should associate the workflow to.

The reason why SPD workflows cannot be ported

So actually, there is nothing in the workflow engine and SharePoint object model that prevents you from using the same workflow XOML over and over in different sites, and thus packaging up and porting your SPD created workflows. Frankly, I think it’s pretty decent model if it was documented well. The actual thing that makes it impossible to port SPD workflows is the OOB activities that come with SharePoint itself. These activities when serialized to XOML, store GUIDS to lists itself instead of ‘pointers’ to list, or any other indirection or that matter. And here is the weakness exposed, because List-IDS are generated each time a list is instantiated the XOML workflows aren’t reusable, since they will most likely point to non existing lists.

You can see an example of this on line 26, of the XOML file (Lookup activity)

Also the configuration is hard bound to the list id.

In my opinion it feels like the workflow activities where a bit rushed, an that the indirection layer that made portability possible has somehow not survived the PBI prioritization queue.

In part two..

Well, this should be enough to explain how SPD workflows work, for now, in the next post we will go deeper into the workings and I will release an sample tool that can be used to export and import basic SharePoint designer workflows to overcome this problem.

Watch this space!

Happy 2008 all!

Yes I know! 7 days late but still I wish you all a good, healthy and successful 2008! Right now I am building an TFS2008 Continuous Integration / SharePoint solution, when this is done I’ll start blogging regularly again!

Regards,

Emile Bosch

Using SharePoint site’s Regional Settings as your UICulture for WCM presence sites.

This post describes how I use resource files in SharePoint on the custom created publishing pages and DVWP web-parts. The solution provided uses the standard <%resources … %> tag and reads the culture from the regional settings then uses this culture to load the resource file from the APP_GLOBAL_RESOURCES and display the localized text. Therefore enabling to display localized static text to users on a per-site-basis.

I am using variations as a part of an solution for a client. I’ve created quite a lot of web-parts in using the DVWP SharePoint designer, layout pages and master-pages. At some point I needed to localize the static text’s on the pages. The requirements stated that the resources language used on the sites should be on a per-site-basis (such as contact form for nl-NL/de-DE), and had to be made possible to switch it afterwards. The authoring environment or back-end (such as lists etc) should be in English. The first thing that popped in my head was the use of resource functionality provided by the SharePoint or actually ASP.NET framework itself. In this way I could just use the OOB <%$Resources: MyFile, MyKey %> functionality provided by ASP.NET.

1. Creating the resource file for translation
This is pretty straightforward, just create your resources files such as. MyFile.nl-NL.resx, and MyField.de-DE.resx etc. NOTE: don’t forget to alter the localeID between the comments. This is essential if you want SharePoint to pick up the locales.

2. Deploying your resource files to the app_globalresources.
However, to get the resources loaded, files needed to be placed in the “app_globalresources” in the IIS directory of the web site. This can be done manually, but since this requires copy pasting by inferior humans such as I, it is a risk we can’t take (And it would be a showstopper in our automatic build). So we’ll do it by featuring. Unfortunately there is no default way of provisioning this directory using feature functionality out of the box.  Luckily, using SharePoint’s extensive object model and feature receiver API we can create our own logic for copying the resources to the IIS “app_globalresources” directory as pointed out by this post. However, I adapted it slightly to support retracting of resource files, but the main functionality kept intact.

3. Making SharePoint use the regional settings as your culture for displaying publishing pages.
So I created a couple of resource files, in different locale’s but SharePoint didn’t pick it up when I changed the regional settings. This is because ASP.NET uses UICulture for loading resources instead of the Culture setting and ignores the Regional Settings. To bend SharePoint to our will. I derived a class from PublishingPageLayout override the PreInit and the IntializeCulture culture to set the the:

CurrentThread.Current.UICulture = SPContext.Current.Web.Locale.

I used my own PublishingPageLayout (made it RegionalAwarePublishingPageLayout) and used this instead of the original PublishingPageLayout that is used for creating page layouts. This way, SharePoint uses my RegionalPublishingPageLayout implementation instead of its own.

And there you go! Now you can use the standard resources approach in SharePoint, and let SharePoint load the correct language by reading from the Regional Settings.

Exporting lists for features programmatically using SolGen’s API and reflection

This post describes how I re-use SharePoint Solution Generator’s API by reverse engineering the code using Reflector and calling using reflection. By using the SPListTemplateExporter provided by SharePoint Solution Generator we can export List definitions programmatically with necessary companion files and use them in my features for automatic build and packaging. Thus eliminating tedious error prone manual writing of XML files.

For my WCM Presence Factory (I’ll talk about this puppy in my upcoming posts) I needed to export lists I’ve created with SharePoint designer. This had to be done programmatically, so that my factory can export lists from existing sites for automatic builds. I could then use this list template to package it and support incremental updates on my WCM environment. SharePoint Solution Generator (solgen) already hosts functionality to do so, so I decided to use their API to export the lists. Unfortunately MS has all it types set to internal so I couldn’t just reference their API and use their classes. Luckily with some handy reflection work you can still instantiate internal types and call its methods on it to achieve the desired result.

I used Lutz’ reflector to inspect the API and came up with the SPListTemplateExporter to export the lists definitions. (I cramped it all into something that is sort of readable)

private static void ExportTemplate(string
    outputdir, string url, string listname)
{
  using (SPSite site = new SPSite(url))
  {
     using (SPWeb web = site.OpenWeb())
     {
       //Access internal ListTemplateExporter using reflection
       Type t = Type.GetType(“Microsoft.SharePoint.Tools.IO.”+
       “SPListTemplateExporter,SPSolGen, Version=12.0.0.0″);

       //Create a new instance
       object instance = Activator.CreateInstance(t);

       //Create a list of guids
       Guid[] g = new Guid[] { web.Lists[listname].ID };

       t.GetProperty(“TargetSiteUrl”).
           SetValue(instance, url, null);

       t.GetProperty(“ExportLocation”).
           SetValue(instance, outputdir, null);

       //Call Export(Guid[] ids)
       t.GetMethod(“Export”, new Type[]
         { typeof(ICollection<Guid>) })
         .Invoke(instance, new object[] { g });
     }
  }
}

And now you can easily generate a list with its companion files  such as schema.xml. etc which you can use for your feature packaging. Calling the method would be something like:

ExportTemplate(“C:\\”, “http://myserver/”, “MyList”);

PS. Don’t forget to create a reference to solution generator’ exe.
PSPS. You can use the same method to export other artifacts from SharePoint.
PSPSPS. Lutz Roeder should get a medal, or two.

Addendum: Just some general information. If you have lookups defined in your list’ schema.xml, it will not work by default since its referenced to the lookup list by ID. If you change this to ListName=”Lists/<TheNameOfListToLookupTo>”, Sharepoint will try to fix the relationship. Just make sure that the list you are referring to exists.

MS releases WSS/Office/MOSS service packs!

Find all the information you need here:
http://www.sharepointblogs.com/dwise/archive/2007/12/11/sharepoint-service-pack-1-links.aspx

Fixing the ‘Failed on restarted (retrying)’ error with corrupt SPD workflows after restore from template

This post describes an issue of using custom created SharePoint designer workflows when trying to instantiate a previously saved site with SharePoint designer workflows. As a result of this limitation, SharePoint designer workflows won’t start due to incorrect versions of files. This post describes a technical plus pragmatic workaround and provides an utility to fix this. Therefore enabling the re-use of workflows in sites that are saved as template and instantiated.

Pfew! Look at that sentence full with buzzwords. As everyone knows we can create workflows easily in SharePoint Designer (SPD). However, after some modifications and we wish to save the site as template and later restore it, the workflows do not seem to work on the newly created site. The error is something like: Failed on Start (retrying). This is because the versions of the workflow files and the workflow association config xml doesn’t correspond anymore. I’ve created a little STSADMCOMMAND that repairs this association by opening the configuration file, synchronizing version information and saving it back to the server, it also re-attaches the workflow using the WebPartPages webservice, which basically the same SOAP command when you are editing workflows in SPD and saving them. This tool comes in quite handy if you need restore loads of workflows on a site.

The command for this STSADM command is:

stsadm.exe -o repairworkflowassociations -url <url of site where WF are broken> [-resetversions]

The url parameter is the parameter of the web that need its workflows repaired.
The resetversions basically sets the version of the workflows back to 1.0.

So you can save your templates and let fix the workflow the following way:

1. Create a site, edit your workflows as you like.
2. When you are done save your site as template
3. Instantiate your template somewhere else, try to start the workflow, see it fail.
4. Run the tool, on that site. It should correct the workflow.
5. Run the workflow again, it should work now.

Optionally you could also run the tool with the -resetVersions option. This saves the workflow files as version 1.0. If you save that template, and instantiate the site using that template. The workflow will work, and every site instantiated with that template will work. Be sure to run it in the instantiated site and not in the site you are authoring your workflow because it will destroy the WF version history. WARNING: If you use the -resetVersions option, the workflow version history will be destroyed, so make sure to create an backup.

Download the WSP file here.

NOTE: The WSP file hasn’t been thoroughly tested yet, if you find any bugs, please send me an e-mail. I’ll post the source later..

And the now pragmatic approach if you dont have too many WFS: You could also save your template, instantiate the new site using that template, disable versioning in Sharepoint Designer on the workflows list (right click, properties), open each workflow and  save it using SharePoint designer. Save that template, that template can also be instantiated.

This need to be done in that exact order.

Customized _LAYOUTS folder on site collection basis

I’m not really a blog-linker but heck. This one needs to get in the open. Microsoft has now an supported method of customizing the _LAYOUTS folder on SharePoint! This was actually quite a common question I got from my clients. Read the full scoop here: http://support.microsoft.com/kb/944105/en-us.

Developer awareness and the bad reputation of ‘SharePoint Designer’ – A post to all developers.

This post ferociously tries to persuade standard .NET developers by offering them candy to give SharePoint a designer a chance, cause it is indeed a black-belt ninja. And you want it, you know you do.

As a contractor, I often run into the same thing over and over again. For instance, when an client need to get things done in SharePoint (and I’m not available), they often acquire an traditional ASP.NET developer. However, this is what usually happens. Because traditional developers are technical experts they often try to solve the problem using their comfort zone and start hacking in Visual Studio. Most developers haven’t even considered using SharePoint Designer or InfoPath because it’s ‘a WYSIWYG tool, meant for analysts that are wannabe developers, and just looks like front-page’ . Well, don’t blame them, front-page wasn’t that good and Visual Studio has been superior in many ways. And heck, my first SharePoint experience was  indeed in combination with Visual Studio. (I’ve always been an developer, and still am!) But don’t be fooled here guys! It’s time for a change..!

At least 70% of the solutions that customers want in SharePoint can be done using SharePoint designer, without typing any or little code. It provides work-flows, custom forms, designing pages, roll-up functionality and loads more. If you are unsure of its capabilities, check out: WSS 3.0 40 Application templates. Its a nice showcase on what SPD can do.

Therefore, the first thing I do is inform the new developer to learn that custom development in SharePoint is only meant as a last resort, should not be taken lightly, and should be aimed at developing specific bits of reusable functionality. Such as WF activities, site creation logic or other things that don’t come out of the box. I say this, because as soon as coding place, the project gets a lot harder to manage. 

This is usually because the developer has an incredible extensive object model to take in account with all its features such as security, deployment and performance. Also actual development process is slower in general because proper SharePoint development consists somewhat out of an more complicated development cycle: Build, sign, build WSP, addsolution, deploysolution, find bug,  attach debugger, etc etc. And then there is the feature framework to take in account. Of course many things can be automated but it is still somewhat slower than traditional ASP.NET development. Also, the development tools available for SharePoint are still a bit lacking. Another pitfall with SharePoint development is that a lot is often assumed to work in a particular way. Furthermore, the custom developed solution needs to be tested, and maintained trough every version iteration of SharePoint. This doesn’t mean that SharePoint development is a no-go, or a bad thing to do, its just something that needs to be considered thoroughly, planned and carefully managed. Also, one needs to be 100% sure that their isn’t an alternative available out of the box. SharePoint development should never be done ad-hoc or sold that way by managers to customers. I personally recommend every developer to try to solve the problem using SharePoint designer first, and if that isn’t possible see whether one can extend SharePoint designer’s functionality by for instance developing custom workflow activities or web-parts that can be reused in SPD.

However, SharePoint designer is not the holy grail. And at some points you need to have a hybrid solution, thus half of you project will be done using SharePoint designer, and the other half will be solved using custom development. It is not a hard line, but just make sure you get the highest efficiency using both products.

All traditional developers should become aware of this, should be thinking less about the technical part, and more about the functional aspect of the solution. SharePoint is an product intended to be used by businesses for creating solutions to gain more ease and speed by using out of the box software. Make sure you know every bit of SharePoint before you start up that Visual Studio and hack away!