Agile|Direct - Emile Bosch’s Sharepoint Blog

Purpose of this article

The purpose of this article is to explain some of the inner workings of SharePoint designer workflows. It will also explain why SharePoint designer workflows are not portable by default. Additionally, in the upcoming blog there will be a tool available that allows for basic packaging of  SharePoint designer workflows to features, thus enabling SPD workflows to become a part of your solution packaging process.

SharePoint Designer workflow creation process explained

When you create a workflow in SPD. There is a lot of server-client interaction involved cleverly hidden from the end user. In a nutshell, this is what happens behind the scenes when a user creates a workflow in SharePoint designer.

1. User initiates the creation of a new workflow by choosing ‘New Workflow’ from the SPD menu.
2. SharePoint designer queries the remote server and downloads any workflow activities registered in authorized types (in proxy form) from the server using web services. SPD opens the workflow editor, and allows the user to create the workflow using the available activities on the server.
3. Upon save, SPD checks whether there is a Workflows library available in the site. If not it will create one. Additionally, SPD creates a folder in the Workflows library baring the same name of the workflow to store the files into.
4. SPD serializes the workflow to its XOML (WFName.xoml) equivalent, if any rules logic is used; it will also serialize this information to a (WFName.xoml.rules file).
5. SPD uploads the files into the folder using RPC.
6. After uploading the files it will validate the XOML for any errors by calling a web service method on the server passing the workflow XOML.
7. If the validation is passed it will create a configuration file that states which version of the workflow should (the actual version number of the xoml file) be used and to which list the workflow is associated. It will also specify workflow initiation conditions such as start on item creation, update or delete, and whether it should allow users to start workflow manually. This configuration file is uploaded to the server.
8. The configuration file is processed and the workflow is associated to the list by the server by SPD calling a method on an web service passing the URL and version of the configuration file.

Anatomy of an SharePoint Designer workflow

When you create and save a workflow in SharePoint designer, it stores a couple of files in a folder baring the same name of the workflow in the workflows library as mentioned in the process above.

Each workflow folder consists out of the following files;

• The workflow XOML file (My Workflow On Test A.xoml)
  The workflow XOML file contains the serialized workflow markup. This is the serialized XML/XOML version of the workflow you’ve edited in SharePoint designer. It basically contains a serialized activity graph of all the workflow activities used in your workflow.

     

• Additional rules file (My Workflow On Test A.xoml.rules)
  if you have used rules logic in your workflow; Sharepoint will create a rules file in the folder as well. This file will contain all custom rules logic generated by the rules engine in the SPD workflow editor.

  
  

• A workflow configuration file (My Workflow On Test A.xoml.config)
  The workflow configuration file stores information on which version of workflow and rules file it should use upon workflow association. It will also contain initiation information such whether the workflow should be started upon creation of an item in the associated list, or when an item is updated and so on. Furthermore, the configuration file will have an id of a list to which it should associate the workflow to.

  

The reason why SPD workflows cannot be ported

So actually, there is nothing in the workflow engine and SharePoint object model that prevents you from using the same workflow XOML over and over in different sites, and thus packaging up and porting your SPD created workflows. Frankly, I think it’s pretty decent model if it was documented well. The actual thing that makes it impossible to port SPD workflows is the OOB activities that come with SharePoint itself. These activities when serialized to XOML, store GUIDS to lists itself instead of ‘pointers’ to list, or any other indirection or that matter. And here is the weakness exposed, because List-IDS are generated each time a list is instantiated the XOML workflows aren’t reusable, since they will most likely point to non existing lists.

You can see an example of this on line 26, of the XOML file (Lookup activity)

Also the configuration is hard bound to the list id.

In my opinion it feels like the workflow activities where a bit rushed, an that the indirection layer that made portability possible has somehow not survived the PBI prioritization queue.

In part two..

Well, this should be enough to explain how SPD workflows work, for now, in the next post we will go deeper into the workings and I will release an sample tool that can be used to export and import basic SharePoint designer workflows to overcome this problem.

Watch this space!

Yes I know! 7 days late but still I wish you all a good, healthy and successful 2008! Right now I am building an TFS2008 Continuous Integration / SharePoint solution, when this is done I’ll start blogging regularly again!

Regards,

Emile Bosch

I am using variations as a part of an solution for a client. I’ve created quite a lot of web-parts in using the DVWP SharePoint designer, layout pages and master-pages. At some point I needed to localize the static text’s on the pages. The requirements stated that the resources language used on the sites should be on a per-site-basis (such as contact form for nl-NL/de-DE), and had to be made possible to switch it afterwards. The authoring environment or back-end (such as lists etc) should be in English. The first thing that popped in my head was the use of resource functionality provided by the SharePoint or actually ASP.NET framework itself. In this way I could just use the OOB <%$Resources: MyFile, MyKey %> functionality provided by ASP.NET.

1. Creating the resource file for translation
This is pretty straightforward, just create your resources files such as. MyFile.nl-NL.resx, and MyField.de-DE.resx etc. NOTE: don’t forget to alter the localeID between the comments. This is essential if you want SharePoint to pick up the locales.

2. Deploying your resource files to the app_globalresources.
However, to get the resources loaded, files needed to be placed in the “app_globalresources” in the IIS directory of the web site. This can be done manually, but since this requires copy pasting by inferior humans such as I, it is a risk we can’t take (And it would be a showstopper in our automatic build). So we’ll do it by featuring. Unfortunately there is no default way of provisioning this directory using feature functionality out of the box.  Luckily, using SharePoint’s extensive object model and feature receiver API we can create our own logic for copying the resources to the IIS “app_globalresources” directory as pointed out by this post. However, I adapted it slightly to support retracting of resource files, but the main functionality kept intact.

3. Making SharePoint use the regional settings as your culture for displaying publishing pages.
So I created a couple of resource files, in different locale’s but SharePoint didn’t pick it up when I changed the regional settings. This is because ASP.NET uses UICulture for loading resources instead of the Culture setting and ignores the Regional Settings. To bend SharePoint to our will. I derived a class from PublishingPageLayout override the PreInit and the IntializeCulture culture to set the the:

CurrentThread.Current.UICulture = SPContext.Current.Web.Locale.

I used my own PublishingPageLayout (made it RegionalAwarePublishingPageLayout) and used this instead of the original PublishingPageLayout that is used for creating page layouts. This way, SharePoint uses my RegionalPublishingPageLayout implementation instead of its own.

And there you go! Now you can use the standard resources approach in SharePoint, and let SharePoint load the correct language by reading from the Regional Settings.

For my WCM Presence Factory (I’ll talk about this puppy in my upcoming posts) I needed to export lists I’ve created with SharePoint designer. This had to be done programmatically, so that my factory can export lists from existing sites for automatic builds. I could then use this list template to package it and support incremental updates on my WCM environment. SharePoint Solution Generator (solgen) already hosts functionality to do so, so I decided to use their API to export the lists. Unfortunately MS has all it types set to internal so I couldn’t just reference their API and use their classes. Luckily with some handy reflection work you can still instantiate internal types and call its methods on it to achieve the desired result.

I used Lutz’ reflector to inspect the API and came up with the SPListTemplateExporter to export the lists definitions. (I cramped it all into something that is sort of readable)

private static void ExportTemplate(string
    outputdir, string url, string listname)
{
  using (SPSite site = new SPSite(url))
  {
     using (SPWeb web = site.OpenWeb())
     {
       //Access internal ListTemplateExporter using reflection
       Type t = Type.GetType(”Microsoft.SharePoint.Tools.IO.”+
       “SPListTemplateExporter,SPSolGen, Version=12.0.0.0″);

       //Create a new instance
       object instance = Activator.CreateInstance(t);

       //Create a list of guids
       Guid[] g = new Guid[] { web.Lists[listname].ID };

       t.GetProperty(”TargetSiteUrl”).
           SetValue(instance, url, null);

       //Call Export(Guid[] ids)
       t.GetMethod(”Export”, new Type[]
         { typeof(ICollection<Guid&gt })
         .Invoke(instance, new object[] { g });
     }
  }
}

And now you can easily generate a list with its companion files  such as schema.xml. etc which you can use for your feature packaging. Calling the method would be something like:

Find all the information you need here:
http://www.sharepointblogs.com/dwise/archive/2007/12/11/sharepoint-service-pack-1-links.aspx

Pfew! Look at that sentence full with buzzwords. As everyone knows we can create workflows easily in SharePoint Designer (SPD). However, after some modifications and we wish to save the site as template and later restore it, the workflows do not seem to work on the newly created site. The error is something like: Failed on Start (retrying). This is because the versions of the workflow files and the workflow association config xml doesn’t correspond anymore. I’ve created a little STSADMCOMMAND that repairs this association by opening the configuration file, synchronizing version information and saving it back to the server, it also re-attaches the workflow using the WebPartPages webservice, which basically the same SOAP command when you are editing workflows in SPD and saving them. This tool comes in quite handy if you need restore loads of workflows on a site.

The command for this STSADM command is:

stsadm.exe -o repairworkflowassociations -url <url of site where WF are broken> [-resetversions]

The url parameter is the parameter of the web that need its workflows repaired.
The resetversions basically sets the version of the workflows back to 1.0.

So you can save your templates and let fix the workflow the following way:

1. Create a site, edit your workflows as you like.
2. When you are done save your site as template
3. Instantiate your template somewhere else, try to start the workflow, see it fail.
4. Run the tool, on that site. It should correct the workflow.
5. Run the workflow again, it should work now.

Optionally you could also run the tool with the -resetVersions option. This saves the workflow files as version 1.0. If you save that template, and instantiate the site using that template. The workflow will work, and every site instantiated with that template will work. Be sure to run it in the instantiated site and not in the site you are authoring your workflow because it will destroy the WF version history. WARNING: If you use the -resetVersions option, the workflow version history will be destroyed, so make sure to create an backup.

Download the WSP file here.

NOTE: The WSP file hasn’t been thoroughly tested yet, if you find any bugs, please send me an e-mail. I’ll post the source later..

And the now pragmatic approach if you dont have too many WFS: You could also save your template, instantiate the new site using that template, disable versioning in Sharepoint Designer on the workflows list (right click, properties), open each workflow and  save it using SharePoint designer. Save that template, that template can also be instantiated.

This need to be done in that exact order.

I’m not really a blog-linker but heck. This one needs to get in the open. Microsoft has now an supported method of customizing the _LAYOUTS folder on SharePoint! This was actually quite a common question I got from my clients. Read the full scoop here: http://support.microsoft.com/kb/944105/en-us.

As a contractor, I often run into the same thing over and over again. For instance, when an client need to get things done in SharePoint (and I’m not available), they often acquire an traditional ASP.NET developer. However, this is what usually happens. Because traditional developers are technical experts they often try to solve the problem using their comfort zone and start hacking in Visual Studio. Most developers haven’t even considered using SharePoint Designer or InfoPath because it’s ‘a WYSIWYG tool, meant for analysts that are wannabe developers, and just looks like front-page’ . Well, don’t blame them, front-page wasn’t that good and Visual Studio has been superior in many ways. And heck, my first SharePoint experience was  indeed in combination with Visual Studio. (I’ve always been an developer, and still am!) But don’t be fooled here guys! It’s time for a change..!

At least 70% of the solutions that customers want in SharePoint can be done using SharePoint designer, without typing any or little code. It provides work-flows, custom forms, designing pages, roll-up functionality and loads more. If you are unsure of its capabilities, check out: WSS 3.0 40 Application templates. Its a nice showcase on what SPD can do.

Therefore, the first thing I do is inform the new developer to learn that custom development in SharePoint is only meant as a last resort, should not be taken lightly, and should be aimed at developing specific bits of reusable functionality. Such as WF activities, site creation logic or other things that don’t come out of the box. I say this, because as soon as coding place, the project gets a lot harder to manage. 

This is usually because the developer has an incredible extensive object model to take in account with all its features such as security, deployment and performance. Also actual development process is slower in general because proper SharePoint development consists somewhat out of an more complicated development cycle: Build, sign, build WSP, addsolution, deploysolution, find bug,  attach debugger, etc etc. And then there is the feature framework to take in account. Of course many things can be automated but it is still somewhat slower than traditional ASP.NET development. Also, the development tools available for SharePoint are still a bit lacking. Another pitfall with SharePoint development is that a lot is often assumed to work in a particular way. Furthermore, the custom developed solution needs to be tested, and maintained trough every version iteration of SharePoint. This doesn’t mean that SharePoint development is a no-go, or a bad thing to do, its just something that needs to be considered thoroughly, planned and carefully managed. Also, one needs to be 100% sure that their isn’t an alternative available out of the box. SharePoint development should never be done ad-hoc or sold that way by managers to customers. I personally recommend every developer to try to solve the problem using SharePoint designer first, and if that isn’t possible see whether one can extend SharePoint designer’s functionality by for instance developing custom workflow activities or web-parts that can be reused in SPD.

However, SharePoint designer is not the holy grail. And at some points you need to have a hybrid solution, thus half of you project will be done using SharePoint designer, and the other half will be solved using custom development. It is not a hard line, but just make sure you get the highest efficiency using both products.

All traditional developers should become aware of this, should be thinking less about the technical part, and more about the functional aspect of the solution. SharePoint is an product intended to be used by businesses for creating solutions to gain more ease and speed by using out of the box software. Make sure you know every bit of SharePoint before you start up that Visual Studio and hack away!

Greg Clark of C3 associates has been attending the ‘AIIM Sharepoint Meets ECM’ sessions in Chicago and has written a couple of excellent blogs about Sharepoint as an ECM platform. The blog covers different opinions on the richness of MOSS as an ECM system. I recommend reading them if you are into ECM.

Liveblogging AIIM SharePoint Meets ECM Session (1 of 5)

SharePoint Meets ECM: Doculabs Summary (2 of 5)

SharePoint Meets ECM: Document Imaging Breakout Sessions (3 of 5)

SharePoint Meets ECM: SharePoint as an ECM Platform (4 of 5)

SharePoint Meets ECM: Doculabs on the Positioning of SharePoint and Traditional ECM Tools (5 of 5)

In my ECM article I mentioned that WSS didn’t provide support external storage for files and documents. However, Microsoft has recently released new functionality in an API trough a hotfix (which is probably why I didn’t notice it before. Who puts new functionality in a hotfix?) . I didn’t have any time yet to investigate further in this API, but sounding from the description it looks promising when there is an requirement for CAS & HSM storage of Sharepoint documents.

“An external storage API is available for Microsoft Windows Sharepoint Services 3.0. The external storage API lets you store documents or files on an external storage device other than Microsoft SQL Server. This API also lets you upgrade existing Windows Sharepoint Services 3.0 sites to point to an external storage device.”

It could be a very powerful feature in the records center in combination with the DoD-5015.2 that should be released fall 2007. Too bad that  there isn’t any documentation available yet.

You can find information on the hotfix here:
http://support.microsoft.com/kb/938499/en-us