Are you choosing the most efficient way for your SharePoint solutions? A look at CRM 4.0 and SharePoint.

Introduction

Now that SharePoint development becomes easier now the tools and MSDN documentation are up to date and available for everyone it seems that everyone is hammering away in SharePoint custom development. There are tons and tons of blogs explaining how to build your own functionality, custom workflows, forms, field controls and event-handlers. Which makes me wonder whether people are also considering other products of Microsoft stack to aid in there product development and reach their business goals. 

It seems that the complexity and extensibility of the SharePoint platform automatically justifies for developer code frenzy and resort to building custom applications or so called application templates on to SharePoint without considering for instance building MS CRM or CRM / WSS / SharePoint integrated solutions.

SharePoint and CRM 4.0

Now with coming CRM 4.0 with its new workflow and entity relationship features it becomes more attractive to build custom solutions in CRM. Especially for the applications that require complex workflows, entity ownership & sharing and relations between entities. What I have seen a lot of developers do with SharePoint is create applications on top of SharePoint and using the lists as an semi-related database. Create an set of event-handlers or workflow activities to set rights or ownership on items and for handling complex logic. Now this is not necessarily a bad way, but maybe it could have been done more efficient when using an product like CRM and integrating it with SharePoint or WSS.

Probably the best way to explain this is by having an example, the service request scenario:

A common requirement for a large intranet is to streamline and manage service requests coming from the users using the Intranet to support the Intranets governance model.

These service requests can vary from “I want to like to request an new project site” and ‘I want to report offensive or erroneous content” to simple ones like “I would like to retrieve a deleted document”.

When one has an large organization to support, the amount of daily service requests can become unmanageable without an proper service request process. Service requests need to be assigned to different people or teams and handled with a lot of user interaction.  In the past I have written elaborate custom functionality, complex workflows and endless forms to overcome this problem in SharePoint while nowadays this sort an similar functionality can easily be created in CRM. 

As an bonus CRM 4.0 provides out of the box functionality to create service requests, cases and find associated knowledge base items to handle the cases accordingly.

The only thing that needs to be written in order to integrate both parts is an web-part to publish the service request and track its status from CRM using CRM’s web-services.

By using this kind of integration you save a lot of development effort as oppose to building all functionality custom.

A couple of other exciting scenarios with CRM and SharePoint integration:

Governed project portal and site lifecycle management	Creating an project in CRM, which triggers the creation of an project site in a governed taxonomy in the SharePoint environment and notifies users that the site has been created. When the project site’s documents are all passed for review, SharePoint triggers an event in CRM which updates the CRM entity that the project is finished and backups the site accordingly.  Because the status of the site is tracked in CRM, the CRM reporting functionality can be used to generate reports. Additionally, the site’s lifecycle policy  can be monitored and endorsed from CRM.
Custom CRM based user authentication	Create an custom authentication provider for SharePoint that uses CRM as its back-end user-store to authenticate users. Providing functionality to manage SharePoint users for, for instance, an extranet, in CRM.
Employee time-off request application	Users can request time off by filling in an form on SharePoint with creates an request in CRM, which can then be handled by an manager accordingly.
Service request and incident management	Users can report incidents by filling in an form on SharePoint with creates entities in CRM with can be handled accordingly. Reporting can be done with SRS or from CRM.

When to choose what?

The most difficult question to answer is when to choose what development strategy when building this kind of business applications. There is no golden rule for as it largely depends on the situation and business need. To make matters worse both products tend to get more overlapping functionality.

For instance, you can store documents in CRM as attachments and without versioning, (of course it still doesn’t match SharePoint) and you can make your SharePoint applications look like CRM based applications such as presented in the “Microsoft 40 application templates” examples.

Apart from the obvious license and performance considerations there are some other facets one might consider.

Are you building an more database based application with complex workflows and relations between entities? Or more of an document or item repository with basic approval workflows? How big is the group of users that will use this application? When this functionality is build in CRM what impact will this have your CRM license model? Is it worth getting extra CRM CALs against building the application completely custom in SharePoint? What for an impact will building hybrid solutions have for your release management?

Probably the best way is to dive into both products and get a feel for them. Here are a couple of pointers to help you consider what development strategy to choose what when developing applications:

CRM only

• When building large complex workflow or process intensive back-end solutions, for a select group of users go for a CRM solution as CRM provides a lot of configuration, workflow flexibility and options like team sharing, user-assignment and queues already out of the box.  
• Furthermore, when you are basically creating an large application in SharePoint where the lists more or less start to look like relational databases an option might be to port it to CRM. Especially, when there is complex functionality like ownership of list-items and elaborate user permissions involved.

SharePoint only

• Document storage based application and collaboration based solutions
• Simple easy to develop workflows
• Applications with simple entity relations and entities with limited changing ownership.
• Large user base (for instance whole parts of the organization) that uses the application.

SharePoint and CRM hybrid applications

• Back-end is a relatively small selective group of users, logic is complex and results into elaborate tracking of tasks and items.
• Front-end need to be accessible by a large amount of users, everyone on the portal needs for instance be able to create new cases and entities in CRM using an created form.
• Monitoring, managing and reporting needs to be done centrally.

Notes on CRM and SharePoint integration

Please note that Microsoft has plans to integrate SharePoint and CRM in its upcoming office release.

Also be sure to understand when writing to the CRM data model in any way such as web-services is only allowed when you have the CRM External Connector license.

SharePoint Designer Workflow (WSP) Packager source-code released!

Source-code on CodePlex!

Recently I got a lot of requests to release the source-code of the SharePoint designer workflow packager. My idea was to wait a bit until I got time to clean up the code and  to fix some bugs.  But since I am still traveling this could take a while.

If you find any bugs, please feel free to become a contributor and upload your fixes

Check it out here at: http://spdtoolkit.codeplex.com/

Installation notes

Be aware that there are still a couple of bugs in the code, and remember that you need C# 3.5 to be installed on your server to run this.

I won’t recommend running this on production servers unless you are a 100% sure that it works.

What’s next?

Depending on the feedback I might extend the workflow packager but it all depends on Microsoft’s roadmap on Workflows for SharePoint’s Next Release. Personally, I think that MS will port the web-based CRM 4.0 workflow editor to SharePoint, or create shared functionality for it, which would supersede this tool. Until then, I try to keep it updated with bug-fixes.

SPD Workflow Packer Source Release (And why I am a bit quiet)

Hi all,

First off, my apologies!The reason for me not blogging lately is because I am traveling Australia and New Zealand and it’s a bit hard getting wireless in my camper.

However, I am releasing the source code for the workflow packaging tool as soon as I can find a place to put it.

Until then, look at my view

Cheers,

Emile

I’m expected to be back and back to civilization in May

Common reasons why corporate Intranet implementations fail

As an contractor, I usually get hired when something goes wrong, or need to be done immediately. Most of the times this is when projects are failing or about to fail.

I have seen a lot of development oriented companies having trouble implementing SharePoint.  Mainly, this is because these companies are used to developing software targeted for a particular part of the stakeholders’  company. However, when it comes to implementing an Intranet an much wider view and understanding is required.  Developing and deploying an Intranet  requires to involve a lot of different functions from an company, and usually results in political warfare when not properly managed.

However, ERP related companies seem to have less of a struggle when it comes to implementing SharePoint. Basically because they have seen what it takes to implement ERP systems.

I have set up an list of most experienced reasons why corporate intranet implementations fail:

• Primary focus on technology instead of functionality.
• Lack of understanding of the product and inexperienced teams.
• Lack of early end-user involvement and not planning for Intranet adoption.
• Lack of an structured process for getting requirements, developing, managing and transitioning a SharePoint project.
• Absence of a well defined information architecture and not doing usability assessment. 
• Stakeholders support organization not able to support SharePoint.
• Not understanding and being able to communicate the impact of an SharePoint implementation to the stakeholder.
• Stakeholder not being able to provide clear requirements.
• Failing to have an governance strategy.

Be sure to asses this risks when you are starting your SharePoint projects.

Meet me at Office DevCon 2008 in Sydney!

Hi! I will be at the office DevCon 2008 in Sydney, if you are interested in meeting me and would like to discuss a bit on SharePoint project management methodologies, or SharePoint development practices I would be pleased!

I would love to share experiences and knowledge on some real-life SharePoint implementations project management approaches – practices  instead all the ´How do I <Insert feature here>´,  sessions.

Since the real challenge of SharePoint, as I see it, doesn’t lie in technology but in specification, implementation and adoption.

Just drop me an e-mail and we’ll grab a couple of drinks!

SharePoint Internals Happiness!

Recently when I was trying to isolate out how the SharePoint internal authentication system (ACL / ACE) worked, I ran into the following documents describing the internals of SharePoint content databases quite detailed.

The documentation is major since august already, but I thought just to give it some more attention in cause you might have missed it.

You can find it here:

[MS-WSSFO]: Windows SharePoint Services (WSS): File Operations Database Communications Protocol Specification

http://msdn.microsoft.com/en-us/library/cc448602.aspx

It also lists the table and views used in the SharePoint content databases. Excellent information if you want to know more on the internals of SharePoint.

BETA SPD Workflow Export Utility on CodePlex!

Its on CodePlex!

In follow up of my two post regarding SharePoint designer workflows http://agiledirect.wordpress.com/2008/07/04/packaging-and-re-using-sharepoint-designer-workflows-part-1/ and http://agiledirect.wordpress.com/2008/09/10/packaging-and-re-using-sharepoint-designer-workflows-part-2/ I’ve now released the BETA utility to export the workflows and package them in a feature. However, its far from production ready, so I still need to do a lot testing and bug-fixing.

Note on this release

In this release you will find two zip files, one containing the beer sample as discussed in the previous post, and the utility it self.

I haven’t tested all the activities yet and I am not sure how it handles the task forms. If you have any comments or bugs please let me know by sending an mail to info@agiledirect.nl

You can download the utility at: http://www.codeplex.com/SPDToolkit

So, what’s next?

I’m releasing this tool as a part from a book I’m writing for my classes on building process and application templates for SharePoint. The book is expected to be released later this year.

Packaging and re-using SharePoint designer workflows (Part 2)

Purpose of this article

The purpose of this article is to explain how to create SPD workflows which are portable using an approach as described in an previous post. I’ve written an small BETA tool to export, import and attach the workflows. The GUID replacement is not the most elegant approach, but is faster than rewriting Microsoft’s SPD activities. Anyway I wanted explain how it should work already.

UPDATE: The tool is live, more on this here: http://agiledirect.wordpress.com/2008/09/11/beta-spd-workflow-export-utility-on-codeplex/ 

Steps for getting it working

The best approach for working with the export workflows tool is as follows;

Preparation

1. Design and create the lists you want to use for you workflow.
2. Export these lists for packaging as you are used to when packaging lists into features, you should already know how to do so.
3. Create an empty site (blank site for instance) to validate and instantiate the lists in using your created feature. This blank site will be your workspace for creating and exporting the SharePoint Designer workflow.

Actual workflow activities

1. Open op SharePoint Designer and create the workflows for that lists in that instantiated site.
2. Use the workflow export utility to export the workflows and generate the workflow feature XML
3. Package the feature in to an WSP (using WSPBuilder for instance), add the SPDToolkit.Workflows.Foundation.DLL to the GAC directory of your WSP solution.
4. Activate the lists feature on an (empty) site and activate the workflow feature. The workflow should now be added to the site and associated with the lists.

Preparation

Just for sample purposes I’ve created the very simple and silly “Beer Orders” and “Approved Beer Orders” lists in a site using the SharePoint UI. Both lists containing nothing more than just column with the default name ‘Title’. The idea of these list is that when user orders an beer with the Title “Heineken” it will immediately go to the approved orders (others will probably never be handled ) . Maybe I will extend the sample later on to let it make more sense.

Now, since we want to use this lists in our workflow, and re-use them on multiple sites, it is important that we package the lists as an feature so we can activate the feature on multiple sites. Activating the feature will instantiate the lists on that particular site. 

So, first we need to convert our UI generated list into a way that we can package it using solution packaging. There are quite some tools out there that can export lists to an list definition XML. You can use whatever tooling you want to create the list definition. I’ve used some self written logic to export the list definition and create the list instance feature. 

The lists are both packaged in an feature cleverly called “Beer Lists Feature”

Since we have packaged the lists, we need to test whether the lists are correctly provisioned. We do this by creating an blank site and activating our “Beer Lists Feature”. When this is correctly provisioned we will use this site for creating our SPD workflows which we will export using our utility.

I’ve activated the “Beer Lists Feature” feature and instantiated the lists in an “Blank Site” which I created at http://litwaredemo:15000/BeerWorkspace/.

Creating the workflow

Now to create the SPD workflow we open up the site at http://litwaredemo:15000/BeerWorkspace/. We can use the new workflow option in SharePoint designer to create the workflow. The result looks a bit like this;

To explain what it does. The workflow checks whether an item is added to “Beer Orders” with the title “Heineken” (Case sensitive). If this is the case it will add an item to “Approved Beer Orders” for processing. 

Exporting the workflow

We will now export our created workflow using the export workflow utility. This utility will export the workflow files and create an analyze XML file to which we can later recreate the workflow.

To export the workflow to a feature we use the following syntax:

SPDToolkit.Workflows.Utility.exe -path C:\Feature\ -name AgileDirect.ExampleSolution.Workflows -title Beer workflow -description Simple beer driven workflow A -url http://litwaredemo:15000/BeerWorkspace/

The arguments should be quite self-explanatory.

This will result in an feature like this;

Packaging it up

Now that we have the feature we need to package it. I use WSP builder, and the solution looks a bit like this in the end. You can see that its nothing more than copying the resulting feature in to the FEATURES folder of the WSP structure.

Now what we need to do is add our custom feature receiver to the GAC, so when the feature gets activated it will execute our custom feature receiver and add and the workflow to the site. To do this add it to the GAC folder of the WSPBuilder folder structure.

Now the feature needs to packaged into an WSP and an solution needs to be build, installed and deployed to an web-application. Again, I used WSPBuilder to do so.

Activating the workflows

To activate the workflows, create an empty site.I called mine, http://litwaredemo:15000/BeerWorkspaceTest/ Activate the “Beer Lists Feature” and the “Beer workflow” feature next. Please note that activating could take quite some time (up to 20 seconds for the first time). This should activate and enable the workflows on that site.

Testing the workflows

Add an item to the list “Beer Orders” with the name  “Heineken” and you will see that the workflow gets triggered and an message gets added to the history, and an item get put in the “Approved Beer Orders”

And you can see the item has appeared on the other list “Approved Beer Orders” as well.

 

Make it easier for your clients using feature stapling

Don’t forget that you can feature staple the features together so that the user only needs to activate one feature.

What’s next?

Well, next I will upload the utility, and I will create an screen-cast to explain how to work with the tool and export workflows. If this is still a bit too complicated to comprehend.

Its out now: Check the post http://agiledirect.wordpress.com/2008/09/11/beta-spd-workflow-export-utility-on-codeplex/

A couple of features on my wish list.

• Create a CodePlex place for it, so I can upload the code and you guys can download and work with the tool.
• Add support for versioning of workflows
• More advanced support for workflows
• Fix some bugs
• Use the object model API instead of the webservice (avoids the long wait when activating)

I’m back from Hawaii

Back from Hawaii, need to get a couple of things in order and then I will post part deux (2) on SPD workflows!

Packaging and re-using SharePoint designer workflows (Part 1)

Purpose of this article

The purpose of this article is to explain some of the inner workings of SharePoint designer workflows. It will also explain why SharePoint designer workflows are not portable by default. Additionally, in the upcoming blog there will be a tool available that allows for basic packaging of  SharePoint designer workflows to features, thus enabling SPD workflows to become a part of your solution packaging process.

SharePoint Designer workflow creation process explained

When you create a workflow in SPD. There is a lot of server-client interaction involved cleverly hidden from the end user. In a nutshell, this is what happens behind the scenes when a user creates a workflow in SharePoint designer.

1. User initiates the creation of a new workflow by choosing ‘New Workflow’ from the SPD menu.
2. SharePoint designer queries the remote server and downloads any workflow activities registered in authorized types (in proxy form) from the server using web services. SPD opens the workflow editor, and allows the user to create the workflow using the available activities on the server.
3. Upon save, SPD checks whether there is a Workflows library available in the site. If not it will create one. Additionally, SPD creates a folder in the Workflows library baring the same name of the workflow to store the files into.
4. SPD serializes the workflow to its XOML (WFName.xoml) equivalent, if any rules logic is used; it will also serialize this information to a (WFName.xoml.rules file).
5. SPD uploads the files into the folder using RPC.
6. After uploading the files it will validate the XOML for any errors by calling a web service method on the server passing the workflow XOML.
7. If the validation is passed it will create a configuration file that states which version of the workflow should (the actual version number of the xoml file) be used and to which list the workflow is associated. It will also specify workflow initiation conditions such as start on item creation, update or delete, and whether it should allow users to start workflow manually. This configuration file is uploaded to the server.
8. The configuration file is processed and the workflow is associated to the list by the server by SPD calling a method on an web service passing the URL and version of the configuration file.

Anatomy of an SharePoint Designer workflow

When you create and save a workflow in SharePoint designer, it stores a couple of files in a folder baring the same name of the workflow in the workflows library as mentioned in the process above.

 

Each workflow folder consists out of the following files;

The workflow XOML file (My Workflow On Test A.xoml)

The workflow XOML file contains the serialized workflow markup. This is the serialized XML/XOML version of the workflow you’ve edited in SharePoint designer. It basically contains a serialized activity graph of all the workflow activities used in your workflow.

   

Additional rules file (My Workflow On Test A.xoml.rules)

if you have used rules logic in your workflow; Sharepoint will create a rules file in the folder as well. This file will contain all custom rules logic generated by the rules engine in the SPD workflow editor.

A workflow configuration file (My Workflow On Test A.xoml.config)
The workflow configuration file stores information on which version of workflow and rules file it should use upon workflow association. It will also contain initiation information such whether the workflow should be started upon creation of an item in the associated list, or when an item is updated and so on. Furthermore, the configuration file will have an id of a list to which it should associate the workflow to.

The reason why SPD workflows cannot be ported

So actually, there is nothing in the workflow engine and SharePoint object model that prevents you from using the same workflow XOML over and over in different sites, and thus packaging up and porting your SPD created workflows. Frankly, I think it’s pretty decent model if it was documented well. The actual thing that makes it impossible to port SPD workflows is the OOB activities that come with SharePoint itself. These activities when serialized to XOML, store GUIDS to lists itself instead of ‘pointers’ to list, or any other indirection or that matter. And here is the weakness exposed, because List-IDS are generated each time a list is instantiated the XOML workflows aren’t reusable, since they will most likely point to non existing lists.

You can see an example of this on line 26, of the XOML file (Lookup activity)

Also the configuration is hard bound to the list id.

In my opinion it feels like the workflow activities where a bit rushed, an that the indirection layer that made portability possible has somehow not survived the PBI prioritization queue.

In part two..

Well, this should be enough to explain how SPD workflows work, for now, in the next post we will go deeper into the workings and I will release an sample tool that can be used to export and import basic SharePoint designer workflows to overcome this problem.

Watch this space!