Friday, August 21, 2009

TFS Admin, Part III - POC Extending TFS Warehouse

If I had a dollar for every time I wished for the capability to configure and store metadata about team project in TFS, I could at least take a very long vacation. Using a custom list in SharePoint to keep track of team projects and their metadata is the best solution I've come up with so far, especially if you use it as a request queue. But it´s still a non integrated solution causing manually work if you want to do reporting or anything useful with it.

Extending TFS Warehouse with metadata
Having metadata in a SharePoint list, which I could write code against, and knowing that the TFS warehouse is extendable gave me the idea to feed the TFS warehouse with team project metadata. For this to work I need the team project names to be the same in both the SharePoint List and TFS. This should not be a big issue for anyone, especially if you use the SharePoint list as a request queue and automatically create team project as described in previous post.
My plan was to write a generic adapter for the TFS warehouse, extending the Team Project dimension in the warehouse with selected fields from the SharePoint list.

Writing a custom TFS Warehouse adapter
My plan was to write a generic adapter for the TFS warehouse, extending the Team Project dimension in the warehouse with selected fields from the SharePoint list.
After some research on MSDN I was ready to start. A custom adapter is simply a class implementing the 3 methods in IWareHouseAdapter, compiled into an assembly and deployed to a special folder. To implement the IWarehouseAdapter interface you have to write code to make
• Make Schema Changes to the TFS Warehouse and cube
• Make Data Changes, i.e. create rows in the database
• Handle Initializing, request to stop processing
The TFS API provides an object model for the warehouse, together with some good articles on msdn and Steve Wrights posts How I built a Team Foundation Server custom data warehouse adapter made it pretty easy writing the adapter and deploying it to the server.

Retrospective - the real challenges
In retrospective I can see that the biggest challenges had nothing to do with writing the adapter or handling a SharePoint custom list. Looking back I spent more time doing other stuff then writing the logic for the adapter. The other things taking most time from the real task were:

Testing and the test environment
Running the test environment (a virtual PC image of o complete TFS setup) locally on my old and slow laptop were not funny. In order to save time, I tried reinitializing the TFS warehouse between tests. But this took too long time, so I ended up with reverting the virtual my machine and setting up my environment for every test run. Maybe a better design/separation and mocking could have saved time in the end.

Trying to make the Warehouse over complex
My lack of real world experience from BI projects took me out on long journey trying to make a too complex warehouse in order to handle the metadata in a generic way. After many hours trying to implement the advanced model with the TFS API I consulted a colleague with real life experienced of BI project and went for a really simple design instead.

Handling configuration for the adapter dll
Needing to store some configuration and have it easily handled I thought that using a .config file for the dll would be the best fit. I simply added settings to the adapter project and used the Properties. Settings class. Things worked nice until I tried changing the value in the .dll.config file. As I was afraid of it didn’t work. After hours of fruitless search for a way to get the framework to actually read my config file, I gave up and derived a custom setting provider from LocalFileSettingsProvider, override the GetPropertyValue method reading the config file as an XML document. Once I decided to write my own settings provider was I was done in 15 minutes, but I wasted several hours looking for a better solution.

Reducing technical dept
As I needed to wrap a SharePoint list in this project to I started to break out the SharePointListWrapper into a utility project. One thing lead to another and in the end I ended up fixing some of the code for the Automated Order Process to.
Doing so I repackage the source for The Automated Order Process, although the code is in better shape, it is like the TeamProejctMetadataAdapter still a proof of concept.


http://cid-5d46cae8c0008cf0.skydrive.live.com/self.aspx/.Public/POC%20TeamProjectMetadataAdapter.zip

Wednesday, August 12, 2009

TFS Admin, Part II - POC Automated Order process

Update: TFS2010 Automated team project creation

In this post we will look at a possible solution covering the team project order process. The solution covers the whole process from order entry to delivery notification and is based upon the requirements listed in the previous post.

A good start, but a long way to go
This is just a proof of concept, a quick hack to prove it’s possible to accomplish an automated team project order process. It's a long easy to go to a releasable product or project with error handling, tracing, packaging and deployment and so on. The best would be to get something like this integrated into the TFS product or into the TFS Admin tool


Architecture and design
One distinctive requirement is the configuration, collection and storage of metadata. As we already have a working instance of WSS, it falls naturally to use its capability of custom lists to handle metadata.

In order to make team project creation fully automated after approval, we would have to programmatically create new team projects. This capability is provided in the latest power tool command line util.
Managing user rights for SSRS, WSS and TFS is also a hard requirement to manage. Fortunately we have the TFS admin tool on Codeplex providing sourcecode for handling this requirement.
Of course some coding and usage of the TFS API is needed to stitch it together.

Creating the TeamProject list
In order to create the teamproject list and the order form I created a custom list at the root sharepoint site of my tfsserver, naming it to TeamProjects.

To this list I added core field as follow. I renamed the Title field to Project Name, added a people column named Project Administrator. I also added a choice column named Process Template with the exact process templates strings.

Under List versioning settings I activated Content Approval enabling a approval workflow.
I also added a view PendingCreation with the filter ApprovalStatus=Approved and TeamProjectCreated=No.

To make it easily accessible I added Quick Launch links to the list as well as to the TeamProjects: New Item form.

The result is a new form for requsting a team project as illustrated to the right.

Automating project creation puting it together
Now we need some code reading the list and creating team projects, giving the project administrator access rights, reassigning the workitems to the project administrator, applying enterprise standard access rights and policies. For the purpose of demonstrating how this can be done, I've put together a a simple solution. The complete solution is attatched as a ZIP file.


http://cid-5d46cae8c0008cf0.skydrive.live.com/self.aspx/.Public/POC%20AutomatedTeamProjectCreation.zip


Code walkthrough
I will also go through the code briefly starting with the main function holding it all together.

static void CreatePendingTeamProject()
{
string serverName = "http://teamsystem.sogeti.se";
string tfsServerPort = ":8080";
string tfsServerUrl = serverName + tfsServerPort;

string listName="TeamProjects";
string viewName="PendingCreation";

string projectNameCol = "LinkTitle";
string projectAdminCol="Project Admin";
string procTemplateCol = "Project template";
string projectCreatedCol = "TeamProjectCreated";

//Fetch an entry for a requested team project form sharepoint list
SharePointListWrapper spReqListWrapper = new SharePointListWrapper(serverName, "", listName, viewName);
ReqForTeamProject reqTP = spReqListWrapper.GetRequestForTeamProject(projectNameCol,procTemplateCol, projectAdminCol);

I have wrapped all access to the Sharepoint list in a special SharePointListWrapper class. The interesting things is in this method


public ReqForTeamProject GetRequestForTeamProject(string projectNameCol, string templateCol, string projectAdminCol)
{
SPSite site = new SPSite(ServerUrl);
SPWeb web = site.AllWebs[RelativeSiteUrl];
SPList myList = web.Lists[ListName];
SPView view = myList.Views[ViewName];
Console.WriteLine("Connected to sharepoint list");

if (myList.GetItems(view).Count > 0)
{
Console.WriteLine("Found a item");

SPListItem itm = myList.GetItems(view)[0];
ReqForTeamProject tp = new ReqForTeamProject();

tp.Id = itm.ID;
tp.Name = itm.GetFormattedValue(projectNameCol); //Project Name, Forced to use this cause Title is renamed.
tp.ProcTemplate = itm.GetFormattedValue(templateCol);
tp.ProjectAdmin = ((SPFieldUserValue)itm.Fields[projectAdminCol]
.GetFieldValue(itm[projectAdminCol].ToString()))
.User.LoginName; ;

Console.WriteLine(tp.ProcTemplate);
Console.WriteLine("Returning a Request for teamproject user " + tp.ProjectAdmin);

return tp;
}
else
{
return null;
}
}
The code opens a SharePoint list, and the view PendingCreation, select the first item if any. As you can see, I’ve used the Microsoft.SharePoint library for access to the SharePoint list. This will require the solution to be deployed at the server running WSS. If you want to deploy it elsewhere you have to use the WSS web services instead.

Back to the main function we check if we got a request to create teamproject.

if (reqTP != null)
{
//Create the Team Project
TFPTWrapper tftp = new TFPTWrapper(tfsServerUrl);
tftp.CreateTeamProject(reqTP.Name, reqTP.ProcTemplate);

The next thing in the code is the actual teamproject creation. This is done by calling the TFPT command line utility. Calling the command line utility is wrapped in the TFPTWrapper class, and the method CreateTeamProject simply builds a command line and execute it by starting a new process.

After the team project is successfully created, we need to give the project administrator access rights.

// Add rights to the ProjectAdmin
TFSAdminTooolWrapper admin = new TFSAdminTooolWrapper(tfsServerUrl);
admin.AddProjectAdmin(reqTP.ProjectAdmin, reqTP.Name);

To set the actual roles to TFS, WSS and RS we rely on the TFS Admintool. I have chosen to wrap a developing branch of the TFS AdminTool. The reason for this is that this branch has separated the code handling the actual tfs, WSS and RS servers into a separate project. making it possible to reference the dll file and not include the whole source for the TFS AdminTool.

This reduces the handing of setting user rights to this function

public void AddUser(string user, string teamProject, string tfsRole, string spRole, string rsRole)
{
TFSAdministrationTool.Proxy.Common.TfsAdminToolTracer.Initialize(null);
Console.WriteLine("Adding user with admin tool " + user);

TFSAdministrationTool.Proxy.ITeamFoundationServerProxy iTFSProxy = TFSAdministrationTool.Proxy.TeamFoundationServerProxyFactory.CreateProxy();
iTFSProxy.Connect(TFSServerUrl);
Console.WriteLine("Admin tool connected ");

iTFSProxy.SelectTeamProject(teamProject);
Console.WriteLine("Team project selected");

iTFSProxy.AddUserToRole(user, tfsRole);
Console.WriteLine("Added TFS role " +tfsRole);

iTFSProxy.SharePointAddUserToRole(user, spRole);
Console.WriteLine("Added WSS role " + spRole);

iTFSProxy.ReportingServiceAddUserToRole(user, rsRole);
Console.WriteLine("Added RS role " + rsRole);
}


public void AddProjectAdmin(string user, string teamProject)
{
AddUser(user, teamProject, "Project Administrators", "Full Control","Content Manager");

}


}

After adding the project administrator we need to setup standard access. In my case I made a xml config file containing the user, TFS, WSS & RS role to add. the users and roles are applied using the same TFSAdminToolWrapper.


//Setup standard team project security
StandardRightsXml stdRightXml = new StandardRightsXml("stdTeamProjectSecurity.xml");
foreach (UserRights stdUsr in stdRightXml.GetTeamProjectRights())
{
admin.AddUser(stdUsr.User, reqTP.Name, stdUsr.TfsRole, stdUsr.SpRole, stdUsr.RsRole);
}

All work items assigned to the creator of the teamproject must be reassigned to the new project administrator.

// Reassign all workitems to the new owner
TFSWorkItemStoreWrapper tfsWIStore = new TFSWorkItemStoreWrapper(tfsServerUrl);
tfsWIStore.ReAssignWI(reqTP.Name, reqTP.ProjectAdmin);

This is done through the TFS WorkitemStore API. Executing a query, opening the work items, and changing the AssignedTo field. This is wrapped into the TFSWorkItemStoreWrapper class.


public void ReAssignWI(string teamProject, string newOwner)
{
WorkItemStore wiStore = new WorkItemStore(TFSServerUrl);

WorkItemCollection wiLst = wiStore.Query(
" SELECT [System.Id], [System.AssignedTo], [System.Title] " +
" FROM WorkItems " +
" WHERE [System.TeamProject] = '" + teamProject + "' ORDER BY [System.WorkItemType], [System.Id]");

foreach (WorkItem itm in wiLst)
{
itm.Open();
itm.Fields["System.AssignedTo"].Value = newOwner;
itm.Save();
}
}
We also need to apply standard check-in policies to our new teamproject. I've made a xml config file containing the list of standard policies to apply to all project.

//Setup standard team project Policies
StandardPoliciesXml stdPoliciesXml = new StandardPoliciesXml("stdTeamProjectPolicies.xml");
TFSPolicyWrapper tfsPolicyStore = new TFSPolicyWrapper(tfsServerUrl);
tfsPolicyStore.AddCheckinPolicies(reqTP.Name, stdPoliciesXml.GetTeamProjectPolicies());

The code for applying policies is based on a postby Jakob Ehn and wrapped into the TFSPolicyWrapper.


public void AddCheckinPolicies(string teamProject, List StdPolicyLst)
{
List policiesToApply = new List();

TeamFoundationServer tfs = new TeamFoundationServer(TFSServerUrl);
VersionControlServer srvVC = (VersionControlServer)tfs.GetService(typeof(VersionControlServer));
TeamProject tp = srvVC.GetTeamProject(teamProject);

policiesToApply.AddRange(tp.GetCheckinPolicies());

foreach (TpPolicy WantedPolicy in StdPolicyLst)
{
Assembly policyAssembly = Assembly.LoadFile(WantedPolicy.PolicyAssembly);
object o = policyAssembly.CreateInstance(WantedPolicy.PolicyType);

if (o is IPolicyDefinition)
{
IPolicyDefinition def = o as IPolicyDefinition;

PolicyEnvelope[] checkinPolicies = new PolicyEnvelope[1];
bool foundPolicy = false;
foreach (PolicyType policyType in Workstation.Current.InstalledPolicyTypes)
{
if (policyType.Name == def.Type)
{
policiesToApply.Add(new PolicyEnvelope(def, policyType));
foundPolicy = true;
}
}
if (!foundPolicy)
{
throw new ApplicationException(String.Format("The policy {0} is not registered on this machine", def.Type));
}
}
else
{
throw new ApplicationException(String.Format("Type {0} in assembly {1} does not implement the IPolicyDefinition interface", WantedPolicy.PolicyType, WantedPolicy.PolicyAssembly));
}
}

if (policiesToApply.Count > 0)
{
tp.SetCheckinPolicies(policiesToApply.ToArray());
}
tp = null;
srvVC = null;
tfs.Dispose();
tfs = null;
}

In order to apply a policy to a project we must be able to load the policy. This will require all policies to be installed on the machine executing the program.

The last thing to do is to mark the request for a new teamproject as finished. This is done by updating the list item.


//Update the item
spReqListWrapper.UpdateRequestForTeamProject(reqTP,projectCreatedCol);
}

Monday, August 3, 2009

TFS Admin, Part I - Requirements for Team project creation

After installing TFS, most organizations sooner or later comes across many questions how to handle team system administration in their organization. In this post I will take a look at one of the first questions, How to handle Team Project Creation. Unfortunately this is one of the dark spots in case of functionality in the team system package. I will look at the requirements for team project creation process based on my experience from different organizations.


Stakeholders
I will start by looking at different stakeholders to the process of team project administration.


The first and obvious stakeholder is the project manager or “customer” who needs a project environment for the team. In many organizations there simply isn’t any process for ordering a new team project. From the project manager perspective it is important that the process for ordering a new team project is clear and simple resulting in a ready to go environment.
The management needs to keep track of existing project and systems, requiring lots of metadata like customer or business area, process and/or system owner. This require the ability to define and store metadata or/and make connection to some external tool for handling metadata. It also requires the metadata to be reportable.
The architectural function is responsible for standards and policies within the organization. Today all efforts to establish and maintaining a mandatory list of policies have to be done manually. The architectural function needs to be able to define the list of policies, and have them automatically deployed to new team projects.
The TFS admin crew is obvious a stakeholder for team project creation. TFS Admin also requires meta data such as project owners, administrators, cost centers and so on. Today creation of team projects is a manual process handling gathering, creation and customization of metadata, policies and reassigning work items and user rights. The whole process should be automated to avoid unnecessary work and errors.
The stakeholders listed above is/or should always be involved in the team project creation process. In addition to this, other stakeholders such as IT Operations for setting up project , staging and production environments, and License managers for handling license could be involved in the process, requiring the process itself to be easily extendable and customizable.

The list of requirements
The functionality could be divided into user stories or use cases and described like this. In addition to the requirements listed there is a n general requirement to be able to extend and customize the process easily.

Defining organizations standard
• Easy configuration of project metadata.
• Configuration of organization standard user rights
• Configuration of organization standard check-in policies

Ordering team project
• Easy accessible ordering form
• Collection of core values (Project name, administrator, process template, description)
• Collection of project metadata
• Approval workflow

Automated team project creation
• project creation
• reassigning of work items
• Applying user rights for project administrator
• Applying standard user rights.
• Applying standard check-in policies
• Notifying project administrator upon completion

Reporting
• Project catalog
• Extending project metadata to the TFS warehouse