Read the intro to this series here.

Ok, so by this stage I hope you have read Part 1 on Authentication.  Understanding how to authenticate with SharePoint from a mobile applications is a critical step before you can interact with it to get data in and out of it etc…

Ok, I have authenticated successfully … now what?

Generally most people want to go and get some data from SharePoint, like a set of data from a list or library for example.  SharePoint provides a plethora of APIs to work with data.  Options include SOAP Web Services, RSS feeds, client side object model, or the fronting the service side object model with a custom WCF web service of the link. Or finally the in built REST/OData endpoints.

The REST/OData endpoints, in my opinion, are the best and most flexible choice for most mobile scenarios.  Why:

• Simple HTTP calls, easy from every mobile platform
• JSON or XML responses
• Read and Write operations
• Easy filtering and query capabilities

“It’s what all the cool kids are doing!”

A lot of services like twitter and Facebook etc. provide REST APIs so generally it’s a well accepted method for working with a service from a mobile platform.  There is a lot of community support for working with these kinds of services from phone platforms too. 

On Windows Phone for example there are two libraries I use in nearly all my mobile projects:

• RestSharp – restsharp.org – For calling Rest services.
• JSON.Net – json.codeplex.com – For dealing with JSON responses

What do the SharePoint REST services offer?

SharePoints REST services follow (mostly) the OData protocol (odata.org).  I say mostly because there are some differences (what would a standard be without a few differences to keep us on our toes!) that I will highlight later on.

SharePoint 2010 provides REST/OData endpoints at the /_vti_bin/ListData.svc url.

e.g. http://MySharePointServer/SiteName/_vti_bin/ListData.svc

There are quite a few ways to get the data you want.  The simplest is just retrieving all the items from a list.

For example if we had a SharePoint list called “Tasks” then we could issue a GET query for the items in that list by calling:

http://MySharePointServer/SiteName/_vti_bin/ListData.svc/Tasks

As I said above, there are quite a few ways to query for data etc… The best place to read about all those is here: http://www.odata.org/documentation/uri-conventions 

However, I will mention a couple of good ones to get you started.  In particular $filter, $select

Get a single Task: 
/_vti_bin/ListData.svc/Tasks(1) <- where 1 is the id of the task

Filter for tasks with a title that starts with “Remember”:
/_vti_bin/ListData.svc/Tasks?$filter=startswith(Title,’Remember’)

Note:  startswith is just one operation. you can find others in the documentation here.

Just get the properties of a object that you want:

/_vti_bin/ListData.svc/Tasks?$select=Title

In this example I just wanted the Title of the Task, not all the other properties.  Being able to select just the data you want is really important in mobile apps given you want to keep the data payloads to the absolute minimum.  Selecting just the data you want really helps this.

I am making the request but I am getting XML back.  How do I request JSON?

JSON (JavaScript Object Notation) is a lightweight data format that is super easy for JavaScript to parse and work with & fortunately for Mobile developers has a lot of community support (like JSON.Net) for dealing with it in .Net applications like Windows Phone.  In fact there is good support for it across the major mobile platforms.

Lightweight is a critical word here.  Like I said above, its really critical to keep your data payloads small and fast.  Another way to assist with this is to use JSON.  It’s lighter than the XML equivalent & it compresses well too.  In mobile apps some people pay for data by the MB, so not only does it make it faster, but cheaper to use your app.

To request JSON data responses from SharePoint’s REST services you need to set the HTTP Accept header to “application/json”.

On Windows Phone using the HttpWebRequest object you do that in one line of code like this:

request.Accept = @”application/json”;

Once you get the JSON response you can use JSON.Net to deserialize it into strongly typed objects like so:

TasksResult tasks = JsonConvert.DeserializeObject<TasksResult>(responseString);

Note:  You need to generate your objects for deserialization.  You can hand craft those if you like or use a tool like http://json2csharp.com/ to help build your classes from a sample JSON response.

What about other stuff I can do via the REST services?

SharePoint not only supports reading of data via REST/OData, but also writing.  Instead of issuing a GET request to query for data we can use the some of the other HTTP verbs to write data. 

Here is a list of the verbs:

• POST – Create
• GET – Retrieve
• PUT – Update (all properties)
• DELETE – Delete
• MERGE – Update (only select properties)

Most of these are fairly self explanatory … however the PUT & MERGE are pretty similar.  The difference is that with PUT you have to send your updated object with ALL of the properties regardless of if you want to update them or not. Whereas MERGE allows you to only specify the properties you want to update.

So for example, if you wanted to Create a new Task in the Tasks list you would send a POST HTTP request to:

POST to http://MySharePointServer/SiteName/_vti_bin/ListData.svc/Tasks

With the body of the POST being the JSON representation of your new Task.  Again you can use JSON.Net to assist with desterilizing your Task object to JSON like this:

var json = JsonConvert.SerializeObject(myNewTaskObject);

To update an existing task you would send a MERGE request to:

http://MySharePointServer/SiteName/_vti_bin/ListData.svc/Tasks(X)

X = the ID of the task you want to update.  Along with the body of the request being the JSON representation of the Task.  You can omit the fields you don’t want to update with a MERGE to keep the payload size down.

How do I deal with someone else updating the list between when I get the list data and updating it?

This is called concurrency management and SharePoint gives you some help here …if you want it 🙂

What you really want to do is say .. I want to update this list item, but only if it has not changed since I retrieved it.

To do this SharePoint issues ETags as properties on objects when you retrieve them, which are effectively sequence numbers that change when an item is updated.  For example in an XML response you would see something that looks like this:

<entry m:ETag="W/&quot;2&quot;">

So when you do a PUT, MERGE or DELETE on an item you need to attach this ETag that you received. This basically says to SharePoint, only let me do this update only if the ETag hasn’t changed.

You add this to your request by adding another HTTP Header called “If-Match”.  For more information on the If-Match HTTP header have a read of section “14.24 If-Match” in the HTTP RFC here: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

e.g. header values:

If-Match: "W/”2”"
If-Match: * <= This says overwrite it regardless

If your ETag doesn’t match SharePoint will throw an error back in the response.  Your code should deal with this situation!  Typical options would be to offer the user the ability to merge the conflicts, overwrite regardless or discard their changes.  There is no one right answer in this situation & it will depend on your business rules for your app what you will need to do.

Summary

SharePoint offers a very mobile friendly API with the REST/OData endpoints.  Using these + requesting JSON data responses gives you a combination that has great community library support of creating and parsing requests/responses as well as being platform independent.  If offers CRUD (Create, Read, Update & Delete) operations on list and library data so covers the bases there.

+ REST/JSON are what all the cool kids are doing on services like Twitter/Facebook … so you won’t sour their SharePoint experience too much by trying to force them to use the long in the tooth SOAP web services instead 🙂

In the sample project you can find an example of requesting a list of Tasks from the default Task list in a team site via the REST with JSON responses.  Check MainPage.xaml.cs  in the LoadTasks() method.

DOWNLOAD Sample

In the next post in the series I will discuss some options on how you can make your SharePoint data available to your mobile apps when you leave the office via Azure and Service Bus.

Thanks,

-Chris.

As promised to those who attended here are my slides from the session:

Developing Mobile Applications for SharePoint

For those after the code samples, I will be including them in the Developing Mobile apps for SharePoint series.

It was a bit of fun and given I didn’t want to carry a massive digital document/phone projector to NZ and AU I improvised as well. 

Here is the recipe:

• 2 kilos of Lack of shame
• 1 HD Microsoft Web Cam
• Lots of ability to make fun of yourself
• 1 LifeCam software running on your laptop that is projected
• 1 Lumia 800 without video out capability for 3rd party apps

And you get ….. #johnsoncam

Photo Credit: @mrhodes

I’m sorry for all those who had to witness this act … but thanks to those who laughed.

-CJ.

Read the intro to this series here.

Have you ever tried hitting one of SharePoint’s APIs from your next million dollar phone app prototype and gotten a HTTP Error 403 Forbidden?

Chances are you are requesting something from SharePoint and its just a bog standard site configured with Windows Auth (NTLM).

On Windows Phone NTLM support isn’t baked into the platform for 3rd party apps.

Note:  I hope Microsoft adds support for NTLM in a later release. Currently the Office Hub can connect and authenticate with SharePoint so long as the site address doesn’t include a “.” in the address.  e.g. http://myserver/ will work, but http://myserver.contoso.com won’t work.  Apparently a “.” in the name makes the stack put it in the “Internet” zone which doesn’t support NTLM.  Anyway, I digress.

As far as I know NTLM support out of the box on the “other” mobile platforms like iOS and Android is also limited.

All this makes authenticating with SharePoint a bit more painful than you might initially think.  However, if you think about it this limitation it is only really a pain if you are building an app that will only ever be used internally on the same network as SharePoint, rather than externally over the internet since NTLM is no good over the internet.  I can think of lots of examples of apps that this would be the case, but many many more where you would want them to connect to SharePoint no matter if you were on the network or out on the internet.

So given NTLM is almost immediately off the table what options does that leave you with?

• Unified Access Gateway (UAG) in front of SharePoint
• Basic Authentication (Auth HTTP header based)
• Forms Authentication

UAG has a feature that lets you configure it to translate Forms Auth to NTLM.  Its effectively logs in on your behalf using the credentials you provide.  Its pretty cool, but unfortunately is another bit of software you need.  If it is an option you are interested in you can read more in this whitepaper: http://technet.microsoft.com/en-us/library/hh180841.aspx

Basic Auth makes things a bit easier.  You need to base64 encode the username and password and stuff it in the Authorization HTTP header. If you do want to use this method you will need to make sure you connect over SSL so you don’t disclose your username and password.

Forms Authentication.  The reason this is my personal preference is that it is typically the option that people use when exposing SharePoint sites for people access over the internet.  For example when you want to invite a partner organization to a SharePoint site you host.  In this situation, if you already have Forms set up then its pretty straightforward to connect from a phone app.

Note:  For a fantastic guide to setting up Forms Auth you should read Mirjam’s excellent blog posts:  Configuring claims and forms based authentication for use with a SQL provider in SharePoint 2010 and/or Configuring claims and forms based authentication for use with an LDAP provider in SharePoint 2010

So how do you programmatically authenticate with a Forms Auth secured site?

SharePoint 2007 and 2010 provide an Authentication SOAP webservice called authentication.asmx that lets you pass a username and password and return a set of authentication cookies that you can attach to future requests.

You can find the auth web service at http://<sharepoint site url>/_vti_bin/authentication.asmx

To ensure you keep a handle on the cookies that come back as part of the response there are a couple of things you should do first before issuing the request.

Note:  Full code is available via the link at the end of this post.  The code contained in the article are only snippets.  Please see the full source.

(Windows Phone specific code) 1st,  create a new CookieContainer and attach it to your request.  When the response comes back your container will be filled with the right authentication cookies:

private CookieContainer cookieJar = new CookieContainer();

2nd, set up the headers to ensure your request is interpreted correctly & attach the CookieContainer your created:

System.Uri authServiceUri = new Uri(AuthenticationServiceURL);
HttpWebRequest spAuthReq = HttpWebRequest.Create(authServiceUri) as HttpWebRequest;
spAuthReq.CookieContainer = cookieJar;
spAuthReq.Headers["SOAPAction"] = "http://schemas.microsoft.com/sharepoint/soap/Login";
spAuthReq.ContentType = "text/xml; charset=utf-8";
spAuthReq.Method = "POST";

Simply POST a SOAP message with your username and password in it. The SOAP message looks like this:

            string envelope =
                    @"<?xml version=""1.0"" encoding=""utf-8""?> <soap:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/""> <soap:Body> <Login xmlns=""http://schemas.microsoft.com/sharepoint/soap/""> <username>{0}</username> <password>{1}</password> </Login> </soap:Body> </soap:Envelope>";

When the response comes back from SharePoint you should see a response returned as follows:

<?xml version=”1.0″ encoding=”utf-8″?><soap:Envelope xmlns:soap=”http://schemas.xmlsoap.org/soap/envelope/” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”>
<soap:Body>
<LoginResponse xmlns=”http://schemas.microsoft.com/sharepoint/soap/”>
<LoginResult>
<CookieName>FedAuth</CookieName>
<ErrorCode>NoError</ErrorCode>
<TimeoutSeconds>1800</TimeoutSeconds>
</LoginResult>
</LoginResponse>
</soap:Body>
</soap:Envelope>

The LoginResult bits are the interesting parts.  The CookieName tells you the name of the authentication cookie that was issued & ErrorCode tells you if everything worked as expected.

Note: On Windows Phone (7 & 7.5 at least) if you look in the CookieContainer you will NOT see any cookies returned.  Don’t let this worry you.  They are in there, but they are marked HttpOnly and wont show up in the CookieContainer.  Don’t belive me?  Take a look with Fiddler 🙂

Ok, so now you have authenticated and you want to get/set some data.  What next?

The really important part is setting those cookies when you make your next request.  Simply take that CookieContainer you attached to the authentication request, and reattach it to your next request to a SharePoint API. e.g.

System.Net.HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.CookieContainer = App.CookieJar;
request.Accept = @"application/json";

request.Method = "GET";
request.BeginGetResponse(new AsyncCallback(GetTasksCallBack), request);

The critical line above is:

request.CookieContainer = App.CookieJar;

This will ensure the “FedAuth” cookie from the authentication process is passed to SharePoint when the request for data is made, and SharePoint will know who you are etc…

In the attached sample code I have wrapped up all the authentication code into a helper class called “Authorization”.  You can use it like so:

auth = new Authorization(Constants.USER_NAME, Constants.USER_PASSWORD, Constants.AUTHENTICATION_SERVICE_URL);
auth.OnAuthenticated += auth_OnAuthenticated;
auth.Authenticate();

Then in the event handler “auth_OnAuthenticated” you can grab the cookie container like:

void auth_OnAuthenticated(object sender, AuthenticatedEventArgs e)
{
     App.CookieJar = e.CookieJar;
}

The sample below is a Windows Phone 7.5 application called “SPMobile”.  It contains the Authorization class mentioned above in the /Auth directory.  Additionally it requests data from the default Tasks list on a Team site using REST and JSON.  The sample illustrates how you can authenticate and then subsequently attach the cookies to your data requests to SharePoint.

The username, password & paths to your SharePoint site should be set in the Constants.cs file accordingly.

DOWNLOAD Sample

In the next part of this series I will post about requesting and interacting with data via SharePoint’s REST services in a mobile friendly manner.

Thanks,

-Chris.

Update:  Fixed all the fat finger “NTML” spelling, should have been “NTLM”.

At the recent New Zealand and Australia SharePoint Conferences I ran a session on what you need to know to build mobile apps for SharePoint.

Topics I covered included:

• Mobile SharePoint options
  • Out of the box functionality
  • 3rd party extensions options
  • Existing Native App options on Windows Phone and iPad
• Authentication with SharePoint
  • Programmatically using Forms authentication from a phone
  • UAG options (Forms to NTLM)
• Accessing SharePoint On Prem
  • Azure Service Bus Relay
• APIs and Data
  • API Options
  • REST and JSON support in SharePoint 2010

To follow up from these sessions I have decided to start a series of posts on some of these topics to help spread the word and hopefully help people get started.

In particular in this series I want to cover off the more developer centric topics from this talk about:

• Programmatic forms authentication with SharePoint
• Azure Service Bus relay options to get access to on prem SharePoint from a device on the internet
• Securing Service Bus relay using ACS

Part 1 – will cover how to authenticate with SharePoint using Forms authentication.  Why forms?  Well most Mobile platforms don’t support NTLM authentication out of the box (Windows Phone included) and so another authentication system is required.  Also NTLM wont work over the internet which is where most mobile devices will be connecting from.

Part 2 – will cover using REST and JSON to push and pull data in and out of SharePoint from your mobile app.

Part 3 – will cover using Azure Service Bus to assist with connecting to on-prem SharePoint environments from a device on the internet.  All without deploying additional hardware/software to your DMZ.  I’m just calling it out as an option since its pretty nifty.

Part 4 – securing your service bus connections to ensure only people you allow can connect to your data.

All the posts will include full source and instructions so you can get cracking and try some of these things out for yourselves.

Stay tuned for Part 1 in the next few days.

-Chris.