Sunday, July 24, 2016

Modern JavaScript in SharePoint (4) - Gulp and PnP

PnP is the Office 365 Developer Patterns and Practices program - a (very) comprehensive resource illustrating techniques for coding against SharePoint and Office 365. Amongst the many offerings is the JavaScript Core Library. This library wraps the calls to the SharePoint REST API in a set of objects (or, to be accurate, in a series of namespaces and functions that mimic an object collection - this is ES5 JavaScript, after all!). The addition of this layer of abstraction simplifies many of the operations that are possible from client-side code.

For example, submitting a search against SharePoint becomes as simple in JavaScript as this:{ Querytext: "Document", RowLimit: 5 })
 .then(function(data) { console.log(JSON.stringify(data)); })
 .catch(function(e) { console.log("Error: " + e.message); });
The calls in the library handle the asynchronous nature of the interactions - hence the "then" and catch" functions for processing the retrieved data or any error that occurs.

How can this library be incorporated into a build workflow using ES6/ES2015 JavaScript in Visual Studio Code? In my trials, I wanted to integrate the build and deployment process using gulp, so first I investigated options for upload files to SharePoint Online within a gulp task. And happily there is a gulp plugin ready to achieve that - its called gulp-spsave. It is very simple to use, with the only disadvantage being that the credentials used to connect to Office 365 are supplied as plain text within the call. An alternative is to use gulp-spsync by Wictor Wilen - but for the purposes of these tests I have stuck with the spsave plugin.

This allows the upload of files into the site assets library in SharePoint to be achieved by:
gulp.task('upload-sp', function () {
    return gulp.src(config.bundleFilePath)
            username: settings.username,
            password: settings.password,
            siteUrl: settings.siteUrl,
            folder: "SiteAssets",
            notification: true
In this task, the account details and the target site are stored in a separate settings JSON file. I also have a separate similar task to upload libraries, such as PnP and any other dependencies.

Compiling PnP in Typescript in Visual Studio Code

So, now to use PnP in JavaScript in Visual Studio Code. First step is to add the module into the code's folder structure using the npm call on the command line (in the useful integrated terminal window):
npm install --save-dev sp-pnp-js
This adds a folder into the node_modules directory tree that includes the pnp.js library.

Next, the PnP library needs to be imported into the JavaScript module in which calls will be made using the library. In ES6/ES2015 this is achieved by the line
import * as pnp from "sp-pnp-js";
That seemed simple. But its not quite that easy. When I added a PnP call in my code in Typescript, and attempted to compile it using the Typescript compiler, I received various errors. Visual Studio Code started giving several naming errors. The TypeScript compiler was looking into PnP.js and finding lots of names it did not recognize.

At run time in the browser, these names would be available in the global namespace as the PnP library is built knowing that various other JS libraries are always referenced from SharePoint pages. But the point of using strongly typed JavaScript is to check its validity at compile time, so the compiler needs to know about all names used in all the current scripts. So I needed to supply the compiler with references for all names used in PnP.js. Experimentation lead to the following inclusions for overcoming the compile errors:
  • PnP is dependent on ES6 Promise and on whatwg fetch at runtime, so the type definitions for these libraries are needed. As an example, a compile error may complain of "promise" being an unknown name. These definitions can be added to the source code in VS Code via the following statements which get the necessary files from the DefinitelyTyped online resource:
    typings install dt~es6-promise --global --save
    typings install dt~whatwg-fetch --global --save
  • PnP also makes considerable use of the Microsoft Ajax library. I could not find a command line statement with which to add this type, so ended up manually getting the Microsoft-ajax.d.ts file from a GitHub location. I then manually added a sub-folder under the typings globals folder structure, and included a reference to this in the index.d.ts file under typings.
  • Last name correction was to add SharePoint-specific name declarations. For these, I found a file SharePoint.d.ts which I also manually added to the typings in this source.
This took a while, but finally managed to get the Typescript compiler to accept PnP!

Bundling The JavaScript for Deployment

Once the code could be compiled into an ES5 file, the ES5 then was ready for use on a test SharePoint page. Or was it? Looking at the file created using browserify in a gulp task, it was a huge file, as it included all of the PnP library. Clearly this is not what we want (as the PnP.js minimized library would be referenced from the page), so needed to tell browserify to exclude that library. This is done using the "external()" command:
    return browserify({
            entries: config.sourceJS + 'spTests.js',
            debug: true //This provides sourcemapping
        .on('error', console.error.bind(console))
        .pipe(source(config.bundleFile)) // Define the name of the bundle
        .pipe(gulp.dest(config.tsOutputPath)); // Destination for the bundle
In this gulp task, the compiled file "sptests.js" is being prepared for upload - the reason I am taking this approach is to be able bundle multiple modules into a single file. The second command in the pipeline tells browserify not to include the pnp.js library in the bundled output. Great, the completed file is now much smaller.

Using the file in SharePoint

The bundled file is uploaded to SharePoint using gulp-spsave, along with the libraries on which it is dependent. It nearly ran in the test page, but showed an error - the name "pnp" was not recognized in the script available on the page. Looking at the PnP library, it exposes "$pnp" rather than "pnp". So the gulp process needed to somehow map all calls to pnp to actually use $pnp.
browserify-sim to the rescue. Adding the following to the package.json file, and running the gulp tasks again, gave a bundled ES5 file that successfully uses PnP calls!
  "browserify": {
    "transform": [
  "browserify-shim": {
    "sp-pnp-js": "global:$pnp"

Thoughts on the Process

As you can see, writing code in Visual Studio Code in Typescript against PnP and getting that code to run in a SharePoint page is at the moment a mission. I am sure it will get easier.... If you need any tips about the concepts here, please drop leave a comment & I'll be in touch.

No comments: