• Home
  • Nativescript
  • Cross Platform Image Manipulation in Nativescript Using WebView Canvas

Cross Platform Image Manipulation in Nativescript Using WebView Canvas


Do you want to implement image manipulation in your Android/iOS app using HTML5 Canvas Api/Library? If yes, then you will find this tutorial helpful.  

I have created a plugin nativescript-canvas-interface, which makes this very easy to implement. This plugin enables you to use any canvas library out there on the web (excluding animation libraries), to do image manipulation and render that image in native image element in your Android/iOS application. The image manipulation is done on canvas element in web-view, and the generated image is transfered from web-view to native application.

This plugin is built upon nativescript-webview-interface.

To quickly get started, you can clone this demo app. In this demo app, I have used CamanJS to do image mainpulation. But, you can use any library of your choice.

Output of the demo appliaction is as shown below:

Android

nativescript-canvas-interface-android-demo

iOS

nativescript-webview-interface-ios-demo

Let’s understand how it works, step by step.

Step 1: Initialization of plugin

First we need to install the plugin by following the instruction at https://www.npmjs.com/package/nativescript-canvas-interface#installation..

After installing the plugin, let’s add web-view and image element to our page.
As web-view rendering performance is poor on some devices, it is good to keep it hidden and show the manipulated image in native Image element.

main-page.xml

<Page xmlns="http://schemas.nativescript.org/tns.xsd" loaded="pageLoaded">
....

<WebView id="webView" src="~/www/index.html" visibility="collapse"></WebView>
<GridLayout id="container" rows="auto" columns="*">
    <Image id="img" src="~/road-nature.jpg"/>    
</GridLayout>
....
</Page>

Now, initialize the plugin, in our main-page.ts file.

main-page.ts

var nsCanvasInterfaceModule = require('nativescript-canvas-interface');
var oNSCanvasInterface;

export function pageLoaded(args) {
    page = <Page>args.object;
    var webView = <WebView>page.getViewById('webView');
    initCanvasInterface(webView);
    ....
}

/**
 * Initializes canvas interface plugin.
 */
function initCanvasInterface(webView: WebView) {
    oNSCanvasInterface = new nsCanvasInterfaceModule.NativescriptCanvasInterface(webView, 'canvasEle');
    // 'canvasEle' is the id of canvas element in web-view
    ....
}

As shown in the above code, the plugin requires a web-view element and id of the canvas element.

Assuming we have copied the plugin files for web-view, by following installation instruction mentioned above, let’s import the plugin files in our index.html

www/index.html

<!doctype html>
<html>
    <body>
        <!--Canvas element on which image manipulation is performed -->
        <canvas id="canvasEle" data-caman-hidpi-disabled="true"></canvas>

        <!--Required -->
        <script src="./lib/es6-promise.min.js"></script>
        <script src="./lib/nativescript-webview-interface.js"></script>
        <script src="./lib/nativescript-canvas-interface.js"></script>

        <!--Optional. Any canvas library-->
        <script src="./lib/caman.full.min.js"></script>

        <!-- Your javascript file-->
        <script src="./index.js"></script>
    </body>
</html>

Let’s initialize canvas interface in web-view by creating instance of NSCanvasInterface global variable.

www/index.ts

(function(){
    /**
     * Initializes canvas inteface for communication between web-view canvas and native app.
     */
    function init() {
        var canvasEle = <HTMLCanvasElement>document.getElementById('canvasEle');
        var oCanvasInterface = new window.NSCanvasInterface(canvasEle);

        registerNSCanvasReqHandlers(oCanvasInterface); // explained later
    }

    init();    
})()

The plugin is now ready to do some cool stuff!!.

Step 2: Set Canvas Image from Native App

Now, to do the image manipulation, we need the image to be set on the canvas element.
We can send image via setImage api to web-view canvas in either of these three format :

  1. Image
  2. ImageSource
  3. Local Image Path

Let’s send the image, set in Image element, to the web-view on load of it.

main-page.ts

import {screen} from 'platform';
function initCanvasInterface(webView: WebView) {
    webView.on('loadFinished', (args) => {
        if (!args.error) {
            var width = screen.mainScreen.widthDIPs;
            var dimRatio = imageView.imageSource.width / imageView.imageSource.height;
            var height = width / dimRatio;
            oNSCanvasInterface.setImage('setCanvasImage', imageView, [{ width: width, height: height }]);
        }
    })    
}

Here, setCanvasImage is the function in web-view, which will set the image to canvas. You can pass the name of any function which can set the image. Note that, in third argument of setImage api call, we are sending image width and height as an extra argument. You can pass any data here.

Now, let’s create setCanvasImage function in web-view.

www\index.js

/**
 * Registers handlers to handle all the requests raised by native app.
 */
function registerNSCanvasReqHandlers(oCanvasInterface) {
    oCanvasInterface.canvasReqHandlers = {
        setCanvasImage: setCanvasImage,
        ....
    };
}

/**
 * Sets image with specified dimension sent from native app to canvas, for doing image manipulation on it.
 */    
function setCanvasImage(canvas: HTMLCanvasElement, ctx: CanvasRenderingContext2D, image: HTMLImageElement, config: any) {
    canvas.width = config.width;
    canvas.height = config.height;
    ctx.drawImage(image, 0, 0, config.width, config.height);
}

To execute any web-view function from native app, we need to register that function in canvasReqHandlers registery of the plugin. Each function registered to canvasReqHandlers are called with registered canvas element and its 2d context.

The functions called by setImage api, are also provided with HTMLImageElement and any extra data passed from native app. In this demo application, we are sending width and height of the image as the extra data and setting canvas dimensions as per that.

Step 3: Apply Filters to Image

Now comes the fun part. Let’s apply some fancy filters of CamanJS library.

main-page.xml

....
<Button cssClass="btn" text="Vintage" id="vintage" tap="setPreset" />
<Button cssClass="btn" text="sunrise" id="sunrise" tap="setPreset" />
<Button cssClass="btn" text="cross Process" id="crossProcess" tap="setPreset" />
....

main-page.ts

/**
 * Applies preset selected by user to the image. 
 */
export function setPreset(args) {
    var presetId = args.object.id;
    performCanvasMainpulation('setPreset', [presetId]);
}

/**
 * Performs image manipulation on canvas in webview, and renders the returned image in Image element.
 */
function performCanvasMainpulation(fnName: string, args?: any[]){
    imageView.animate({
        opacity: 0.5,
        duration: 150
    });
    oNSCanvasInterface.createImage(fnName, args).then(function(result: {data: any, image: ImageSource}) {
        imageView.imageSource = result.image;
        imageView.animate({
            opacity: 1,
            duration: 150
        });
    }, function(error){
        ....
    });
}

We are calling createImage api with the function to be executed in web-view, named setPreset. Notice that we can use nice fading animation effect, as we have the native Image element.

Now, let’s create setPreset function in web-view, which will apply filters to our image.

www\index.ts

/**
 * Registers handlers to handle all the requests raised by native app.
 */
function registerNSCanvasReqHandlers(oCanvasInterface) {
    oCanvasInterface.canvasReqHandlers = {
        ....
        setPreset: setPreset,
        ....
    };
}

/**
 * Sets the requested presetMode to the canvas using CamanJS library and returns promise.
 */    
function setPreset(canvas: HTMLCanvasElement, ctx: CanvasRenderingContext2D, presetMode: string) {
    currentPreset = presetMode;
    return new Promise(function(resolve) {
        // this is CamanJS library specific code. You can write any canvas manipulation code here.
        Caman(canvas, function() {
            this.revert(false);
            this[presetMode]();
            this.render(function() {
                resolve();
            });
        });
    });
}

canvas and ctx are passed by the plugin and presetMode is what we are sending as an extra data in createImage api call.

Note that, if the image manipulation is async, we need to return a promise. If it is not async, then you don’t need to return a promise. Just do the manipulation on canvas and return any data if you want or don’t return anything.

The plugin will create the image based on current canvas context, and pass the nativescript ImageSource object to the success handler of promise returned by createImage api. The resolved data of promise or the returned data from the function in web-view, is also passed to the success handler.

Now you have the image, ready with fancy filters. You can allow application users to download/share this image or to do anything as per your usecase.

You can refer Plugin API, for reference.

Hope, you will find this tutorial helpful. Enjoy!

  • Aaron Ullal

    Hi! This is a great tutorial! I have one problem…I am trying to manipulate images taken from camera. I’ve done the following :

    cameraModule.takePicture({height:pageHeight,keepAspectRatio: true}).then(function(picture){

    imageView.imageSource = picture;

    var width = platform.screen.mainScreen.widthDIPs;

    var dimRatio = imageView.imageSource.width / imageView.imageSource.height;

    var height = width / dimRatio;

    nsCanvasInterface.setImage(‘setCanvasImage’, imageView.imageSource, [{ width: width, height: height }]);

    });

    However whenever I call setPreset it doesn’t modify the picture.
    Am I doing something wrong?

Menu