Development Guide

To Bookmark this document, click

BOOK MARK 30

Creating a Convergence Application

Published 2014-09-15 | Convergence Tutorial. (Compatible with SDK 3.5,4.5,5.0,5.1 and 2012,2013,2014 models)

Tutorial describing how to develop a Samsung Smart TV Convergence Application allowing a TV to interact and communicate with external devices such as tablets, smartphones, laptops, desktops or any other smart device that is able to use UPnP discovery and HTTP protocol.

A Smart TV convergence application integrates Samsung Smart TVs and one or more external smart devices, such as a smart phone, tablet or any smart device that discovers Samsung Smart TV using Universal Plug and Play (UPnP) discovery and communicates with a TV over the HTTP protocol.

This tutorial demonstrates the use of convergence protocol for handheld phone (HHP) devices and TV device interaction by developing a sample application. The tutorial gives an overview of key concepts of developing a convergence application. The sample TV application (Canvas_NewOCI) provides drawing functions on a canvas based on messages received from the client application. The HTTP client application used in this tutorial is a JavaScript client application running on Google Chrome Web Browser. It emulates HHP behavior and is capable of sending and receiving basic messages to and from the TV application.

This tutorial consists of 3 parts:

  1. TV Application as Part of a Convergence Application Demonstrates the creation of a Smart TV application that can interact with an external device.
  2. HTTP Client Application Demonstrates the functions written in the HTTP client application so that the application can discover and then send and receive messages to and from a Smart TV application.
  3. Convergence Application Use Case Describes the functionality of a sample convergence application.

To learn more about the convergence features, see Convergence App and Convergence App API.

Prerequisites

To create applications that run on a TV screen, you need:

  • Samsung TV connected to the Internet
  • SDK or a text editor for creating HTML, JavaScript and CSS files (using Samsung Smart TV SDK is recommended)

Development Environment

Use Samsung Smart TV SDK to create the application. You can also use the emulator provided with the SDK to debug and test the application before uploading it in your TV. Later, you can run the application on a TV, see Testing Your Application on a TV. Note that applications may perform better on the TV than on the emulator. You can find general instructions for creating applications in Implementing Your Application Code.

Source Files

Note

The files needed for the sample application are here.

The client application of sample application are here.

This client application should be run in the Chrome browser.

The directory structure of the application:

File/DirectoryDescription
cssA folder containing CSS files.
jsA folder containing the NserviceSample.js Javascript file.
Config.xmlContains the environment to run the application on TV.
Index.htmlDescription of the elements display.
widget.infoUsed for the widget body opacity adjustment.
NserviceSample.jsWidget sample file.

TV Application as Part of a Convergence Application

This tutorial task demonstrates the creation of a Smart TV application that interacts with an external device application forming a convergence application. The task consists of the following parts:

Including a NServiceDevice Plugin

In index.html file, add the following code with plugin references.

<head>
    ...
    <script type="text/javascript" src="$MANAGER_WIDGET/Common/API/CImageViewer.js"></script>
    <script type="text/javascript" src="$MANAGER_WIDGET/Common/API/TVKeyValue.js"></script>
    <script type="text/javascript" src="$MANAGER_WIDGET/Common/API/Widget.js"></script>
    <script type="text/javascript" src="$MANAGER_WIDGET/Common/webapi/1.0/webapis.js"></script>
    ...
</head>

Loading the TV Application

On startup, the TV application initiates functions described in Main.onLoad() function, starts Nservice interface API, initiates all common objects and tries to get any devices already connected.

Add the following code to NserviceSample.js:

// assign namespace for convenience
var nservice = window.webapis.nservice || {},
    Main = {},

    gWidgetAPI,
    gTVKey,

    gnNserviceDeviceID = -1,    // Device id
    gsDevName = "",
    gsInputMsg = "no input yet";

Main.onLoad = function () {
    alert("[Nservice Tutorial]: Tutorial_onLoad()");

    var saDevInfo = null,
        nDevNum = 0,
        i = 0;

    gWidgetAPI = new Common.API.Widget();    // create Common module
    gTVKey = new Common.API.TVKeyValue();
    gWidgetAPI.sendReadyEvent();             // send ready message to Application Manager

    canvas = document.getElementById('tutorial');

    if (canvas.getContext) {
        ctx = canvas.getContext('2d');
    } else {
        alert("This browser doesn't support CANVAS!");
        return;
    }

    // register nservice manager callback to receive device connect and disconnect events
    nservice.registerManagerCallback(Main.onDeviceStatusChange);

    // initialize nservice device profile and get available devices
    nservice.getNServiceDevices(Main.onCustomObtained);
}

The TV application is activated and ready to receive data from connected devices, and process the data and commands.

Defining N-Service Device Event Handlers

The NServiceDevice interface provides the following event callback functions for messages received from the framework:

FunctionDescription
registerManagerCallbackUsed to register the callback function which is called by nsevrice device manager on any device connectivity status changes such as device connection or disconnection.
registerDeviceCallbackUsed to register the callback function which is called when an event (such as message arrival from a device, device group change) occurs at the device instance level.

The events passed can be processed to the callback function by switching on the sParam. The example below displays processing nservice device manager events such as device connect and device disconnect. In the example, the callback function prints an alert message on the console and calls the getNServiceDevices function to update the list of devices.

Main.onDeviceStatusChange = function (sParam) {
    alert("#### onDeviceStatusChange - Device status change recieved ####");
    alert("#### onDeviceStatusChange - event type is " + sParam.eventType + " ####");
    alert("#### onDeviceStatusChange - event device name is " + sParam.deviceName + " ####");
    alert("#### onDeviceStatusChange - event device type is " + sParam.deviceType + " ####");

    switch (Number(sParam.eventType)) {
        case nservice.MGR_EVENT_DEV_CONNECT:
            alert("#### onDeviceStatusChange - MGR_EVENT_DEV_CONNECT ####");
            if (sParam.deviceType == nservice.DEV_SMART_DEVICE) {
                document.getElementById('txtCS').innerHTML = "Connected NService: " + sParam.deviceName;
            }
            break;

        case nservice.MGR_EVENT_DEV_DISCONNECT:
            alert("#### onDeviceStatusChange - MGR_EVENT_DEV_DISCONNECT ####");
            break;

        default:
            alert("#### onDeviceStatusChange - Unknown event ####");
            break;

    }
    nservice.getNServiceDevices(Main.onNserviceObtained);
}

Following is an example implementation of the device event handling such as message arrival, group join and group leave. It is important that each device instance should register callback in order to get event from that device.

Add the following code to NserviceSample.js:

// Main.onNserviceObtained Called by getNServiceDevices in Main.onDeviceStatusChange
// callback function when a device is connected or disconnected.

Main.onNserviceObtained = function (nservices) {
    if (nservices.length > 0) {
        alert("#### onNserviceObtained - found " + nservices.length + " nservice device(s) ####");
        if (nservices[0]!= null && nservices[0].getType() == nservice.DEV_SMART_DEVICE) {
            alert("#### onNserviceObtained - get device instance ####");
            nservicedeviceInstance = nservices[0];

            nservicedeviceInstance.registerDeviceCallback(Main.onDeviceEvent);
        }
    } else {
        alert("#### onNserviceObtained - no nservice device found ####");
    }
}

// Main.onDeviceEvent is called when device instance event arrive.
// Example displays event type on console
// and on message arrival performs other functions provided by NService interface.

Main.onDeviceEvent = function (sParam) {
    switch (Number(sParam.eventType)) {
        case nservice.DEV_EVENT_MESSAGE_RECEIVED:
            alert("#### onDeviceEvent - DEV_EVENT_MESSAGE_RECEIVED ####");
            Main.onMessageReceived(sParam.eventData.message, sParam.eventData.context);
            break;
        case nservice.DEV_EVENT_JOINED_GROUP:
            alert("#### onDeviceEvent - DEV_EVENT_JOINED_GROUP ####");
            break;
        case nservice.DEV_EVENT_LEFT_GROUP:
            alert("#### onDeviceEvent - DEV_EVENT_LEFT_GROUP ####");
            break;
        default:
            alert("#### onDeviceEvent - Unknown event ####");
            break;
    }
}

Sending a Message to a Connected Device

The NServiceDevice interface provides the API DeviceInstance.sendMessage(Message) to send a message to the smart device related with device instance DeviceInstance.

See the following example:

function myFunction() {
    var message = {"message": "Hello smart device"}; //JSON Message
    /* nservicedeviceInstance is the instance related with one device.*/
    nservicedeviceInstance.sendMessage(message);
    return;
}

Sending a Message to All Connected Devices

The NService interface provides the nservice.broadcastMessage(Message) API to broadcast a message to all the connected smart device(s).

See the following example:

function myFunction() {
    var message = {"message": "Hello smart device, this is a broadcast message."};
    nservice.broadcastMessage(message);
    return;
}

Sending a Message to a Group of Connected Devices

The NService interface provides the nservice.multicastMessage(GroupID, Message) method for multicasting a message to the smart devices identified by the group ID (sGroupID).

See the following example:

function myFunction() {
    var groupID = "12345",
        message = {"message": "Hello smart device"};
    nservice.multicastMessage(groupID, message + groupID);
    return;
}

Handling Messages from a Smart Device

A message sent by a smart device is handled by the Main.onDeviceEvent callback function. The event type in this case is DEV_EVENT_MESSAGE_RECEIVED as shown in the example for Main.onDeviceEvent.

The code below shows how a message sent by a smart device is handled by the application. A message sent by a smart device can be a plain string, an XML string or a JSON string.

// This is example only, it is not provided as sample widget
Main.onDeviceEvent = function (sParam) {
    switch(Number(sParam.eventType)) {
        case nservice.DEV_EVENT_MESSAGE_RECEIVED:
            /* For details of event releated to each event refer to comments in Nservicedevice.js */
            Main.onMessageReceived(sParam.eventData.message, sParam.eventData.context);
            break;
            ...
    }
}

Main.onMessageReceived = function (message, context) {
    // message -> message body
    // context -> message context (headers and etc)
    alert("#### onMessageReceived:" + message);
    Main.drawTo(message);
}

Handling Messages with Attachments

To handle files sent by a smart device as a HTTP multipart message, construct a multipart message with:

  • The first part indicating the names of the files attached and how the application uses them.
  • Each subsequent part representing the binary data for the files.

See the following code that uses curl command as an example:

curl -H "SLDeviceID:12345" -F message='{"type": "ShowPic", "filename": "widget-sample"}' -F upload=@widget-sample.jpg
    http://192.168.1.108/ws/app/Canvas_NewOCI/queue

Each application has a temporary storage assigned by the framework used to store the files uploaded by the smart device. As shown above, a similar mechanism can be used in case the files are sent by the smart device. For example, the message body can look like the following:

MessageBody  =  {"type": "ShowPic", "filename": "widget-sample.jpg"}

The message body has a user defined format, and follows the general rules given below:

  • It should contain the file name of the attachments.
  • It usually contains the expected actions for those attachments.

The application, in this case, uses the filename to load and show the actual image file from the temporary storage. The TV provides the following URL to access the file in the temporary storage: http://127.0.0.1/ws/app/{Canvas_NewOCI}/file/{e63-x-2.jpg}.

See the following example:

Main.drawTo = function (message) {
    // evaluate the JSON message
    touchEvent = eval('(' + message + ')');

    if(touchEvent.type == "ShowPic") {
        var fileURI = "http://127.0.0.1/ws/app/Canvas_NewOCI/file/" + touchEvent.filename;
        document.getElementById("myImage").src = fileURI;
        ...
    }
}

Unloading the TV Application

When a TV application quits, the system initiates the function described in Main.onUnload() function. At this point, the OCI nservice device must be destroyed. See the following example:

Add the following code to NserviceSample.js:

Main.onUnload = function() {
    alert("[NServiceTutorial]: Main.onUnload()");
}

HTTP Client Application

This task demonstrates the code written in the HTTP Client Application that interacts with the Smart TV application. These functions use AJAX calls to form HTTP messages that test the Smart TV convergence framework. This task covers the steps needed to first discover and then send and receive messages to and from a Samsung Smart TV. This model of building and sending HTTP messages can be extended to write any client applications that support HTTP.

Finding a Samsung Smart TV on the Network

This section describes the steps necessary to discover a TV using UPnP. It is assumed that the application developer has access to an existing UPnP stack. There are several open source UPnP implementations for developers and Cyberlink for Java is one of them.

  • CyberLink Site Go to CyberLinkForJava of Networking to find programming guides and source codes. CyberLinkForJava can be used on the Android Platform.

Note

Read the license information in the programming guide before using CyberLinkForJava.

The files needed for the sample application are here.

  1. The mobile device issues an MSearch using a specific search target (ST) header to search Samsung smart TV supporting Multi Screen service on the network.

    ControlPoint _controlPoint = new ControlPoint();
    _controlPoint.search("urn:samsung.com:service:MultiScreenService:1");
    
  2. TV devices that implement the service specified in the search target (ST) field will respond with HTTP/1.1 200 OK and URL of the device description. At this point, the mobile device found TV or Emulator that supports multi screen service.

    // You should override below method (deviceSearchResponseReceived) to getting response for search.
    _controlPoint.addSearchResponseListener(new SearchResponseListener() {
    
    }
    
    @Override
    public void deviceSearchResponseReceived(SSDPPacket ssdpPacket) {
        ...
    }
    
  3. Once the Smart device has the URL, it requests the device description by issuing a GET request on the URL. The Smart device then parses the description.xml file to extract the information on the state variables and actions provided by the device.

    public void deviceSearchResponseReceived(SSDPPacket ssdpPacket) {
        String location = ssdpPacket.getLocation();
        String ip = ssdpPacket.getRemoteAddress();
        Log.d("location = " + location);
        // Get xml content by location
        HttpGet httpget = new HttpGet(location);
    }
    
  4. Once the mobile device has the URL, it requests the device description by issuing a GET request on the URL. The mobile device then parses the description.xml file to extract friendly name of TV.

    @Override
    
    public void deviceSearchResponseReceived(SSDPPacket ssdpPacket) {
    
        String location = ssdpPacket.getLocation();
        String ip = ssdpPacket.getRemoteAddress();
        Log.d("location = " + location);
    
    
        //Get xml content by location
        HttpGet httpget = new HttpGet(location);
        DefaultHttpClient client = new DefaultHttpClient();
        HttpResponse resp = null;
    
        try {
            resp = client.execute(httpget);
    
        } catch (IOException e) {
    
            e.printStackTrace();
        }
    
        //Try to find friendly name in XML
    
        if (resp != null && resp.getStatusLine().getStatusCode() == 200) {
            //Mobile device can get Application URL from Application-URL header.  
            Header[] headers = resp.getHeaders("Application-URL");
    
            if(headers != null && headers.length > 0) {
                //Find real application URL with port
                ApplicationURL = headers[0].getValue();
            }
    
            JaxpParser parser = null;
            try {
                parser = new JaxpParser();
                InputStream is = resp.getEntity().getContent();
                Node deviceNode = parser.parse(is).getNode("device");
    
                if (deviceNode != null) {
                    String friendly = deviceNode.getNodeValue("friendlyName"); // get friendly name
                    Log.d("friendly : " + friendly);
    
                    DeviceBasicInfo dev = new DeviceBasicInfo();
                    dev.setLocation(location);
                    dev.setFriendlyName(friendly);
    
                    devSearchHashMap.put(ip, dev);
                    updateDeviceList(new ArrayList(devSearchHashMap.values()));
                }
            } catch (IOException e) {
                e.printStackTrace();
            } catch (ParserException e) {
                e.printStackTrace();
            }
        }
    }
    

Connecting to a TV Application

To connect to the TV, the Smart device uses the following string:

POST /ws/app/{appID}/connect

The URL requires the IP address (received from the UPnP stack) and port number of the TV (port is not fixed: you can get Application URL(http://<TV IP address>:<TV port number>/ws/app from location response), and the ID of the application to connect to. Note that the application must be running before connect can be called. Perform a POST operation with this information as follows:

http://<TV IP address>:<TV port number>/ws/app/<appID>/connect

If the Smart device has connected successfully, a 200 OK HTTP response code is received. In case of an error, a 4XX or 5XX response code is received. For the exact error codes, see Convergence App API.

See the following sample JavaScript function that makes an AJAX call to perform this task:

function connect() {
    var URL = "http://127.0.0.1:8080/ws/app/Canvas_NewOCI/connect";
    addLogMsg("Connecting...");
    $.ajax({
        type: "POST",
        url: URL,
        headers: {
            SLDeviceID: document.getElementById("deviceID").value,
            VendorID: "VenderMe",
            DeviceName: "IE-Client",
            GroupID: "feiGroup",
            ProductID: "SMARTDev",
            connection: "close"
        },
        async: true,    // if set to non-async, browser shows page as "Loading..."
        cache: false,
        timeout: 3000,  // timeout in ms

        success: onConnect,
        statusCode: {
            404: function () {
                alert("404 TV application not running. ");
            },
            409: function () {
                alert("409 conflict on device ID. ");
            },
            500: function () {
                alert("500 server internal error 500 ");
            },
            503: function () {
                alert("503 server may reach maximum connections ");
            }
        }

    });
}

Sending a Message to the Connected TV Application

  1. To send a message to the TV, use the following URL:

    POST /ws/app/{appID}/queue
    
  2. The URL requires parameters received from the UPnP stack - the IP address and port number of the TV, and the ID of the application currently running on TV. Send an HTTP POST operation to the proper URL with this information as follows:

    http://<TV IP address>:<TV port number>/ws/app/<appID>/queue

If the request is successful, a 200 OK HTTP status code is received from the TV. In case of an error, a 4XX or 5XX response code is received. For the exact error codes, see the Convergence App API.

See the following sample JavaScript function making an AJAX call to perform this task:

function postMsg(message) {
    var URL = "http://127.0.0.1:8080/ws/app/Canvas_NewOCI/queue";
    $.ajax({
        type: "POST",
        url: URL,
        data: message,
        contentType: dataType,
        headers: {
            SLDeviceID: deviceID,
            VendorID: "VenderMe",
            ProductID: "SMARTDev"
        },
        async: false,   // if set to non-async, browser shows page as "Loading..."
        cache: false,
        timeout: 3000,  // timeout in ms

        success:  alert("Success"),
        error:  alert("Failed")
    });
    msgNumber += 1;
}

Sending a Message with an Attachment

Sending a message with an attachment is a special case of sending a message described in the previous section. The only difference is that the HTTP request will be a multi-part request. See the following sample JavaScript function to create a multipart request:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        <title>Tutorial Video App</title>
        <script type="text/javascript">
            function uploadFile() {
                var url = "http://127.0.0.1:8080/ws/app/Canvas_NewOCI/queue",
                    input = document.getElementById("photo"),
                    formData = new FormData();

                formData.append('message', '{"type":"ShowPic", "filename":"' + input.files[0].name + '"}');
                formData.append(input.files[0].name, input.files[0]);

                var xhr = new XMLHttpRequest();
                xhr.open('POST', url, true);
                xhr.setRequestHeader('SLDeviceID', deviceID);
                xhr.onload = function (e) {addLogMsg("upload image:" +input.files[0].name + " success");  };
                xhr.send(formData);  // multipart/form-data
            }
        </script>
    </head>
    <body>
        <fieldset>
            <legend>Upload photo</legend>
            <input type="file" name="photo" id="photo">
            <input type="text" id="photo" value="sampleFile.jpg"> ////The file you wish to upload
        </fieldset>
    </body>
</html>

Disconnecting from a TV Application

  1. To disconnect the Smart device from the TV, use the following URL:

    POST /ws/app/{appID}/disconnect
    
  2. The URL requires parameters received from the UPnP stack - the IP address and port number of the TV, and the ID of the application currently running on TV. Perform a POST operation with this information as follows:

    http://<TV IP address>:<TV port number>/ws/app/<appID>/disconnect

If disconnected successfully, a 200 OK HTTP response code is received from the TV. In case of an error, a 4XX or 5XX response code is received. For the exact error codes, see the API documentation.

Following is a sample JavaScript function making an AJAX call to perform this task:

function disconnect() {
    var URL = "http://127.0.0.1:8080/ws/app/Canvas_NewOCI/disconnect";
    $.ajax({
        type: "POST",
        url: URL,
        headers: {
            SLDeviceID: deviceID
        },
        async: true,
        cache: false,
        timeout: 3000,  // timeout in ms

        success: alert("Success"),
        error: alert("Failed")
    });
}

Convergence Application Use Case

This tutorial task involves the following files:

Canvas_NewOCI.
A sample interactive TV application running on the Smart TV emulator. The application provides drawing functions on a canvas based on messages received from the a client application.
Chrome-JSClient.
A sample HTTP client application. The sample HTTP application is a JavaScript client application running on Google Chrome web browser and connected to the Smart TV emulator via localhost (http://127.0.0.1).

Running a Sample Convergence Application

To run the application,

  1. Launch the sample TV application Convergence_Tutorial_TV on the Smart TV emulator. The following image shows a screenshot of the emulator displaying the Smart Hub main page:

    Figure: Smart Hub on emulator

  2. Select the Convergence_Tutorial_TV TV application. The following screen is displayed:

    Figure: Convergence_Tutorial_TV

  3. Start the Chrome web browser with a command line option that disables web security. You need to disable web security to enable file upload from the Chrome-JSClient to the sample TV application. To do this, please follow the steps:

    1. Go to Google Chrome > Properties.

      Figure: Google Chrome Properties

    2. Add the text --disable-web-security at the end of the string in the Target: field.

      Figure: Google Chrome Shortcut Target field

    3. Copy the entire line in the Target: field. Open the Windows command prompt and paste the copied line there. Press Enter to launch Google Chrome.

      Figure: Path to paste in Windows command prompt.

  4. Launch index.html (under the >Chrome-JSClient folder) using Google Chrome browser. After launch, the following screen is displayed:

    Figure: index.html launched

Depending on the device from which you are running the client, you may have to provide the following parameters:

Service base URL
http://<IP>:<Port>/ws
<IP>
IP address of the PC running Smart TV emulator
<Port>
Port number - default is set to 80

Note

In this case, the client is running on the same machine. Therefore the localhost and the port number used was 8080 and the address http://127.0.0.1:8080/ws.

Device ID

This can be any regular string, for example 12345.

Note

Based on the Device ID, the TV framework differentiates between device(s), therefore you should make sure that this is different for all devices you wish to connect to the TV.

Connecting to the TV Application

Once both the sample TV application and have been launched, connect Chrome-JSClient.

  1. On Chrome-JSClient, provide the Service base URL and the device ID.
  2. Click on the Connect button shown in the screen below.

Figure: Connecting to the TV application

After clicking on Connect button, you see the following screen on the emulator notifying that a device was connected:

Figure: Device connected

Note

You can launch multiple instances of the Google Chrome browser with the index.html (with different device ID) to emulate connecting multiple devices to the Smart TV. A maximum of 4 connected devices is allowed.

Sending Messages to the TV

To demonstrate this, the sample TV application is implemented in such a way that it maps (draws) the coordinates received from the client application onto the sample TV application canvas. On the client, a list of coordinates to be sent to the sample TV application is provided (you can change the list for test purposes). The following image shows the list of coordinates and the Send button that sends these coordinates to the sample TV application:

Figure: Sending coordinates

Once you press the Send button, the following emulator screen is displayed:

Figure: Emulator screen after sending coordinates

When you press the ClearCanvas button the coordinates mapped on the canvas of the sample TV application are removed. See the following images:

Figure: Clear the canvas - client view

Figure: Clear the canvas - TV (emulator) view

Note

Other buttons (SendGroupMessage and SendMessageToDevice) can also be used if multiple devices are connected to the Smart TV emulator (or multiple instances of the Chrome-JSClient).

Uploading Files to the TV Application

The Chrome-JSClient also provides a way to upload files to the sample TV application using HTTP multipart POST message. To do this:

  1. Select the Choose File button on the client
  2. Select any local file from your system
  3. Press the Upload button

Figure: Uploading files to the TV application

Once you have performed the steps above, you see the following screen on the Smart TV emulator:

Figure: Uploaded files

The image selected on the client is visible on the bottom-centre region of the Smart TV emulator. You can perform these steps any number of times.

Disconnecting the Application

You can disconnect the Chrome-JSClient application from the sample TV application at any time by pressing the Disconnect button. This calls the disconnect API, and the TV framework will remove all entries for that device.

Figure: Disconnect button

To Bookmark this document, click

BOOK MARK 30