[Sample App] Phone to Phone Video transmission via UDP over WiFi

[Dave Smart]

Hi Guys,

I thought you might like this little example I made for sending video images from one device to another device via UDP packets.

In this sample I make use of an HTML canvas to render the video frames.

Note1: The next version of DroidScript will include a new net.ReceiveDatagrams() method which will improve the frame rate significantly :)

Note2: It is possible to extract raw data instead of jpegs if you want to do some image processing before transmission (it's just slower to transmit).

Note3: Some WiFi routers will block large quantities of UDP messages (it may be possible to change this in the router config).

SPK's are attached.

Regards

David

[Dave Smart]

This is the improved (faster) way of receiving the UDP packets:-

     

 //Create UDP network object.
 net = app.CreateNetClient( "UDP" );
 net.SetOnReceive( net_OnReceive );
 net.ReceiveDatagrams( "UTF-8", port );


//Called when we get UDP network messages.
function net_OnReceive( chunk )
{
    //Collect data into a frame array
    var chunkNum = parseInt(chunk.substr(0,2));
    var numChunks = parseInt(chunk.substr(3,2));
    frame[chunkNum] = chunk.substr(6);
   
    if( chunkNum==numChunks-1 )
    {  
    //Send image data to html canvas.
    var rawData = frame.join("");
    web.Execute( "ShowImage(\""+ rawData +"\")" );
    }
}

[Manuel Lopes]

thanks steve

[moderator edit: I think you mean "Thanks Dave" :)]

[Syed Munawer Hassan]

Is not working

[JustAnotherDude]

i belive it is not working because:

Updates of DroidScript have broken this sample that exist since 2015

Your device is not supportet

You have edited a line and noe you say that it will not work ;)

Potatoe

I hope it will say all what you need ^^

[Syed Munawer Hassan]

I think the problem is with IP I'm getting 192.168.43.5 pool while 192.168.1.3 is hard coded isn't it ?

[John Calder]

My project, academic, open source, involves programming android phones as monitoring cameras eg wildlife, security. Taking images on motion detection and uploading them to a server. For my 3rd attempt at the client side and 3rd time lucky I started with "Camera Motion". I wanted to upload the "jpgbase64" string but I could not get data out of cam.GetPixelData as in:

request = cam.GetPixelData("jpgbase64", 0, 0, w, h);

I did a lot of searching and experimentation, I even tried saving a file from "cam", opening it in "img" then extracting "img.GetPixelData" and that actually worked on 1 of my 2 test phones.  I found a better fix in "Video Send".

"Camera Motion" has this code:

cam = app.CreateCameraView(0.4, 0.8);

...

cam.SetPictureSize(1024, 768);

GetPixelData started working for me when I changed this to:

cam = app.CreateCameraView(0.4, 0.8, "XGA,UseBitmap");

I therefore suggest this as a change for the "Camera Motion" example.

--------------------------------

I have posted my work-in-progress code "XMMonitorDSL" below.
It is currently working well but needs some tidying up.
Question for the DroidScript leaders - we academics are into "attribution" of our sources like sample code.
How do you like to be attributed? Is this wording good for you?

The server-side code is on the "Github" open source hosting site:

https://github.com/manukautech/XMRemoteMonitor

"XMMonitorDSL" is a work in progress and I plan for it to be part of the next version of the "XMRemoteMonitor" package.

--------------------------------

//XMMonitorDSL

//Remote monitoring camera Android client for use with the XMRemoteMonitor server app

//Part of the open source project "XMRemoteMonitor"

//https://github.com/manukautech/XMRemoteMonitor

//Attribution:

//"XMMonitorDSL" is based on the sample app "Camera Motion" by the DroidScript Project.

//"Camera Motion" is part of the documentation of the "DroidScript" package available in Google Play:

//Smart, D. et al.(2014-2019) "DroidScript". [System including IDE and APIs 

//for programming Android devices with JavaScript]. Retrieved from: 

//https://play.google.com/store/apps/details?id=com.smartphoneremote.androidscriptfree

//Initialise variables 

//with variables most needed for configuration up first

var cameraNumber = 1;

//Server app root URL if you have one. Do NOT end with "/" because the code below 

//will add that. Otherwise "placeholder" or "example.com" (all in lower case) 

//will give local-only recording of images, keeping the most recent 2000

var requestAppURL = "placeholder";

var sensitivity = 50; //percent.

var minPeriod = 200;  //millisecs.

var mosaic = "10x10";

var captureFolder = "/sdcard/modet/";

var fileTemp;

var counter = 1000;

//State management

//2018-12-26 JPC trigger override capture without motion detection on long timeout

var isModet = false;  //code sets isModet to true on motion detection events

var overrideTimeout = 900000; //usually 900000 = 900 seconds ie 15 min

var placeholderBase64DSL = ""; //image to record when scheduler specifies off-time

//Setup AJAX  refs

//https://www.w3schools.com/xml/ajax_xmlhttprequest_send.asp

//https://www.w3schools.com/js/js_ajax_http.asp

var xmlhttp = new XMLHttpRequest();

var cam;

//Scheduled active from 7pm to 6:30am on weekdays plus all weekend

//'return true' effect is to capture images on motion detection

//'return false' effect is to NOT capture images on motion detection

function scheduledActive() {

    //uncomment next statement "return true;" to switch off scheduling

    //return true;

    var d = new Date();

    //Remain active through the weekend

    if (d.getDay() == 6 || d.getDay() == 0) {

        return true;

    }

    var dayMinutes = d.getHours() * 60 + d.getMinutes();

    if (dayMinutes >= 19 * 60) {

        //7pm

        return true;

    } else if (dayMinutes <= 6 * 60 + 30) {

        //6:30am

        return true;

    }

    return false;

}

//Called when application is started.

function OnStart() {

    //Fix orientation to landscape since

    //most phone cameras work this way.   

    app.SetOrientation("Landscape");

    //Create horizontal layout that fills the screen.

    lay = app.CreateLayout("Linear", "Horizontal,FillXY");

    //Create frame layout on left for camera view.

    layLeft = app.CreateLayout("Frame");

    layLeft.SetMargins(0, 0.1, 0.02, 0);

    lay.AddChild(layLeft);

    //Create camera view control. Work with image resolution "XGA" = 1024 x 168

    cam = app.CreateCameraView(0.4, 0.8, "XGA,UseBitmap");

    cam.SetSound(false);

    layLeft.AddChild(cam);

    //Create vertical layout on right for other stuff.

    layRight = app.CreateLayout("Linear", "Vertical,FillXY");

    lay.AddChild(layRight);

    layLogin = app.CreateLayout("Linear", "Horizontal,FillX");

    layLogin.SetMargins(0, 0, 0, 0);

    layLogin.SetPadding(0, 0.01, 0, 0.01);

    layRight.AddChild(layLogin);

    //2019-01-25 JPC Create a text control for server password

    txtAccess = app.CreateTextEdit("", 0.2, 0.1, "password");

    txtAccess.SetTextColor("#ffffffff");

    txtAccess.SetBackColor("#ff222222");

    layLogin.AddChild(txtAccess);

    btn = app.CreateButton("Start", 0.2, 0.1);

    btn.SetOnTouch(StartDetection);

    btn.SetMargins(0.02, 0, 0, 0);

    btn.SetPadding(0, 0, 0, 0.01);

    btn.SetTextColor("#ffffffff");

    btn.SetBackColor("#ff222222");

    layLogin.AddChild(btn);

    //Create a text control to show logs.

    txt = app.CreateText("", 0.5, 0.3, "Multiline,Left");

    txt.SetMargins(0, 0.1, 0, 0);

    txt.SetPadding(0.01, 0.01, 0.01, 0.01);

    txt.SetTextColor("#ffffffff");

    txt.SetBackColor("#ff222222");

    layRight.AddChild(txt);

    //Create sensitivity seek bar label.

    txtSens = app.CreateText("Sensitivity");

    txtSens.SetMargins(0, 0.05, 0, 0);

    layRight.AddChild(txtSens);

    //Create sensitivity seek bar.

    skb = app.CreateSeekBar(0.3, -1);

    skb.SetOnTouch(skb_OnTouch);

    skb.SetValue(sensitivity);

    layRight.AddChild(skb);

    //Create mosaic spinner label.

    txtTiles = app.CreateText("Mosaic");

    txtTiles.SetMargins(0, 0.05, 0, 0);

    layRight.AddChild(txtTiles);

    //Create mosaic spinner.

    var layouts = "3x2,3x3,5x5,10x10";

    spin = app.CreateSpinner(layouts, 0.2, -1);

    spin.SetOnTouch(spin_OnTouch);

    spin.SetText(mosaic);

    layRight.AddChild(spin);

    //Set menus.

    app.SetMenu("Capture,Exit");

    //Add main layout to app.

    app.AddLayout(lay);

    //Initialise AJAX

    xmlhttp.onreadystatechange = function () {

        ajaxCallback();

    };

    //"Start capture after 1 second" replaced with Start button

    //setTimeout( "StartDetection()", 1000 ); 

}

//Start motion detection.

function StartDetection() {

    //2019-01-26 JPC

    //Get app settings and info especially the placeholder image from an AJAX call to server-side

    xmlhttp.open("GET", requestAppURL + "/Home/GetSettings", true);

    //xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");

    xmlhttp.send();

    //Create an image control over the top of the 

    //camera view with transparency (alpha) and with a

    //fixed internal bitmap the same size as camera view.

    w = cam.GetImageWidth();

    h = cam.GetImageHeight();

    img = app.CreateImage(null, 0.4, 0.8, "fix", w, h);

    img.SetAlpha(0.5);

    layLeft.AddChild(img);

    //Enable 10x10 matrix motion detection and mark

    //motion detections in the image control.

    //for "mosaic" value settings like "10x10", split on "x" to get the numeric values

    var mx = parseInt(mosaic.split("x")[0]);

    var my = parseInt(mosaic.split("x")[1]);

    cam.MotionMosaic(mx, my, (100 - sensitivity) / 5, minPeriod, img);

    cam.SetOnMotion(OnMotion);

    //Create folder for saved pictures 

    app.MakeFolder(captureFolder);

    //cam.SetPictureSize(1024, 768);

    //Create array to hold log messages.

    log = new Array();

    Log(captureFolder);

    //Start preview.

    cam.StartPreview();

    //Send a placeholder image if no motion detection for the specified time

    setInterval(idleOverride, overrideTimeout);

}

//Handle menu selections.

function OnMenu(name) {

    if (name == "Capture")

        idleOverride();

    else if (name == "Exit")

        app.Exit();

}

//Called when motion is detected.

//(data contains an array of detection strength 

//values corresponding to each mosaic tile)

function OnMotion(data) {

    //Save picture to sdcard.

    //file = captureFolder + "T" + xgetISONow().split("T")[1] + ".jpg";

    if (requestAppURL.indexOf("example.com") > -1 || requestAppURL == "placeholder") {

        sendImage(scheduledActive());

    } else if (scheduledActive()) {

        isModet = true;

        sendImage(true);

    } else {

        isModet = false;

        Log("Scheduled off ignore " + xgetTime());

    }

}

//Send new image or placeholder to the server

function sendImage(isScheduled) {

    //Running locally without a server

    if (requestAppURL.indexOf("example.com") > -1 || requestAppURL == "placeholder") {

        if (isScheduled) {

            localRecordImage();

            Log("Local recording only " + xgetTime());

        }

        return;

    }

    var request;

    if (isScheduled) {

        request = cam.GetPixelData("jpgbase64", 0, 0, w, h);

    } else {

        request = placeholderBase64DSL;

        Log("Save Placeholder image " + xgetTime());

    }

    var formData = new FormData();

    formData.append("CameraNumber", cameraNumber);

    formData.append("AccessKey", txtAccess.GetText());

    //2019-01-25 JPC this base64 does not have any prefix so no need to slice

    formData.append("ImageBase64", request);

    //appPath only applies to web app version, phone app server is assigned above

    //throw new Error("Test error");

    xmlhttp.open("POST", requestAppURL + "/Home/ImagePost", true);

    //xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");

    xmlhttp.send(formData);

    localRecordImage();

}

function localRecordImage() {

    var file = captureFolder + counter + ".jpg";

    cam.TakePicture(file);

    counter++;

    if (counter >= 3000) counter = 1000;

}

function ajaxCallback() {

    if (xmlhttp.readyState === 4 && xmlhttp.status === 200) {

        var response = xmlhttp.responseText;

        //our AJAX methods return json strings so can use same json handling

        //function xreceive as SignalR

        xreceive(response);

    }

}

//SignalR and AJAX responses in json format both handled by this one function

function xreceive(response) {

    var jsonObject = JSON.parse(response);

    if (response.indexOf("issuccess") > -1 && !jsonObject.issuccess) {

        app.ShowPopup(jsonObject.message);

    } else if (jsonObject.categoryid && jsonObject.categoryid == 20) {

        placeholderBase64DSL = jsonObject.xdata[0];

        //Code variation for this DroidScript Layout-based app

        headerLength = placeholderBase64DSL.indexOf(";base64,") + 8;

        placeholderBase64DSL = placeholderBase64DSL.slice(headerLength);

        Log("Success: download of setup data.");

        hasReadSettings = true;

    } else if (jsonObject.categoryid && jsonObject.categoryid == 21) {

        Log("Image saved OK " + xgetTime());

    } else {

        app.ShowPopup(response + "ERROR: Data from server is badly formatted or otherwise not making sense.")

    }

    //if (debug) DebugDisplay(response);

}

function idleOverride() {

    if (!isModet) {

        if (scheduledActive()) {

            //No motion detection in the last 15 min so send a capture

            OnMotion([0]);

        } else {

            //send a placeholder image.

            sendImage(false);

        }

    }

    isModet = false;

}

//-------------------------------------------

//Utilities

//Called when user touches sensitivity seek bar.

//( value ranges from 0 to 100 )

function skb_OnTouch(value) {

    sensitivity = value;

    ChangeSettings();

}

//Called when user touches mosaic spinner.

function spin_OnTouch(item) {

    mosaic = item;

    ChangeSettings();

}

//Change the motion detection settings.

function ChangeSettings() {

    var x = 3, y = 2;

    if (mosaic == "3x3") { x = 3; y = 3; }

    else if (mosaic == "5x5") { x = 5; y = 5; }

    else if (mosaic == "10x10") { x = 10; y = 10; }

    cam.MotionMosaic(x, y, (100 - sensitivity) / 5, minPeriod, img);

}

//Add messages to log.

function Log(msg) {

    var maxLines = txt.GetMaxLines() - 1;

    if (log.length >= maxLines) log.shift();

    log.push(msg + "\n");

    txt.SetText(log.join(""));

}

//This is returning Greenwich Mean Time (Universal Time) so deprecated 

//and replacing beginning with function xgetTime()

function xgetISONow() {

    //Finding issues with non-alphanumeric characters in file names so substitute

    //Date and time ends up looking like this. Split on "T" to separate date and time.

    //2019-01-25T213344D23

    var d = new Date();

    var s = d.toISOString();

    s = s.replace(/\:/g, "");

    s = s.replace(".", "D");

    s = s.slice(0, 20);

    return s;

}

//Returns local time in format HH:mm:ss.sss example 21:03:16.812

function xgetTime() {

    var d = new Date();

    var hours = ("00" + d.getHours()).slice(-2);

    var minutes = ("00" + d.getMinutes()).slice(-2);

    var seconds = ("00" + d.getSeconds()).slice(-2);

    var decimalSeconds = ("000" + d.getMilliseconds()).slice(-3);

    nowTime = hours + ":" + minutes + ":" + seconds + "." + decimalSeconds;

    return nowTime;

}

[Dave]

Hi John,

That looks like an interesting project :)  

I'm generally happy with that attribution, perhaps change the bit that says "by the DroidScript Project" to "by droidscript.org"

You might be interested to know that I'm planning to add a Tensor Flow module to DS soon...so maybe you could recognize certain species and count them...or in the case of rats, maybe zap em ;)

Regards

David

[Omer Meshy]

Hello

Since the last DS version I can't run these codes, because I get an warning message:

WARNING: Net.SendDatagram() failed! (Attempt to invoke interface method 'void com.smartphoneremote.ioioscript.INetComms.Send(byte[], boolean)' on a null object reference)

• WARNING: Net.SendDatagram() failed! (Attempt to invoke interface method 'void com.smartphoneremote.ioioscript.INetComms.Send(byte[], boolean)' on a null object reference)

Why is it happens and how can I fixed it?

[Dave]

The latest alpha should fix this problem:-

http://androidscript.org/alpha/DroidScript_179a1/