In this article, I'm going to describe architecture enhancements for the control system of the WebRTC-controlled telepresence robot I built a few months ago, presented in a previous article.
The four-wheel base of the telepresence robot
Since I did not manage to have a satisfying WebRTC support directly in a native Android app, I previously settled for a hack where the smartphone of the Telebot uses two different connections to the signaling channel: one to receive user control in the Android app, and one to handle the WebRTC session in the browser.
This was bad for two reasons:
- The robot can enter an incoherent state if one connection is closed and not the other.
- User control commands do not benefit from WebRTC, instead they travel through the server, adding latency and jitter.
The idea for the new architecture is to have the Android app run a small HTTP server in background that can accept motor control commands and send them to the Bluetooth device. We will send users commands on an RTCDataChannel and forward them to this small HTTP server with JavaScript in the browser.
General schematic of the enhanced architecture
Let's put together a tiny single-threaded HTTP server for our Android app. For the sake of simplicity, requests and responses content will be formatted as JSON. I know the implementation is only partial, but it is totally sufficient and standard-compliant for our limited needs.
In particular, we need to properly handle Cross-Origin Resource Sharing (CORS) because since the server is local, we are going to hit it with cross-origin requests.
/**
* Tiny HTTP server implementation for JSON requests
* HTTP access control (CORS) is supported
*/
public class HttpServer implements Runnable {
private final int mPort;
private ServerSocket mServerSocket;
[...]
/**
* Server main loop
*/
@Override
public void run() {
try {
// Open the server socket
mServerSocket = new ServerSocket(mPort);
// Loop on accepted connections
while(true) {
Socket socket = mServerSocket.accept();
handle(socket);
}
} catch(SocketException e) {
// Stopped
} catch(IOException e) {
e.printStackTrace();
}
}
/**
* Receive an HTTP request and send the response
*/
private void handle(Socket socket) {
BufferedReader reader = null;
PrintStream output = null;
try {
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
output = new PrintStream(socket.getOutputStream());
// Read request line
String requestLine = reader.readLine();
Log.d(TAG, requestLine);
// Parse request line
String[] tokens = requestLine.split(" ", 3);
if(tokens.length != 3) {
sendResponse(output, "400 Bad Request", false);
return;
}
String method = tokens[0].toUpperCase();
String route = tokens[1];
// Read the headers
Map headers = new HashMap<>();
String line;
while(!(line = reader.readLine()).isEmpty()) {
String[] s = line.split(":", 2);
headers.put(s[0].trim(), (s.length == 2 ? s[1].trim() : ""));
}
// Get the content length
int length = 0;
if(headers.containsKey("Content-Length"))
length = Integer.parseInt(headers.get("Content-Length"));
// Get the content and parse it
JSONObject content = null;
if(length > 0)
{
char[] buffer = new char[length];
reader.read(buffer);
try {
content = new JSONObject(new String(buffer));
} catch(JSONException e) {
e.printStackTrace();
}
}
// Handle CORS preflight OPTIONS request
if(method.equals("OPTIONS")) {
sendResponse(output, "200 OK", true);
if(headers.containsKey("Access-Control-Request-Method")) {
output.print("Access-Control-Allow-Methods: POST, GET\r\n");
output.print("Access-Control-Allow-Headers: Content-Type\r\n");
}
else output.print("Allow: POST, GET\r\n");
output.print("\r\n");
output.flush();
return;
}
// Process the request
JSONObject response;
try {
response = process(method, route, content);
} catch(Exception e) {
e.printStackTrace();
sendResponse(output, "500 Internal Server Error", false);
return;
}
if(response == null) {
sendResponse(output, "404 Not Found", false);
return;
}
// Send the response
byte[] b = response.toString().getBytes();
sendResponse(output, "200 OK", true);
output.print("Content-Type: application/json\r\n");
output.print("Content-Length: " + b.length + "\r\n");
output.print("\r\n");
output.write(b);
output.flush();
} catch(IOException e) {
e.printStackTrace();
}
finally {
try {
if(output != null) output.close();
if(reader != null) reader.close();
socket.close();
} catch(IOException e) {
e.printStackTrace();
}
}
}
/**
* Send an HTTP response on stream
*/
private void sendResponse(PrintStream output, String response, boolean otherHeaders) {
output.print("HTTP/1.1 " + response + "\r\n");
output.print("Connection: close\r\n");
output.print("Access-Control-Allow-Origin: *\r\n"); // CORS
if(!otherHeaders) {
output.print("\r\n");
output.flush();
}
}
/**
* Process a request, should be overridden in subclasses
*/
public JSONObject process(String method, String route, JSONObject content) throws Exception {
return null;
}
}
Now that our small HTTP server is ready, we have to handle the control requests by specializing the class.
[...]
/**
* Process a control request
*/
@Override
public JSONObject process(String method, String route, JSONObject content) throws Exception {
if(route.equals("/") || route.equals("/control")) {
if(method.equals("POST") && content != null) {
mLeft = content.getInt("left");
mRight = content.getInt("right");
mHandler.setControl(mLeft, mRight);
}
JSONObject response = new JSONObject();
response.put("left", mLeft);
response.put("right", mRight);
return response;
}
return null;
}
The setControl() function writes the command over the Bluetooth serial connection.
[...]
/**
* Send motor control command, values are in percent
*/
public void setControl(int left, int right) {
if(mSerialThread != null) {
mSerialThread.writeln("L " + left); // left
mSerialThread.writeln("R " + right); // right
mSerialThread.writeln("C"); // commit
}
}
We are now able to send requests containing control commands from the JavaScript code!
[...]
// Local control API
var localControlUrl = "http://127.0.0.1:11698/control";
// Send control to local API
function localControl(left, right) {
var xhr = new XMLHttpRequest();
xhr.open("POST", localControlUrl, true);
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.send(JSON.stringify({
left: left,
right: right
}));
}
This assumes that resources on 127.0.0.1 are not considered mixed content, even if accessed over HTTP from an HTTPS page. The W3C specification is strangely not clear about that, but it seems to me this is the right behaviour, since you can't get a TLS X.509 certificate for localhost. Chromium allows access, anyway.
The last step is to set up the RTCDataChannel. The user side opens it, and the robot side accepts it:
[...]
if(active) {
// Create control data channel
var controlChannelOptions = {
ordered: true
};
controlChannel = peerConnection.createDataChannel("control", controlChannelOptions);
}
else {
// Accept control data channel
peerConnection.ondatachannel = function(evt) {
if(evt.channel.label == "control") {
controlChannel = evt.channel;
controlChannel.onmessage = function(evt) {
var message = JSON.parse(evt.data);
if(message.control) {
var left = parseInt(message.control.left);
var right = parseInt(message.control.right);
localControl(left, right);
}
};
controlChannel.onclose = function() {
localControl(0, 0);
};
}
};
}
Finally, it works! It is way easier to move the robot thanks to the reduced latency. Of course, the complete source code is available on my repository on GitHub.
