Using an Android device as a thin UI for C ++ programs
I want to share with the community a project that I have been doing slowly for the past few months.
Often there are situations when you want to manage your program from a phone or tablet, but it is inappropriate (or not possible) to write a separate application on the phone for this (too much work for this project, no development experience for android, etc. ) This situation periodically occurred to me, and in the end I decided to deal with this problem once and for all. The result is a system whose architecture and use are described in this article.
Creating a system that allows implementing in C ++ programs a plug-in UI based on android devices. At the same time, I wanted to minimize the dependencies of custom C ++ code on third-party libraries, as well as abstract it from the data transfer protocol. The system should consist of two parts: C ++ libraries and android applications.
The system has a client-server architecture, where android devices act as clients, the server is a user program. Communication between them is carried out using TCP / IP sockets. To implement the communication protocol, the TAU library was created .
The main tasks for which the library is responsible:
The library consists of the following namespaces:
We will begin the demonstration of using the library with a simple example, in which a welcome message will simply be displayed on the screen. Here's what the custom code will look like in our case:
The main class that contains all the user logic that interacts with the client device ( MyEventsDispatcher ) should be inherited from tau :: util :: BasicEventsDispatcher . It overrides 2 methods from the base class: onClientConnected () and packetReceived_clientDeviceInfo () . The first is called when the client connects. The second method will be executed when information about the client device comes after the connection to the server (the first packet after the connection is sent by the client).
In our case, the first method is trivial - it only displays an informational message to the console. In the second method, the server sends a layout to the client - data about which interface should be displayed on the client.
All code responsible for transmitting data over the network is in main (). In this case, the boost :: asio library is used to implement communication . There are corresponding abstractions in the tau :: util namespace , which makes this example as compact as possible. The use of boost is optional - any implementation of TCP / IP sockets can be quite easily used together with the library.
As an example, for compilation we will use g ++. In our case, the command will be as follows:
As you can see, several additional parameters are passed to the compiler:
This set of options is the most common build option, which will allow you to include the library in any project with minimal effort.
You can get rid of them all if you wish. If you use the library inside the project, you do not need to specify -I $ LIBRARY_LOCATION and -D TAU_HEADERONLY . For compilers compatible with C ++ 11, the -D TAU_CPP_03_COMPATIBILITY option is not needed. Dependence on boost :: asio has only the network part, which can be quite easily rewritten without dependencies.
After compiling and starting, the server starts listening on port 12345.
We launch the client on the phone, create a connection and connect to it to display the message. Here is how it will look (I started the server on a remote computer via PuTTY, and the client started in the emulator):
This example does not provide for the transmission and receipt of additional notifications between the client and server, so let's move on to the next example.
In this example, we will add several different elements to our server, learn how to receive notifications from them, change their state and switch pages.
Server code will look like this:
All changes from the previous example were made in the MyEventsDispatcher class . The following client event handler methods have been added:
In addition, the layout sent to the client upon connection has changed accordingly.
Since we have a demo, the code in the handlers is as simple as possible - dumping information about events to the console, as well as sending various commands to the client.
All commands will be sent to the client from the button handler packetReceived_buttonClick () (of course, this is not necessary to do there, but it’s easier and more visual).
Each of the commands corresponds to a packet transmitted from the server to the client. The formation and sending of these packages occurs when the special methods defined in BasicEventsDispatcher are called :
Here's how this example works:
As you can see, for each action on the client device, the corresponding code is executed on the server. When the values of user input elements change, the server receives a notification about the new value of the variable in this element. In the button click handler, various packets are sent from the server to the client (the type of packet depends on which button was pressed). This example also shows how page switching works. Paging layouts allows you to group elements according to their functions. Only one of the pages is always displayed on the client’s screen, which reduces the load on the interface.
The last example for today is the partial implementation of one of the tasks for which I started this project. This is the simplest keyboard input emulator for windows (uses the sendInput () winapi function ).
The code for this example is on my github . I will not give it here - it does not demonstrate anything new in using the library in comparison with the second example. Here I will give only a demo of his work:
The code in this example is easy to extend to more complex keyboard emulation tasks.
Instead of a conclusion, I want to appeal to the community. Do you need a similar system? Am I inventing another bike? What needs to be described in more detail? In what direction do you need to develop further?
Now the following development paths come to my mind (they are almost completely orthogonal, so I would like to prioritize):
In addition, I will be glad to hear criticism about the architecture and implementation of the library in the state in which it is now.
References:
Foreword
Often there are situations when you want to manage your program from a phone or tablet, but it is inappropriate (or not possible) to write a separate application on the phone for this (too much work for this project, no development experience for android, etc. ) This situation periodically occurred to me, and in the end I decided to deal with this problem once and for all. The result is a system whose architecture and use are described in this article.
purpose
Creating a system that allows implementing in C ++ programs a plug-in UI based on android devices. At the same time, I wanted to minimize the dependencies of custom C ++ code on third-party libraries, as well as abstract it from the data transfer protocol. The system should consist of two parts: C ++ libraries and android applications.
System architecture
The system has a client-server architecture, where android devices act as clients, the server is a user program. Communication between them is carried out using TCP / IP sockets. To implement the communication protocol, the TAU library was created .
The main tasks for which the library is responsible:
- generation and processing of data transmitted between the server and the client
- transferring control to user code to respond to various events that occurred on the client (event UI handlers)
- creating and maintaining a connection between the server and the client
- generation of necessary data structures to describe the configuration of UI elements displayed on clients
The library consists of the following namespaces:
- tau :: communications_handling - is responsible for generating packets, parsing data received from the client, calling handlers in user code. Everything that happens between the moments of connecting and disconnecting the client is controlled by the class code from this namespace.
- tau :: layout_generation - contains functionality that allows you to create json structures that describe the location and behavior of user interface elements. This data is then sent to the client and it displays the corresponding UI.
- tau :: util - contains various auxiliary functionality, which is not required for using the library in a custom C ++ project, but is often useful. The classes in this namespace are the only ones that can use third-party libraries or non-standard compiler extensions. Therefore, here are the classes responsible for working with TCP / IP sockets - this is platform-specific code. Now there are two network communication implementations: based on boost :: asio and C ++ / CLI. Removing the implementation of all network calls outside tau :: communications_handling allows the library user to write the entire network part independently if he wishes.
- tau :: common - contains classes used from other parts of the library (they are derived here so that there are no dependencies between tau :: layout_generation and tau :: communications_handling)
Using. Example # 1 - hello, world
We will begin the demonstration of using the library with a simple example, in which a welcome message will simply be displayed on the screen. Here's what the custom code will look like in our case:
Hidden text
#include
#include
#include
class MyEventsDispatcher : public tau::util::BasicEventsDispatcher
{
public:
MyEventsDispatcher(
tau::communications_handling::OutgiongPacketsGenerator & outgoingGeneratorToUse):
tau::util::BasicEventsDispatcher(outgoingGeneratorToUse)
{};
virtual void packetReceived_requestProcessingError(
std::string const & layoutID, std::string const & additionalData)
{
std::cout << "Error received from client:\nLayouID: "
<< layoutID << "\nError: " << additionalData << "\n";
}
virtual void onClientConnected(
tau::communications_handling::ClientConnectionInfo const & connectionInfo)
{
std::cout << "Client connected: remoteAddr: "
<< connectionInfo.getRemoteAddrDump()
<< ", localAddr : "
<< connectionInfo.getLocalAddrDump() << "\n";
}
void packetReceived_clientDeviceInfo(
tau::communications_handling::ClientDeviceInfo const & info)
{
using namespace tau::layout_generation;
std::cout << "Received client information packet\n";
std::string greetingMessage = "Hello, habrahabr!";
sendPacket_resetLayout(LayoutInfo().pushLayoutPage(
LayoutPage(tau::common::LayoutPageID("FIRST_LAYOUT_PAGE"),
LabelElement(greetingMessage))).getJson());
}
};
int main(int argc, char ** argv)
{
boost::asio::io_service io_service;
short port = 12345;
tau::util::SimpleBoostAsioServer::type s(io_service, port);
std::cout << "Starting server on port " << port << "...\n";
s.start();
std::cout << "Calling io_service.run()\n";
io_service.run();
return 0;
}
The main class that contains all the user logic that interacts with the client device ( MyEventsDispatcher ) should be inherited from tau :: util :: BasicEventsDispatcher . It overrides 2 methods from the base class: onClientConnected () and packetReceived_clientDeviceInfo () . The first is called when the client connects. The second method will be executed when information about the client device comes after the connection to the server (the first packet after the connection is sent by the client).
In our case, the first method is trivial - it only displays an informational message to the console. In the second method, the server sends a layout to the client - data about which interface should be displayed on the client.
All code responsible for transmitting data over the network is in main (). In this case, the boost :: asio library is used to implement communication . There are corresponding abstractions in the tau :: util namespace , which makes this example as compact as possible. The use of boost is optional - any implementation of TCP / IP sockets can be quite easily used together with the library.
Compilation
As an example, for compilation we will use g ++. In our case, the command will be as follows:
g++ -lboost_system -pthread -lboost_thread -D TAU_HEADERONLY -D TAU_CPP_03_COMPATIBILITY -I $LIBRARY_LOCATION main.cpp -o demoAs you can see, several additional parameters are passed to the compiler:
- include path to library sources (option -I $ LIBRARY_LOCATION )
- additional libraries needed for boost :: asio (options -lboost_system -pthread -lboost_thread )
- declarations of additional macros indicating how we compile the library in our project ( -D TAU_HEADERONLY -D TAU_CPP_03_COMPATIBILITY )
This set of options is the most common build option, which will allow you to include the library in any project with minimal effort.
You can get rid of them all if you wish. If you use the library inside the project, you do not need to specify -I $ LIBRARY_LOCATION and -D TAU_HEADERONLY . For compilers compatible with C ++ 11, the -D TAU_CPP_03_COMPATIBILITY option is not needed. Dependence on boost :: asio has only the network part, which can be quite easily rewritten without dependencies.
After compiling and starting, the server starts listening on port 12345.
We launch the client on the phone, create a connection and connect to it to display the message. Here is how it will look (I started the server on a remote computer via PuTTY, and the client started in the emulator):
Creating a server connection
This example does not provide for the transmission and receipt of additional notifications between the client and server, so let's move on to the next example.
Example No. 2 - a more detailed demonstration of the capabilities of the system
In this example, we will add several different elements to our server, learn how to receive notifications from them, change their state and switch pages.
Server code will look like this:
Hidden text
#include
#include
#include
namespace {
std::string const INITIAL_TEXT_VALUE("initial text");
tau::common::LayoutID const LAYOUT_ID("SAMPLE_LAYOUT_ID");
tau::common::LayoutPageID const LAYOUT_PAGE1_ID("LAYOUT_PAGE_1");
tau::common::LayoutPageID const LAYOUT_PAGE2_ID("LAYOUT_PAGE_2");
tau::common::ElementID const BUTTON_WITH_NOTE_TO_REPLACE_ID("BUTTON_WITH_NOTE_TO_REPLACE");
tau::common::ElementID const BUTTON_TO_RESET_VALUES_ID("BUTTON_TO_RESET_NOTES");
tau::common::ElementID const BUTTON_TO_PAGE_1_ID("BUTTON_TO_PG1");
tau::common::ElementID const BUTTON_TO_PAGE_2_ID("BUTTON_TO_PG2");
tau::common::ElementID const BUTTON_1_ID("BUTTON_1");
tau::common::ElementID const BUTTON_2_ID("BUTTON_2");
tau::common::ElementID const BUTTON_3_ID("BUTTON_3");
tau::common::ElementID const BUTTON_4_ID("BUTTON_4");
tau::common::ElementID const TEXT_INPUT_ID("TEXT_INPUT");
tau::common::ElementID const BOOL_INPUT_ID("BOOL_INPUT");
tau::common::ElementID const LABEL_ON_PAGE2_ID("LABEL_ON_PAGE2");
};
class MyEventsDispatcher : public tau::util::BasicEventsDispatcher
{
public:
MyEventsDispatcher(
tau::communications_handling::OutgiongPacketsGenerator & outgoingGeneratorToUse):
tau::util::BasicEventsDispatcher(outgoingGeneratorToUse)
{};
virtual void packetReceived_requestProcessingError(
std::string const & layoutID, std::string const & additionalData)
{
std::cout << "Error received from client:\nLayouID: "
<< layoutID << "\nError: " << additionalData << "\n";
}
virtual void onClientConnected(
tau::communications_handling::ClientConnectionInfo const & connectionInfo)
{
std::cout << "Client connected: remoteAddr: "
<< connectionInfo.getRemoteAddrDump()
<< ", localAddr : "
<< connectionInfo.getLocalAddrDump() << "\n";
}
virtual void packetReceived_clientDeviceInfo(
tau::communications_handling::ClientDeviceInfo const & info)
{
using namespace tau::layout_generation;
std::cout << "Received client information packet\n";
LayoutInfo resultLayout;
resultLayout.pushLayoutPage(LayoutPage(LAYOUT_PAGE1_ID,
EvenlySplitLayoutElementsContainer(true)
.push(EvenlySplitLayoutElementsContainer(false)
.push(BooleanInputLayoutElement(true).note(INITIAL_TEXT_VALUE).ID(BOOL_INPUT_ID))
.push(ButtonLayoutElement().note(INITIAL_TEXT_VALUE)
.ID(BUTTON_WITH_NOTE_TO_REPLACE_ID)))
.push(TextInputLayoutElement().ID(TEXT_INPUT_ID).initialValue(INITIAL_TEXT_VALUE))
.push(EmptySpace())
.push(EmptySpace())
.push(EmptySpace())
.push(EvenlySplitLayoutElementsContainer(false)
.push(ButtonLayoutElement().note("reset notes").ID(BUTTON_TO_RESET_VALUES_ID))
.push(EmptySpace())
.push(ButtonLayoutElement().note("go to page 2").ID(BUTTON_TO_PAGE_2_ID)
.switchToAnotherLayoutPageOnClick(LAYOUT_PAGE2_ID))
)
)
);
resultLayout.pushLayoutPage(LayoutPage(LAYOUT_PAGE2_ID,
EvenlySplitLayoutElementsContainer(true)
.push(EvenlySplitLayoutElementsContainer(false)
.push(ButtonLayoutElement().note("1").ID(BUTTON_1_ID))
.push(ButtonLayoutElement().note("2").ID(BUTTON_2_ID)))
.push(EvenlySplitLayoutElementsContainer(false)
.push(ButtonLayoutElement().note("3").ID(BUTTON_3_ID))
.push(ButtonLayoutElement().note("4").ID(BUTTON_4_ID)))
.push(EvenlySplitLayoutElementsContainer(true)
.push(LabelElement("").ID(LABEL_ON_PAGE2_ID))
.push(ButtonLayoutElement().note("back to page 1").ID(BUTTON_TO_PAGE_1_ID)))
));
resultLayout.setStartLayoutPage(LAYOUT_PAGE1_ID);
sendPacket_resetLayout(resultLayout.getJson());
}
virtual void packetReceived_buttonClick(
tau::common::ElementID const & buttonID)
{
std::cout << "event: buttonClick, id=" << buttonID << "\n";
if (buttonID == BUTTON_TO_RESET_VALUES_ID) {
sendPacket_updateTextValue(TEXT_INPUT_ID, INITIAL_TEXT_VALUE);
} else if (buttonID == BUTTON_TO_PAGE_1_ID) {
sendPacket_changeShownLayoutPage(LAYOUT_PAGE1_ID);
} else if (buttonID == BUTTON_1_ID) {
sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 1 pressed");
} else if (buttonID == BUTTON_2_ID) {
sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 2 pressed");
} else if (buttonID == BUTTON_3_ID) {
sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 3 pressed");
} else if (buttonID == BUTTON_4_ID) {
sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 4 pressed");
}
}
virtual void packetReceived_layoutPageSwitched(
tau::common::LayoutPageID const & newActiveLayoutPageID)
{
std::cout << "event: layoutPageSwitch, id=" << newActiveLayoutPageID << "\n";
}
virtual void packetReceived_boolValueUpdate(
tau::common::ElementID const & inputBoxID,
bool new_value, bool is_automatic_update)
{
std::cout << "event: boolValueUpdate, id="
<< inputBoxID << ", value=" << new_value << "\n";
}
virtual void packetReceived_textValueUpdate(
tau::common::ElementID const & inputBoxID,
std::string const & new_value, bool is_automatic_update)
{
std::cout << "event: textValueUpdate, id="
<< inputBoxID << ",\n\tvalue=" << new_value << "\n";
sendPacket_changeElementNote(BOOL_INPUT_ID, new_value);
sendPacket_changeElementNote(BUTTON_WITH_NOTE_TO_REPLACE_ID, new_value);
}
};
int main(int argc, char ** argv)
{
boost::asio::io_service io_service;
short port = 12345;
tau::util::SimpleBoostAsioServer::type s(io_service, port);
std::cout << "Starting server on port " << port << "...\n";
s.start();
std::cout << "Calling io_service.run()\n";
io_service.run();
return 0;
}
All changes from the previous example were made in the MyEventsDispatcher class . The following client event handler methods have been added:
- packetReceived_buttonClick button event handler . The button ID is passed to the method as a parameter.
- handlers for packets that transfer variable values from client to server: packetReceived_boolValueUpdate, packetReceived_intValueUpdate, packetReceived_floatPointValueUpdate, packetReceived_textValueUpdate
- handler of event of change of the displayed page with packetReceived_layoutPageSwitched elements
In addition, the layout sent to the client upon connection has changed accordingly.
Since we have a demo, the code in the handlers is as simple as possible - dumping information about events to the console, as well as sending various commands to the client.
All commands will be sent to the client from the button handler packetReceived_buttonClick () (of course, this is not necessary to do there, but it’s easier and more visual).
Each of the commands corresponds to a packet transmitted from the server to the client. The formation and sending of these packages occurs when the special methods defined in BasicEventsDispatcher are called :
- sendPacket_resetLayout () - replacement of the entire layout
- sendPacket_requestValue () - request for a variable value in one of the inputs
- sendPacket_updateBooleanValue (), sendPacket_updateIntValue (), sendPacket_updateFloatPointValue, sendPacket_updateTextValue () - change the value of variables in inputs
- sendPacket_changeElementNote () - change any read-only text (text on buttons, checkboxes, labels)
- sendPacket_changeShownLayoutPage () - switch to another page with elements
- sendPacket_changeElementEnabledState () - switch the active status of elements (inactive elements are displayed, but they cannot be interacted with)
Here's how this example works:
Demonstration of work with UI elements
As you can see, for each action on the client device, the corresponding code is executed on the server. When the values of user input elements change, the server receives a notification about the new value of the variable in this element. In the button click handler, various packets are sent from the server to the client (the type of packet depends on which button was pressed). This example also shows how page switching works. Paging layouts allows you to group elements according to their functions. Only one of the pages is always displayed on the client’s screen, which reduces the load on the interface.
Example No. 3 - something useful
The last example for today is the partial implementation of one of the tasks for which I started this project. This is the simplest keyboard input emulator for windows (uses the sendInput () winapi function ).
The code for this example is on my github . I will not give it here - it does not demonstrate anything new in using the library in comparison with the second example. Here I will give only a demo of his work:
Keyboard Emulation
The code in this example is easy to extend to more complex keyboard emulation tasks.
Epilogue
Instead of a conclusion, I want to appeal to the community. Do you need a similar system? Am I inventing another bike? What needs to be described in more detail? In what direction do you need to develop further?
Now the following development paths come to my mind (they are almost completely orthogonal, so I would like to prioritize):
- adding client applications for other platforms (IOS, PC)
- expanding the functionality of the data transfer protocol (heartbeat packets for connection monitoring, communication control commands)
- adding new UI elements (drop-down boxes, images, etc)
- deeper customization of the appearance of elements on the client (colors, fonts, styles)
- support for more specific client device functions (notifications, sensors, volume buttons, etc)
- adding server libraries for other programming languages
In addition, I will be glad to hear criticism about the architecture and implementation of the library in the state in which it is now.
References: