bobaos.pub - KNX TP / UART, Raspberry Pi and Redis


    There is no limit to perfection. It would seem that everything worked well, minor bugs were fixed and so on.


    Now I will tell, firstly, about the problems that I faced during all the time that has passed since the previous article, and, secondly, about the decisions that contributed to the current status of the project.


    → Article about the previous version


    Legend


    bobaos- npm module for interacting with BAOS 83x using the UART. Returns raw data. Used in all other modules listed below.


    bdsd.sock- script to work with KNX objects. Stores the list of datapoints, when sending / receiving converts values. From DPT1 to true / false, from DPT9 to float. Also listens to Unix Socket for receiving requests from other processes.


    bobaos.pub- A new version that uses redisfor interprocess communication.
    KNX объект/датапоинт- the BAOS 83x module communication object configured in ETS to which the group address (s) corresponds (or not). In current versions of iron, the maximum amount is 1000.


    Task


    The main task is the same as the previous version. A connection to a serial port can only open one. Scripts working with KNX, I want to run a lot. In addition to this, I wanted to implement interprocess communication. Those. so that not only one process bdsd.socklistened to a socket, but each running script could both send and receive requests.


    Idea


    An idea was born in my head to make my message broker on node.js on top of Unix sockets, to which clients would connect, subscribe to topics and receive / send messages according to the code written in them. I knew that there were ready-made solutions, about which only lazy people hadn’t heard recently, I studied, but the idea of ​​making my decision was intrusive.


    And in the end the service started.


    Wrote a logger who sends messages to the topic. Subscribers receive and are free to do anything, or rather what is written. Conveniently - logs from several sources can be viewed in one console output.


    I wrote, posted to npm package bobaos.pub, which, unlike bdsd.sock, no longer creates a socket file, but connects to a broker. At first glance, everything works as it should.


    Problem


    Then I launched a script that periodically sends requests to the KNX bus for the night. Waking up in the morning, by the blinking of the LEDs signaling the sending / transfer of data, I realized that something was wrong. Messages did not reach on time. Found out that the samopisny message broker took up almost all of the RAM available from 512MB (from BeagleBoard Black). Further work with nodejs confirmed that memory is the weak point of js scripts.


    Decision


    As a result, it was decided to switch from samopisny Unix sockets to Redis (by the way, he also knows how to work with them). Perhaps it was worth figuring out the memory, finding the leaks, but I wanted to run faster.


    bobaos means communicating over UART with wrapping messages in FT1.2, our communication is synchronous. Those. <..сообщение-подтверждение-ответ..>. The bobaos module, which is responsible for communication, stores all requests in the array, pulls out from there one by one, sends it to the UART, and when an incoming response is resolved, the promise responsible for this request is allowed.


    You can go the following way: the service listens to the PUB / SUB channel redis, accepts requests, sends to KNX. In this case, the load on the request queue falls on the js module bobaos. To implement, you need to write a simple module, subscribed to the channel and convert the message method JSON.parse(). Further, this module can be used in other scripts.


    Another option, which eventually stopped: use the existing task manager on top redis. There are several choices made on bee-queue.


    → Under the hood


    Here is described how the queue works bee-queue. If you implement this library for other programming languages, you can thus make client libraries and for bobaos.


    In the second version, all requests are stored in lists redis, are pulled out in turn and sent to the serial port.


    Next, the previous version should be rewritten, but I already keep all the data on datapoint in redisthe database. The only inconvenience that I feel is that all requests are asynchronous, respectively, getting an array of values ​​is a little harder than just turning to an array.


    Small optimizations have been made.


    If earlier there were separate methods getValue/getValues/readValue/readValues/setValue/setValues, now getValue/readValue/setValuesthey take both one value and an array.


    The method getValue([id1, id2, ...])in the previous version sent a request to the serial port for each datpoint. But it is possible to send a request for several values. Restrictions - the size of the response should be equal BufferSize, the maximum - 250 bytes; also it is impossible to go beyond the number of objects, for current versions of BAOS 83x modules - 1000.


    The lengths of the values ​​are known, the title too. Further, a rather simple algorithm with while loops and awaits =)


    1. Sort the array, delete duplicate elements, if any. get the array idUniq.
    2. We start a cycle i < idUniq.lengthin which we do the following:
      a) We take start: idUniq[i], for it we count the maximum number of values ​​that we can get. For example, if all objects are of type DPT1 / DPT5 (1 byte), then we can get values ​​in the amount of 48. There is one note: if, for example, we have configured objects in ETS #[1, 2, 3, 10, 20], then when queried GetDatapointValue.Req(1, 30), zero one-byte values will be returned values ​​even for non-existent data points [4, 5, 6, ..., 30].
      b) Counting occurs in a new cycle j < i + max(where maxis 50, or, if close to 1000, then the maximum 1000 - id + 1, for 990 it will be 11, for 999 - 2), if in the counting process we meet the elements of the array from the original query, then we iassign the variable its index .
      c) If in a cyclejthe calculated length is greater than the maximum length of the buffer, then we form the element of the "map" of requests {start: start, number: number}, throw it into a separate array, increase the variable i(or assign the index found in the array idUniq), interrupt the cycle j, both cycles are restarted.

    Thus, we form several requests for bobaos. For example, if you send a request getValue([1, 2, 3, 40, 48, 49, 50, 100, 998, 999, 1000]), then the requests may be for the particular case of the following:


    {start: 1, number: 48}, // 1, 2, 3, 40, 48
{start: 49, number: 48}, // 49, 50
{start: 100, number: 48}, // 100
{start: 998, number: 3} // 998, 999, 1000

    It could be done differently: 


    {start: 1, number: 48}, // 1, 2, 3, 40, 48
{start: 49, number: 2}, // 49, 50
{start: 100, number: 1}, // 100
{start: 998, number: 3} // 998, 999, 1000

    Requests would be the same, given less. But I stopped at the first option, since the values ​​obtained are saved to the database redis, respectively, they can be obtained by a method getStoredValuethat I try to use more often than by getValuesending data on a serial port.


    For service methods ping/get sdk state/reseta separate queue is created. Thus, if something is wrong with the serial port communication (the frame counter was lost, etc.) and the execution stopped for some task, you can send a request resetto another queue, and restart accordingly sdk.


    Client part - bobaos.sub


    A bobaos.submodule can be used to manage KNX data points in user scripts .


    The following example covers all module functions:


    const BobaosSub = require("bobaos.sub");
// в параметры можно передать опционально:
// redis: объект либо url
// request_channel: "bobaos_req" по умолчанию, 
// service_channel: "bobaos_service" по умолчанию,
// broadcast_channel: "bobaos_bcast" по умолчанию
let my = BobaosSub();
my.on("connect", _ => {
  console.log("connected to ipc, still not subscribed to channels");
});
my.on("ready", async _ => {
  try {
    console.log("hello, friend");
    console.log("ping:", await my.ping());
    console.log("get sdk state:", await my.getSdkState());
    console.log("get value:", await my.getValue([1, 107, 106]));
    console.log("get stored value:", await my.getValue([1, 107, 106]));
    console.log("get server item:", await my.getServerItem([1, 2, 3]));
    console.log("set value:", await my.setValue({id: 103, value: 0}));
    console.log("read value:", await my.readValue([1, 103, 104, 105]));
    console.log("get programming mode:", await my.getProgrammingMode());
    console.log("set programming mode:", await my.setProgrammingMode(true));
    console.log("get parameter byte", await my.getParameterByte([1, 2, 3, 4]));
    console.log("reset", await my.reset());
  } catch(e) {
    console.log("err", e.message);
  }
});
my.on("datapoint value", payload => {
  // Имейте в виду, что payload может быть как объектом, так и массивом
  // так что необходима проверка Array.isArray(payload)
  console.log("broadcasted datapoint value: ", payload);
});
my.on("server item", payload => {
  // Имейте в виду, что payload может быть как объектом, так и массивом
  // так что необходима проверка Array.isArray(payload)
  console.log("broadcasted server item: ", payload);
});
my.on("sdk state", payload => {
  console.log("broadcasted sdk state: ", payload);
});

    bobaos.tool


    The command line interface has been rewritten. About how I implemented it the following article:


    → We write CLI on NodeJS


    The teams have become shorter, clearer and more functional.


    bobaos> progmode ?
BAOS module in programming mode: false
bobaos> progmode 1
BAOS module in programming mode: true
bobaos> progmode false
BAOS module in programming mode: false
bobaos> description 1 2 3
#1: length = 2, dpt = dpt9, prio: low, flags: [C-WTU]
#2: length = 1, dpt = dpt1, prio: low, flags: [C-WT-]
#3: length = 1, dpt = dpt1, prio: low, flags: [C-WT-]
bobaos> set 2: 0
20:27:06:239,    id: 2, value: false, raw: [AA==]
bobaos> set [2: 0, 3: false]
20:28:48:586,    id: 2, value: false, raw: [AA==]
20:28:48:592,    id: 3, value: false, raw: [AA==]

    Afterword


    It turned out working stable system. Redisas a backend, it works stably well. During the development, a lot of cones were stuffed. But the learning process is such that sometimes it is inevitable. From the experience gained, I note that the nodejsprocesses consume quite a lot of RAM (20MB at the start) and there may be leaks. For home automation, this is critical - because the script should work all the time, and if it grows over time more, then by a certain moment it can take up all the space. Therefore, you need to carefully write scripts, understand how the garbage collector works and keep everything under control.


    Updated documentation can be found here .


    In the next article I will talk about how with the help redisand bee-queuemade the service for software accessories.


    UPD: bobaoskit - accessories, dnssd and websocket


    I would appreciate any feedback.

    Tags:

    Also popular now: