Controller "Virt2real", or simply - Virtualrilka
“You drank beer,” Joe spoke quietly. - The can opener worked poorly.
You said you made a can opener yourself, bigger and better. This is me.
Henry Kuttner, The Know-It-All Robot
UPD> The collection of pre-orders began habrahabr.ru/post/163865
Who would have thought that with our own hands we can develop such a beauty and good girl. Not only that - not just develop on paper, but also get a dozen working samples.
A long time ago, a colleague and I began to manage all kinds of gizmos such as webcams and small cars via the Internet . A little later, we got more serious driving devices, capable of carrying their controller and capacious battery on board. In addition, I was carried away by all sorts of flying gizmos - airplanes and copters, which made an additional contribution to the list of Wishlist.
The idea that it would be nice to make a video not a “side or top view”, but a view from the side of the device, in the first person, did not leave. The fact that the airplanes / characteristics are called FPV. It was about a couple of years ago. They tried to put an analog camera and a transmitter on a typewriter, received video on a TV tuner, then gave it to a video server for distribution by taxis. But immediately nuances surfaced - at the same time, only 3-4 video streams can be broadcast, otherwise pickups from one channel to another appear. Plus the high cost of such a solution, plus problems with the allowed frequencies, plus the lousy picture quality with constantly appearing noise ... In general, we tried and scored it. In addition, the capabilities of the control controller (then everything was done on the basis of the well-known Arduinka) began to be lacking. They began to look for other options, so that it was inexpensive, cheap, convenient and practical. But commercially available (abroad, by itself and with a rather high cost) at that time pieces of iron for the needs did not fit. Some were able to do everything except video, others only knew how to video, but they did not allow connecting peripherals, etc. etc.
We looked for suitable controllers for sale in Russia - there are in small quantities and for wild money. With double or even triple wrap. And we do not need one or two pieces, we need a lot of such controllers. And the wallet is not rubber. And somehow I did not want to indulge the price lawlessness of reselling intermediaries.
We tried to order a debug board in order to make a device on it that could control the machine and transmit video. But they ran into a problem - foreign stores do not want to send controllers to Russia, since they are on list A - dual-use goods. That is, devices that can be used for both peaceful and military purposes. At first glance - complete nonsense, but in fact it is. All the controllers that suit us in terms of functionality are on that ill-fated list. I personally called Farnell and had a fascinating half-hour conversation with a sweet (hopefully) girl who, with a thousand apologies, said that they could not send us the required piece of iron (if I am not mistaken, it was a BeagleBoard). It was she who said that the EU sacredly observes all the rules, so that they just can’t send, and to send them you need to receive a certificate for a piece of iron, and the certificate is valid for a limited time and is obtained for each customer. The term for obtaining a certificate is about a month, and it is not at all a fact that it will be received. In addition, the customer (i.e. me, in this case) will need a handwritten document certified by a notary, where there will be a full description of the planned use of the piece of iron. I was about to write such a document, but the girl asked me not to strain them in vain, because the probability of getting a positive result is very insignificant, and having such a problem ... having such problems because of the unit sale is absolutely unprofitable for them. me, in this case) you will need a handwritten document certified by a notary, where there will be a full description of the planned use of the piece of iron. I was about to write such a document, but the girl asked me not to strain them in vain, because the probability of getting a positive result is very insignificant, and having such a problem ... having such problems because of the unit sale is absolutely unprofitable for them. me, in this case) you will need a handwritten document certified by a notary, where there will be a full description of the planned use of the piece of iron. I was about to write such a document, but the girl asked me not to strain them in vain, because the probability of getting a positive result is very insignificant, and having such a problem ... having such problems because of the unit sale is absolutely unprofitable for them.
After this unsuccessful attempt, we tried to order pieces of iron from the states. The piece of iron was called LeopardBoard. She was sent to us without unnecessary hassle, but problems suddenly surfaced at our, Russian, customs. A piece of paper came from the customs officers by mail, where they demanded that we prove that the coveted piece of iron did not have strong cryptography modules or anything else on board (I don’t remember exactly what, sorry). A colleague personally went to customs, but they failed to explain anything worthwhile - the piece of iron lay at customs for about a month, after which he left back to the sender.
After such ordeals, we finally got angry with the whole world and began to think what to do.
One of the rainy autumn days, the same St. Petersburg colleague came to visit me, with whom they sat in the kitchen and, drinking beer, started an intimate conversation about who came up with something interesting to solve our problem with choosing iron. When it was already drunk quite a lot, a colleague uttered a thought that at first seemed wild. And why, he said, do not make our own
The rest of the evening went into frenzied fantasies about how many different interesting controlled things can be done by hiring a suitable controller ... A
list of requirements for the piece of hardware has been formulated:
- Low price of the final device!
- A powerful processor that knows how to hardware-compress video in H264 (i.e., having the appropriate DSP)
- Interface for connecting a digital camera
- USB-host for connecting peripherals (3G / 4G modems are especially relevant)
- Low power consumption
- Wide power supply range
- Small dimensions (so that you can install it in a small machine, for example)
- Linux on board (a large number of software already developed)
- Wireless communication on board (Bluetooth was not considered, only Wi-Fi)
- Wired connection (Ethernet)
- PWM hardware generator
- Large number of pins per water-output, for connecting a variety of peripherals
- Composite video output (for video transmission using analog video transmitters)
Some time has passed. That wild idea expressed by the beer began to be implemented slowly. We chose the right SoC with DSP on board, able to process and reload video in h264 on the fly with an acceptable delay. Which, in addition, has a bunch of everything in its composition (I / O pins, or GPIO, plus all kinds of interfaces). This is the percent of the Texas Instruments DaVinci family - TMS320DM365. Digital Media System-on-Chip (DMSoC). www.ti.com/product/tms320dm365 The choice was very influenced by the price of the processor, its capabilities, as well as the scope of its application - it is very widely used in IP cameras, DVRs, multimedia players, etc. etc.
Next, it remains “only” to develop a board based on this processor. At this stage, we stalled slightly, but soon found our man who agreed to do the development.
At this time (summer 2011) we first heard the mysterious word "Raspberry PI" :-) The first announcement of this piece of iron on Habré was a little worrying, because the bourgeois announced the development of a device that was very close to what we had been looking for a long time, and then decided do it yourself. But, having read everything about this Malinka, they calmed down - the piece of iron, although it is pretty close to ours, is not suitable for us in practical use. For RaspberryPi, nevertheless, is more focused on desktop applications, although it has the ability to connect peripherals for DIY applications. In addition, the price, taking into account delivery, is quite high, given the need to add Wi-Fi, a camera, etc. Around the same time, other similar glands appeared (Beagleboard, Beaglebone, Pandaboard, etc.), but none of the glands fully suited us. It gave us strength
By the way, when I once again called Farnell about the Raspberry Pi, I asked them why the BeagleBone is on the list of export restrictions, but Raspberry is not. They really didn’t explain anything to me, but confirmed - Raspberries are not on this list. Perhaps not yet :-)
The development process has been pretty fun. Just the poet’s dream. You sit, think up what you want it to be on board, and Alexander (a colleague who is directly involved in the circuitry of the piece of iron) either says “easy, damn it” and adds the functionality just invented to the circuit, or answers that “it won’t come out very expensive” and offers another option.
All development took place in free time from the main work (i.e., in the evenings-nights) and with our own money. More precisely, with the money of one of our colleagues, who, as far as possible, financed the entire project and our entire team. So in all seriousness, you can sculpt the sticker “Designed and Made in Russia” on the piece of iron. True, there is nothing to be done about imported parts. Unfortunately, we did not find the necessary microprocessors and other parts that were produced in Russia.
So, after some time we got a complete set of documentation for the production of hardware. By that time, the name had already been invented. I was driving in a car, listening to the radio, thinking about a piece of iron, about cars and how not to crash into a lady on a yellow beh, who very insidiously cut me at the intersection at the XXL shopping center. And on the radio “Echo of Moscow” the discussion of the next opposition rally was just going on and the presenter said the phrase “welcome to reality”. And somehow, a combination came to my mind - virtuality into reality, virt into ril, virt2real. He immediately phoned his colleague in Peter, who approved the name and who also just came up with something similar. In general, the name immediately took root. And the piece of iron, respectively, began to be called a "virtualka".
For debugging purposes, it was decided to produce ten samples. Which was done at one of the Moscow enterprises. In fact, the first time we made a sample as close as possible to the serial.
Virt2real assembled - two floors of
Virt2real disassembled. A small scarf - for debugging purposes, in the final form, the microSD and USB slot will be on the main board.
Disassembled sandwich, another angle. You can’t imagine the
first floor with chips outside
(well, with the exception of those people who often develop devices) what it is to hold in your hands the finished board, which was invented a year ago, sitting in the kitchen and drinking beer. This is buzz, delight and pride to the nth degree. Even if this is just a sample in which a couple of jambs have already been found.
In general, the process of catching bugs followed. Although, I must say, Alexander did a great job - just a couple of jambs with such a volume of iron is an excellent result. Moreover, the jambs turned out to be eliminated quite quickly. Unfortunately, there is no video showing a hole being drilled with a small cutter to the third layer of the board (there are 10 layers in total on the board) in order to swap the lines of the differential pair. It was epic :-)
Gradually, as I was picking with the virtual machine, more minor nuances surfaced that did not affect the quality of operation, but if they were done a little differently, it would be more convenient. Such minor corrections were postponed to the next stage - a small series in which all the catching shortcomings and inconveniences could be fixed right away.
The most difficult and unpleasant part of the development was waiting for us where, frankly, they did not really wait for it. The problem was that none of the team had ever been involved in porting Linux to various hardware platforms. That is, in general, we knew how to do this, even had the experience of writing our own OS, for our specific tasks, WinCE was again ported, but somehow fate didn’t reduce Linux to this time. But it turns out there are good Linux builds, if bad ... To revitalize the board, we used the RidgeRun SDK. This SDK is distributed on a paid basis, but for non-commercial applications it can be used for free. Software was required to ensure work with the equipment and generally see what it was capable of. We liked RidgeRun because there is support for a real board on DM365 (Leopard Board), Support for hardware codecs and a pretty good file system are included. But as usual, the devil is in the details (for details, you can refer to the classic workfreeworld.thc.org/root/phun/unmaintain.html ). It turned out that this SDK by the myriad of crutches and props is sharpened precisely for the piece of iron for which it was originally created - LeopardBoard. A step to the left - a step to the right and everything falls apart so that then it is not even going to. Moreover, the execution of make clean (attention !!!) leads to the fact that not only the traces of the compiler’s vital activity are deleted, but the source codes are also reloaded from the RidgeRun repository - forgive the changes. Apparently, it didn’t occur to the SDK developers that for some reason someone might need to edit their code. By a strange coincidence, it took us 90% of the time to fight against the SDK instead of implementing what we want. However, in the end, the SDK could not resist the pressure and the piece of iron worked.
In general, the result of porting the RidgeRun SDK was the decision to develop your own SDK for our platform.
And at this stage we are now - the piece of iron can be put into production even tomorrow, but with the SDK, not everything is as we would like. That, however, does not affect usability - the virtual machine already perfectly steers a cart and a boat (see the video at the end of the topic) :-)
In terms of the use by the end user-self-seeker, we assume three levels:
1. Beginner - a piece of iron with our firmware, SSH access and web admin. The user can upload their scripts in high-level languages (PHP, Node.js, Perl, Python, Erlang) or ready-made cross-copied executable applications.
2. Advanced - everything is the same as for a beginner, plus an SDK for building your own and any third-party applications
3. Pros - uses third-party distributions and SDKs (Arago / OE / Angstrom / other)
Now on board are available high-level programming languages from the usual gentleman's set - PHP, Node.js, Perl, Python, Erlang. I’m silent about Bash. This kit is enough to meet 80% of the needs in terms of driving gizmos. The remaining 20% is C / C ++, but cross-compilation is required.
Almost all of the controller’s functionality is available from the “admin panel” - a self-written control web interface. Convenient - no need to climb into the console.
The prologue is over, now the most interesting thing is the description of the board.
The virt2real device, Virtualka, is based on a Texas Instruments processor from the DaVinci TMS320DM365 family. You can install a more powerful modification, TMS320DM368 (400Mhz), they are fully compatible on the legs.
Since it was originally planned to use controlled glands on board, they did not mount a digital video output on the board, although this allows for a percent (HDMI). Perhaps we will add in a specialized version of the board. But on board there is an analog video output, i.e. it is possible to transmit video using an analog video transmitter, which is quite relevant when controlling over long distances.
Mounting plate (controller itself)
The device is two-story.
The first floor is the main board on which all the main modules are mounted. The board has ten layers.
Second floor - expansion board. It is developed separately for each specific application, in the case of serial production of a particular product. At the moment, the second floor is common for DIY applications. The signals GPIO, RG45 are output to it. Power chips are mounted and terminal blocks for connecting the power load - motors, light bulbs, etc. are separated from them.
Dimensions: 70h42h10mm (first floor)
ARM9 210-300Mhz (ARM926EJ-S rev 5) On-Chip L1 Cache 32 KB (ARM9)
ARM Jazelle Technology
cache Displacement commands - 16K
volume data cache - 8KB
volume integrated in processor RAM 32KB
Two video coprocessors (HDVICP, MJCP)
Real-time clock PRTCSS (Power Management and Real Time Clock Subsystem)
Four 64-bit general-purpose timers
One 64-bit Watch Dog timer
The processor is made using 65nm technology
Extended temperature range (-40 ° C - 85 ° C) (for the processor version in the industrial version), for the usual
Hardware acceleration MJCP and HDVICP
Hardware support H.264, MPEG4, MPEG2, MJPEG, JPEG, WMV9 / VC1
Resolution of encoded video D1 and less, 720p (1080p for DM368) 30 fps
4: 2: 2 (8- / 16-bit)
8- / 16-bit YCC to 24-bit RGB888 digital output
Hardware On-Screen Display (OSD)
ISIF (Image Sensor Interface) and CMOS Imager Interface
Lens distortion correction module (LDC)
Hardware 3A statistics collection module (H3A)
Composite NTSC / PAL video output
Component video output (the corresponding signals are displayed on the second floor, but have to implement the final support)
LCD (supported by the processor, but not used in the current version of the board. Will involved in future versions)
DDR2 128 - 256Mb
Interfaces and peripherals
GPIO - 51 pcs. Logic levels 3.3V and 1.8V
PWM - 4 pcs.
SPI - 5 pcs.
UART - 2 pcs.
DAC - 3 pcs.
ADC - 6 pcs.
USB 2.0 High-Speed host port
Analog component video output
Composite analog video output (maximum 480p)
CQ93VC audio codec (AAC-LC, G.711, MP3, WMA)
Audio line output (mono)
Speaker audio output (mono, 300 mV at 8 ohm load) 10 / 100Mb RJ45 Ethernet
analog microphone audio input Input for Wi-Fi b / g digital camera (HDG104 chip) Power load connectors (motors) (3 L293D microcircuits) McBSP interface (Multi-Channel Buffered Serial Port) Camera Currently tested OV7690, OV7675 (8-bit bus), generally We should work with any bus 8/10 bits, in the presence of a corresponding camera driver Nutrition
2.8 - 15 Volts. Current consumption (main board with wi-fi turned on) at voltage> 5V - 220mAh. When the supply voltage is less than 5V, the current consumption is greater. The power supply of the circuit is made according to the SEPIC circuit, which can either increase or decrease the supply voltage in order to provide the necessary voltages in the circuit, for example, when powered from 3 volts, the board will still give the required 5V to USB.
In general, the second floor was conceived so as to make it easy to implement the specifics of a variety of user applications. All kinds of supply voltages and interfaces are brought to the second floor, despite the fact that standard PLD connectors with a pitch of 2.54 mm are used. If you really want, a person can, at home, design, draw an etch and solder a board to solve his problem - there are practically no requirements for it. And he will receive a compact device with a powerful processor to solve his problems. You can, on the contrary, integrate the board into your own development. All necessary information will be provided. No need to go into the wilds about how the processor and its peripherals are arranged. Linux
SDK RidgeRun (currently), MontaVista, Arago / OE, Angstrom,
Remembering the clever saying that it’s better to see once than to hear a hundred times, he shot several demos showing some aspects of using the virtual machine. Of course, this is just a drop in the ocean of all possible options, but at least something for the seed.
So, video No. 1 - General review of virt2real device (or Virtualrilki)
Video No. 2 - The Most Important Video. All arduino-miners should break a tear of nostalgia, because the blinking of a LED is photographed here.
Video No. 3 - Installing a virtualka on a tracked platform. Part 1.
By the way, the chassis of the cart is very good. It is called DAGU RP5, you can buy here (MSC). It’s never an advertisement, it’s just that I ordered several small things in this store several times, they bring it quickly and without too much hassle. Although, in principle, yes - this is my free advertisement to them :-) Honestly - respect to the store, everyone would work like that.
Video No. 4 - Installing a virtualka on a tracked platform. Part 2.
Video # 5 - Crawler platform driven by Virt2real in action
Video No. 6 - view from the side of the cart from the previous video
Video No. 7 - installation of a virtualka on board a radio-controlled boat in order to make control of it from a smartphone (of course, from video from the board)
Video No. 8 is a continuation of the last video about installing a piece of iron on a boat.
Video No. 9 - Offline broadcast to a video server
Video No. 10 - Using a 3G modem and autonomous video broadcast (via RTMP) to a video server
Video No. 11 - Virturica as an analog camera (composite video output is used)
Video No. 12 - Again Virtualka as an analog camera, but this time with OSD. Those. with a separate layer on top of the video
A little explanation: since the virtual machine was primarily conceived as a controller for controlling traveling / flying / floating devices, there is no digital video output on board. Those. You cannot connect a monitor to it. But you can connect an analog video transmitter, which is important for flying platforms with long-range video (airplanes / copters). Long-range video is only analog so far, so that’s what we tuned for. The virtual machine has a low-frequency video output (composite), which can be connected to a video transmitter to transmit video far, far. At the same time, video can be recorded to an SD card.
In addition to the composite output, there is also a component output. It is not so easy to transmit through the air, so they left it unused for now.
Video No. 13 - Virtualrilka and FPV video transmitter
Of course, these are just a few small examples. Opportunities at a piece of iron - the sea. It turned out even more than planned. And this is very good, as the appetite comes with eating. As we master new, previously unknown functions, we will modernize the initial use. And when it turns out that some functionality is practically supported, but there is not enough small strapping - we make changes to the circuit of the piece of iron, or supplement the expansion board.
Thus, the idea came up to use the virtualka as a video module of a home security alarm system. Just to disgrace - I was going to do video surveillance on the floor (where my apartment is located). For all sorts of bad people began to paint the walls, set pieces of paper on fire, and simply naughty. I was going to put a surveillance camera, but problems surfaced - the computer that works for me around the clock does not have a video capture card. And somehow I do not want to load him with additional care. So it is necessary that the video from the surveillance camera comes already in digital form, so that it can only be written to disk.
Of course, the idea arose immediately of using a virtual machine as the main unit of home video surveillance. But the trouble is, the board only has a connector for a digital video camera. And CMOS cameras are not famous for photosensitivity, which is quite relevant in the conditions of the staircase, where there is a rather lousy lighting.
I wrote to my colleague about the problem, as a result of which there was a solution - a small adapter is connected to the virtual computer, in the digital video camera connector, which has several inputs for ordinary analog CCD video cameras. The DSP's power is enough to digitize four cameras (in the case of DM365) and eight (for the DM368) at once. The system sees several devices (/ dev / video0, / dev / video1, etc.), each of which can be processed - compressed in h264 and either saved to an SD card or transferred over a network. So it turned out the required thing - the virtual machine digitizes the picture from several cameras and transfers it to the server.
I can’t demonstrate this functionality yet, because the adapter board is still in development. But as soon as we do, there will be a corresponding demo video.
Another interesting functionality for video processing is provided by OpenCV. This library is on board, but in terms of practical application, I have not tested it yet, sorry.
Initially, we planned to use the hardware only in DYI projects, especially in terms of remote control (via the Internet and locally, from ordinary computers, from a smartphone, from a tablet, etc.) with different mechanical devices. But gradually came to the conclusion that it is possible to use in the final industrial devices. Since we are author-developers and have all the possibilities to modify the board for specific needs, we can throw out all that is unnecessary and leave only the necessary (to reduce the cost of the final device) and release, for example, video recorders based on our platform. DVR with a connection to a wireless network (Wi-Fi or 3G / 4G) - you can immediately broadcast the video to a video server or connect to the machine remotely and see what is happening in it, etc. etc. The end use options are sea.
If you have any suggestions for cooperation, write letters. With creative ideas - feel free to comment.
Official site - http://virt2real.ru
Forum - http://forum.virt2real.ru
Wiki - http://wiki.virt2real.ru
e-mail - firstname.lastname@example.org
Youtube channel - http: // www. youtube.com/user/virt2real
Twitter - http://twitter.com/virt2real
PS Taking this opportunity, I would like to express the deepest respect and respect to our spouses on behalf of all developers. Which for quite some time endured our sitting at computers. And we were engaged in development, instead of paying attention to our halves. We love you and try our best. Smack-smack, in short.
PPS Our team:
Eugene Realizator MIPT, "Faculty of Radio Engineering and Cybernetics"
Alexander AV2R candidate physico-mathematical sciences, MIPT, "Faculty of Radio Engineering and Cybernetics"
Max mkrentovskiy TSU, "Department of Technical Cybernetics", "computers, systems and networks"
Seryoga Gol TSU , "Faculty of Technical Cybernetics", "Computers, Complexes, Systems and Networks". But kicked from the 4th year for permanent gouging :-)