Python-IronPython-Jython Interpreter Interactions
There was a need to solve such a problem: how to exchange data between different Python interpreters ?! I have found several solutions, but I want to talk about one, in my opinion, the most convenient one.
For starters, I tried to choose the most convenient solution from the many. It may not correctly compare message queue, sockets and RPC, but I came from the task. Here is what is available:
Using sockets, the easiest way to interact, you can transfer any data, but you will have to saturate the code with the implementation of your own protocol. This path is thorny, and mistakes are inevitable. Conclusion: suitable, but difficult.
A very promising development brokerless message queue. Implemented as pyzmq and pyzmq-static library . For the first case, you need to install ZeroMQ itself - a shared library written in C. The second one already contains a binary. It all started quite funny, the code worked fine until I switched to IronPython ...
for IronPython pyzmq is not implemented, but no problem, there is .NET Integration, which allows you to use clrzmq- binding for .NET; and here the problems started. The binary is available through the plugin for VisualStudio - nuget, so I had to compile the library. I downloaded the source, everything is convenient there, the script is built by the build command, but not so simple. Version 2 did not compile at all, although it resolved dependencies correctly through nuget (maybe it took a broken commit?), 3 beta compiled, but it crashed even with the tutorial as an example)
As a result, I found the binary 2 network, but it didn’t please me at all. Everything worked, but the API in 3-ke changed so much that I did not want to rewrite the code a hundred times later. Conclusion: good, but raw outside of pure Python.
At first, I decided to finish 0mq, but I remembered the existence of such a library as Pyro : a library from the crisis of the 90s, which is doing well and developing. I will not list all its advantages, now I am only interested in one thing - the interaction of interpreters. Immediately to the point, we try an example from the tutorial , but with the difference that we run everything under different interpreters (the code did not change, it just entered its nickname).
We start the name server under Jython, by the way, here is the first huge plus, at 0mq the name server has not yet been completed:
Wow! It started ...
Now we register a server that will run under IronPython:
Registration was successful.
The next step is to launch the client. We do it under the classic python (I have a lot of them right away, but I started EPD):
Perfectly! Everything worked.
As an option, I also tried to run one of the parts under Python 3. Without a hack, it didn’t work :( The latter uses pickle of the third version + there are no standard modules (removed, migrated), although this behavior can be circumvented. But the main reason Pyro4 uses pickle for serialization with the latest version, here's how to get around this flaw and connect Python 3 to IronPython 2 and Jython 2:
This must be added to the client code before importing Pyro. Everything worked:
Pay attention to the use of 64-bit Python 3 along with deuces.
By the way, as an option, you can use the pyrolite library to connect directly from .NET or Java without a Python interpreter. This is actually a mini pickle.
In general, the solution with Pyro has so far been liked more than others, but if the system contains a lot of components of different versions, then it will probably be more profitable to use 0mq. Although I will still focus on Pyro4.
Choice of approach
For starters, I tried to choose the most convenient solution from the many. It may not correctly compare message queue, sockets and RPC, but I came from the task. Here is what is available:
TCP Sockets
Using sockets, the easiest way to interact, you can transfer any data, but you will have to saturate the code with the implementation of your own protocol. This path is thorny, and mistakes are inevitable. Conclusion: suitable, but difficult.
ZeroMQ Library
A very promising development brokerless message queue. Implemented as pyzmq and pyzmq-static library . For the first case, you need to install ZeroMQ itself - a shared library written in C. The second one already contains a binary. It all started quite funny, the code worked fine until I switched to IronPython ...
for IronPython pyzmq is not implemented, but no problem, there is .NET Integration, which allows you to use clrzmq- binding for .NET; and here the problems started. The binary is available through the plugin for VisualStudio - nuget, so I had to compile the library. I downloaded the source, everything is convenient there, the script is built by the build command, but not so simple. Version 2 did not compile at all, although it resolved dependencies correctly through nuget (maybe it took a broken commit?), 3 beta compiled, but it crashed even with the tutorial as an example)
As a result, I found the binary 2 network, but it didn’t please me at all. Everything worked, but the API in 3-ke changed so much that I did not want to rewrite the code a hundred times later. Conclusion: good, but raw outside of pure Python.
Pyro, set it on fire!
At first, I decided to finish 0mq, but I remembered the existence of such a library as Pyro : a library from the crisis of the 90s, which is doing well and developing. I will not list all its advantages, now I am only interested in one thing - the interaction of interpreters. Immediately to the point, we try an example from the tutorial , but with the difference that we run everything under different interpreters (the code did not change, it just entered its nickname).
We start the name server under Jython, by the way, here is the first huge plus, at 0mq the name server has not yet been completed:
Wow! It started ...
Now we register a server that will run under IronPython:
Registration was successful.
The next step is to launch the client. We do it under the classic python (I have a lot of them right away, but I started EPD):
Perfectly! Everything worked.
Pickle version
As an option, I also tried to run one of the parts under Python 3. Without a hack, it didn’t work :( The latter uses pickle of the third version + there are no standard modules (removed, migrated), although this behavior can be circumvented. But the main reason Pyro4 uses pickle for serialization with the latest version, here's how to get around this flaw and connect Python 3 to IronPython 2 and Jython 2:
import pickle
pickle.HIGHEST_PROTOCOL = 2
This must be added to the client code before importing Pyro. Everything worked:
Pay attention to the use of 64-bit Python 3 along with deuces.
By the way, as an option, you can use the pyrolite library to connect directly from .NET or Java without a Python interpreter. This is actually a mini pickle.
In general, the solution with Pyro has so far been liked more than others, but if the system contains a lot of components of different versions, then it will probably be more profitable to use 0mq. Although I will still focus on Pyro4.