Waste paper megatons with a flick of the wrist

    Good afternoon. In this article, we will talk about how printing works in our platform .

    A bit of history


    For understanding, it will be useful to tell about the history of this function. Initially, the decision of what was called “head on”. A screw spooler was used and printing was carried out through standard procedures. Actually, such a solution did not last long, exactly until the moment when it was required to implement printing from the application server, and not from the client application.
    Just in case, I’ll give a diagram from the first article:



    As you can see, the application servers were in the data center. And the offices were connected into a single network. As soon as we implemented printing from the application server, several problems formed:
    1. The load on communication channels has grown very significantly. In some offices up to 100%
    2. The application server began to freeze.
    3. Sometimes a piece of paper sent for printing was simply not printed.
    4. System performance dropped (users started complaining that they had to wait several minutes for operations to complete)
    5. Confusing print order

    The investigation itself quickly showed that the reasons for the high load on the channels, as everyone probably guessed, were the transfer of the rendered printed form directly to the printer.
    I had to tinker with reason number 2 as it should, but it turned out that people write printer drivers too! It was they (drivers) that hung at high load and dragged along the entire application server. It was possible to get him out of the catatonic state only by restarting it.
    Reasons 3 and 5 were found in the Windows print spooler.
    Problem 4 was resolved by carefully examining the expectations of the sessions. The reasons, as it now seems, lie on the surface, but then we were young ... It turned out that due to printing from the application server, the access time to the printer had grown significantly - now we were printing via Internet channels, and not on the local network. All this time, the transaction continued to block system objects that other server calls came across and arranged for us fun minutes. Of course, we could fix the transaction before sending it to print, but this is wrong from our point of view. We apologize, but about this in another article about transaction management.
    From the stories of acquaintances, I know that such problems were not only ours.
    Then they decided to do what is indicated on the diagram as a print server.

    Printing forms


    It is necessary to retreat and tell what actually we printed.
    For printing, the platform implements such a thing as “Printed forms”. Sorry, we could not come up with a more original name. The printing form is a template for a report book and a script that prepares a data plate based on some algorithm specified by the developer. Yes, sometimes there are cases when this cannot be formulated as a single request. It should be clarified - not as a table in the database, but as an object of the “type” DataTable. In quotation marks - because it uses its own class for storing tables, less memory intensive and more compactly serializable.
    If we talk about the details, then the report is used from DevExpress XtraReports.
    Both the script and the report template are editable at run-time through the GUI of the main client application.
    The script, I recall, is always executed on the application server. The vast majority of papers printed by a trade organization are all kinds of invoices, invoices, waybills, receipts, set sheets in a warehouse, and so on and so forth. At the same time, the template does not change, only the data changes!
    Thus, the first thing we did was to submit to the print server the task of rendering and sending the rendered form to the printer. The application server only prepared the data and sent it to the print server. Saving on traffic - from (on average) 2MB per document up to 40kb, the print server caches templates locally.

    Device and features


    Consistently increasing the functionality of the print server, we were able to get rid of these problems.

    So what we have now.

    Synchronous and asynchronous printing.

    As a rule, there is no need for a guaranteed delivery time of the printing plate to the printer (i.e. print right now). And, on the other hand, the script that sends the document to print has already made some changes in the database and it is likely that another transaction may run into these locks. In this case, you need to commit the changes as soon as possible. For this, the application server can accept a print job to process it asynchronously. In a separate transaction, execute the data preparation script and send it to the appropriate print server. Synchronous printing implies that the data preparation script will be executed in the current transaction.
    Illustrative diagram:



    The PrintBuilder and PrintSender procedures working in separate transactions (and threads) inside the application server process the queues of incoming jobs. Technically, these are threads within an application server that execute an infinite loop. The amount of these stream is configured by the administrator depending on the load. PrintBuilder processes the jobs received asynchronously, executes scripts to prepare data for printed forms and passes them to PrintSender in the next turn. Synchronized print jobs fall into the same queue.
    In turn, PrintSender removes jobs from the queue and transfers them to the print server.
    As you can see, with this approach, the blocking time in Transaction 1 is drastically reduced, and the shorter the blocking time, the higher the number of transactions that the server can process!

    Batch printing

    Guaranteed printing on a printer of documents in a predetermined sequence. Regardless of other tasks, all documents in the batch will be printed inextricably. The opportunity was required for us when implementing the printing of documents for drivers ( an example of application in practice of a real business). This is the task when you need to print a routing sheet (a document listing all addresses and a map) to the printer and then accounting documents for each order. Everything was fine until the drivers were over 100 and the Windows spooler did not start to get confused. The situation worsened if someone else tried to print other documents on the same printer. For some reason, these documents sometimes appeared among the documents for the driver. I had to abandon the spooler and make my own.

    Guaranteed delivery

    A document sent to print will be guaranteed to be sent to the printer. If the printer is unavailable, or the office with the print server is unavailable, the system will wait for the connection to be restored and send SMS and email to administrators.

    Here is a diagram illustrating the print server operation diagram:



    In this diagram, PrintWorker are separate processes that run for each connected printer. If the printer driver freezes, then the process stops responding, and after a timeout the print server kills the process and starts again.

    Print accounting

    A side effect is accounting of who, when, what and how much printed, and users no longer have direct access to printers, which greatly reduced paper consumption for printing dissertations. Whether this is a cool thing is up to you.

    Printer Virtualization

    Printers that can be printed from the system are listed in the system settings and only connect to the print server. This greatly simplifies the work of admins - no need to connect a printer to each node in the cluster of application servers, and on each client machine, install drivers there and engage in other shamanism. In addition, listing the printers in the system makes it easy to implement a UI for ordinary users to select a printer. For example, which printer to print paper in the warehouse.



    Thus, each printer has its own identifier, which is saved when the printer is changed, and the code for printing looks something like this:



    In this example, the printed form for the specified document is printed to the printer specified by the employee.

    Queues in RDBMS


    They promised to talk about the queue. We are telling.
    In the print subsystem, as you can see, the queue mechanism is actively used. Fortunately, in Oracle Database there is an opportunity to organize them inside a database without resorting to special separate services. They are implemented quite simply using the design
    select .. for update skip locked
    
    . Executing such a query returns only rows NOT blocked by other transactions. Some inconvenience is caused by the fact that it cannot be used together with rownum, i.e. You can only get all the lines. However, there is a workaround - all of the above is true only for executing a request from a client (with respect to the database) application! So, you can write a procedure:



    The main thing in this procedure is to declare the cursor and pull out one record. Thus, without unnecessary locks and very conveniently, it is possible to organize parallel processing of various queues.

    For example, in a similar way, the task of recalculating prices for 400,000 goods is solved.

    Anticipating the question of why they did not use Oracle Advanced Queuing, we answer:

    1. We use Managed ODP.NET, and it still does not support AQ.
    2. Using your queues allows you to store data in a structured form, and implement built-in mechanisms for managing these same queues:


    Conclusion


    I hope that we were able to approximately reveal the implementation of the printing subsystem in our platform and come up with ideas for solving problems in your applications. Further we will try to open the topic of transaction management and related problems.

    Also popular now: