Dynamic applications with Ocsigen or Yoba returns

What does a normal person do on a cold Sunday morning? Anyone will answer you: on a cold Sunday morning, a person is sleeping. Because he worked all week and wants to relax.
What does a programmer do on a cold Sunday morning? On a cold Sunday morning, the programmer drinks hot tea and writes the code. He drinks tea because the morning is cold, and he has not woken up yet, and he writes the code because he wants to. A programmer always wants to write a code, only on weekdays he writes a code for money and is very tired of this, and on weekends for himself, so he rests.

This morning we will be writing our first application for Ocsigen. It would be nice for those who wish to first familiarize themselves with the official manualhowever, one should not hope for much, because the manual is unfinished, replete with perplexing lines a la "??????" and foul language in French. Therefore, I will be the main manual.

As you may remember, we once wrote the interpreter of the Yoba language. Since then, the interpreter has been slightly improved, allocated to a separate class, began to take a line to the input, give the line to the output (instead of working with the console). Now our task will be to introduce Yoba as the main language of Google, turning the Yoba interpreter into a web application, and not just a simple one, but a client one. Although I added an operation counter to the class so that I couldn’t become too impudent, but still - let the user spend on his computer computing power, and not on the server.

First, we need to install the ocsigen server. Since we have not yet managed to compile 2.0 for distributions, we will follow this instruction and install the bundle server in our home directory. In order for the bundle to turn out to be correct and tasty, before running make, edit Makefile.config and write it there: we will not collect Ocaml and Findlib, they are already in the repositories. This time we won’t need O'Closure, but we’ll collect it just in case — so that we don’t have to reassemble oxygen later, if you yourself are interested in it or want an article from me.
LOCAL := ${HOME}/bin/ocsigen
DEV := YES
OCAMLDUCE := YES
OCLOSURE := YES
OTHERS := YES


The next item will immediately edit the file $ {HOME} /bin/ocsigen/etc/ocsigenserver/ocsigenserver.conf: make sure that the appropriate port is written there, as well as the name and group of the user to start from. Now it's time to prepare the config for the future site. Create $ {HOME} /bin/ocsigen/etc/ocsigenserver/conf.d/yoba.conf and fill it with the contents:

utf-810000application/x-javascript

Briefly about the content:

• staticmod allows the server to return static data (in our case, this will be a compiled .js file
• ocsipersist-sqlite - allows us to work with persistent data, then we'll see why this is necessary
• deflatemod, as expected, presses the data when it is sent. Below you can see the deflate section in the site settings, which says that we only press js. Note: you cannot choose the compression method - gzip or deflate - it is automatically selected based on client expectations.
• eliom.server provides the entire server part of the framework
• In the hostfilter parameter of our host, we say that we respond to all domains
• And indicating an empty path for the site, we say that the site is located in the root of the domain. Replacing the empty line with, say, “yoba”, we will force our site along with all the static content to be given to the address hostname / yoba - an unambiguous profit, you can keep several sites on the same domain and shuffle them as you like
• / home / username / yoba will be our directory with the code (and the compiled yoba.js file), and in _build / server will be the compiled server module. Of course, ideally, you need to compile, then transfer the code to a separate folder, and still do not store static content in the same folder with the compiled module, but now we will do so to quickly check the code

Hurrah! We turn to writing code.
We create the notorious folder / home / username / yoba / and download the archive with the Yoba language - we already wrote the language itself, but it’s not interesting for us to file it a bit and turn it into a class, so we’ll take it right away. We unpack it into the same folder and download the standard Makefile.config and Makefile.rules to the same place - building a project is not easy, you can write a fig file from scratch.
The time has come to fix something and immediately learn about the first syntax innovations: in Eliom code can be placed in sections. Section {server {...}} (the same as code without a section) is compiled and executed on the server, section {client {...}} - on the client, and{shared {...}} - available both there and there.
Since our interpreter will work on the client, we rename the yobaLang.ml file to yobaLang.eliom, open it, and at the very beginning add the line "{client {", and at the very end replace ";;" (two semicolons - this is the end of the instruction only at the top level of the code, they can no longer be used inside the section) to "}}". The generated ocamllex and ocamlyacc code will be similarly edited in the makefile when we write it.
In the meantime, let's write a file that will do everything. Call it, say, home.eliom.
At the beginning of the file, open the modules for the future and immediately create a line with sample code that will be offered to the visitor.

{shared{
  open Eliom_pervasives
  open Lwt
  open HTML5.M
  open Eliom_parameters
  open Eliom_request_info
  open Eliom_output.Html5
  open Ocsigen_extensions
  let code_example = ref "
чо люблю сэмки йоба
чо люблю пиво йоба
чо люблю яга йоба
чо люблю итерации йоба
чо пиво это 1 йоба
чо яга это 2 йоба
усеки результат это
чо покажь итерации йоба
чо покажь сэмки йоба
йоба
усеки фибоначчи это
чо сэмки это пиво и яга йоба
чо пиво это яга йоба
чо яга это сэмки йоба
чо итерации это итерации и 1 йоба
чо есть итерации 50 тада хуйни результат или хуйни фибоначчи йоба
йоба
чо хуйни фибоначчи йоба"
}}

Next, we will create a module of our application - in order for the client-server interaction to work correctly, all services are registered on behalf of an application. Fortunately, this is easy:

module My_appl =
  Eliom_output.Eliom_appl (
    struct
    let application_name = "yoba"
    end)

Add a function that will execute the code on Yobe and return the result, the function will be purely client, the server will not even know about it

{client{
  let yoba_execute str = (
    let yparser = new YobaLang.yoba_interpretator () in
    yparser#parse str;
    yparser#get_output)
}}

Create a service that will process user requests. Creating services in ocsigen is unusually simple and convenient, each service is characterized by a way and a set of strongly typed (!) GET / POST parameters. Accordingly, the server decides which service to process the request, based on the request. You can create a default service that will process requests without parameters, a second service at the same address that will process requests with one GET parameter, and a third service with one POST parameter. And they will not be confused. But for now, we only need one service:

let empty_service = Eliom_services.service
  ~path:[""]
  ~get_params:(Eliom_parameters.unit)
  ();;

The path symbolizes that the service will be given along the default path for the site (as an index page in Apache), and in ~ get_params we indicated that the service does not accept parameters.

It's time to write a page template:

let page_template code_input counter_value =
  html
    (head
      (title (pcdata "Yoba interpreter")) []
    )
    (body [
      h1 [pcdata "Yoba! For human beings"];
      p [pcdata "Вас приветствует Йоба! Йоба — это чотко!"];
      div [
        raw_textarea ~a:[a_id "clientcode"] ~name:"clientcode" ~rows:25 ~cols:60 ~value:code_input ();
        raw_button ~button_type:`Button ~name:"clientbutton" ~a:[a_id "clientbutton"] ~value:"Кликай!" [pcdata "Кликай!"];
      ];
      pre ~a:[a_id "clientoutput"] [];
      hr ();
      p [pcdata "Йоба-скриптов хуйнули уже: "; b [pcdata (string_of_int counter_value)]]
    ]);;

As you can see, all page elements are functions that take strictly defined parameters as input, due to which the static typing of the created HTML page is carried out. So, the html function accepts two parameters as an input - one of type `Head, and the second of type` Body. And div - takes as input a list of elements allowed to be inside the div.

But we will dwell in more detail on the other. Firstly, our page_template function accepts two parameters as input - a code and a visitor counter. The first is placed in textarea, and the second is inside the tag

at the bottom. Secondly, the names of the raw_textarea and raw_button functions are so raw - for good reason. There are similar simple non- “raw_” functions, but they are designed to create elements inside wonderful, strongly typed forms that always refer to some service (in other words, when creating a form, we immediately verify that it will send the strictly required list to where parameters). And our textarea and button (this is not a tag, and the most natural tag from HTML5) they won’t send anything anywhere, but they will frolic inside the page, therefore they don’t have a form either. Third, we created a special

, which will store the results of our interpreter. 


By the way, did I mention the visitors counter? I completely forgot, let's write it. To do this, we will immediately get acquainted with two new modules: Lwt and Ocsipersist. The first is responsible for working with cooperative threads, and the second is for persistent storage. 

The oxygen flow system is cooperative. This means that instead of traditional threads requiring a new process, call stack and other crap, we get very lightweight threads (so light that they are used for almost every call). Instead of creating all sorts of nonsense, we, turning to threads, create in the code the so-called cooperative points, on the basis of which the compiler itself does everything that is needed, minimizing the probability of deadlock.
let get_count =
  let counter_store = Ocsipersist.open_store "counter_store" in
  let cthr = Ocsipersist.make_persistent counter_store "countpage" 0 in
  let mutex = Lwt_mutex.create () in
  (fun () ->
    cthr >>= (fun c ->
      Lwt_mutex.lock mutex >>= (fun () ->
        Ocsipersist.get c >>= (fun oldc ->
          let newc = oldc + 1 in
          Ocsipersist.set c newc >>= (fun () -> Lwt_mutex.unlock mutex; return newc)
          )
        )
      )
    )
  ;;


If you’re unaccustomed to OCaml (as I did at first), you may notice that our function is tricky. Get_count itself is an “object” that stores the mutex and the storage object. When we write “get_count” in the code, we are returned a function that takes input () and only then does all the work. Now we get into the function. We immediately see the tricky operator ">> =" - this is a special operator that passes the result of the first argument - the thread - to the input of the second argument - the function that creates the new thread. Speaking formally, the signature of the operator is as follows:
val (>>=) : 'a t -> ('a -> 'b t) -> 'b t


And the return function at the very end returns the result of the thread. 

With Ocsipersist, everything is clear, even nothing to tell. 


Where will we call our get_count and generate the page template? And here it is, the interpret function:
let interpret code =
  let req = Eliom_request_info.get_ri () in
  let ref = match Lazy.force_val req.ri_referer with | None -> "" | Some x -> x in
  Ocsigen_messages.accesslog ("Referer: " ^ ref);
  get_count() >|= (page_template code);;


This feature introduces us to a few more interesting features. Oksigen, alas, writes exclusively brief information about requests to the log - who, what user agent, which host, which page, when. And I also wanted to get a referer. Well, we will get information about the request, we will pull the referrer out of it. It is again crafted cunningly - it is a value that may not exist (such as Null in other languages), and which is also wrapped in lazy computing, that is, until I requested it, it was not stored anywhere. 

Another new operator "> | =" is similar to ">> =" - with the only difference being that the result of the thread is passed to the input of a function that the new thread does not plan to return at all. 


That's it, come to an end. It's time to register our service and learn how to interpret the code:
My_appl.register empty_service
  (fun () () ->
  Eliom_services.onload
  {{
    Js.Opt.iter (Dom_html.document##getElementById (Js.string "clientbutton")) (
      fun clntbutton ->
        clntbutton##onclick <- Dom_html.handler (fun _ ->
          Js.Opt.iter (Dom_html.document##getElementById (Js.string "clientcode")) (
            fun cdinput ->
              Js.Opt.iter (Dom_html.document##getElementById (Js.string "clientoutput")) (
              fun cdoutput ->
                let cdinputarea = Dom_html.CoerceTo.textarea cdinput in
                Js.Opt.iter cdinputarea (fun x ->
                  let i = Js.to_string x##value in
                  cdoutput##innerHTML <- Js.string (yoba_execute i)
                )
              )
            );
          Js._true
          )
      )
  }};
  interpret !code_example);;


The {{...}} block is like a client function - we register it in the onload handler of the page. 

In the client code, our syntax is slightly different. To access the methods of Js objects, instead of a single sharp, double is used, so Dom_html.document ## getElementById corresponds to a simple "document.getElementById".

The numerous Js.Opt.iter that can be observed here is due to the fact that the getElementById function does not necessarily return anything to us. Accordingly, Js.Opt.iter will perform an action on the result only if the result really is. Therefore, for the three objects on the page we were looking for, we needed four Js.Opt.iter. Four - because by default the getElementById function returns an object of type element that has only the most common properties. And in order to get to the value property in textarea, we try to cast (Dom_html.CoerceTo) our object into type textarea, which does not guarantee the result in the general case. 


Thus, the registered service handler does only two things - it calls the Js-code at the start of the page and returns to us our template mentioned above.


An attentive reader might already notice and wonder why, at the very beginning, I declared code_example as a link to a string (ref), and then dereferencing it everywhere. And the thing is that at some point it occurred to me that our Yoba should be truly collective. Let's save what the user tried to interpret and show to others. 

To do this, next to the first service, we will create a specially trained second service that will take a line with the code and put it in code_example. To call the service from javascript, create it on behalf of Eliom_output.Caml:
let update_code_service = Eliom_output.Caml.register_service
  ~path:["update code"]
  ~get_params:(string "f")
  (fun f () -> code_example := f; return ());;



Now we change our client function (which is the most internal), adding only one line:
                  let i = Js.to_string x##value in
                  ignore(Eliom_client.call_caml_service ~service:%update_code_service i ());
                  cdoutput##innerHTML <- Js.string (yoba_execute i)


Voila! Now everyone who comes to our page will see the code that was executed last. True, when you restart the server, our example will still return to its place. 


Finally, write the Makefile:
MODULE = yoba
APP = yoba
include Makefile.config
SERVERFILES := home.eliom
CLIENTFILES := yobaType.ml yobaLexer.eliom yobaParser.eliom yobaLang.eliom home.eliom
SERVERLIB := -package eliom.server,ocsigenserver,lwt
CLIENTLIB := -package js_of_ocaml
INCLUDES =
EXTRADIRS =
include Makefile.rules
yobaParser.eliom:
        ocamlyacc yobaParser.mly
        echo '{client{' >yobaParser.eliom
        cat yobaParser.ml >>yobaParser.eliom
        echo '}}' >>yobaParser.eliom
        sed -i 's/;;//' yobaParser.eliom
        rm yobaParser.ml yobaParser.mli
yobaLexer.eliom: yobaParser.eliom
        ocamllex yobaLexer.mll
        echo '{client{' >yobaLexer.eliom
        cat yobaLexer.ml >>yobaLexer.eliom
        echo '}}' >>yobaLexer.eliom
        sed -i 's/;;//' yobaLexer.eliom
        rm yobaLexer.ml
_build/client/yobaLexer.cmo: _build/client/yobaParser.cmo
_build/client/yobaLang.cmo: _build/client/yobaLexer.cmo _build/client/yobaParser.cmo
$(STATICDIR)/$(APP).js: _build/client/${MODULE}.cmo
        ${JS_OF_ELIOM} -jsopt -pretty -verbose ${CLIENTLIB} -o $@ $^
        #yui-compressor --charset utf-8 $@ > $@_min
        #mv $@_min $@
pack: $(STATICDIR)/$(APP).js


Since we need to generate yobaLexer.eliom and yobaParser.eliom, we wrote the appropriate rules for this. Similarly, alas, the default dependency generator does not cope with determining in which order to compile our lexer and parser, so we helped him with a couple of rules. 

Now you can run: 
The first will generate the compilation order, the second will compile the server code, and the third will generate a javascript file that will be automatically called by the server side, although we did not specify this call in the page template. If you uncomment the call to yui-compressor and the subsequent mv, you can compress the js code a bit (in my case, from 400kb to 209kb). 
You must complete all the instructions by writing the export PATH about which the ocsigenserver assembly told you at the very end.
make depend

make

make pack




After that, go to $ HOME / bin / ocsigen / bin and say ./ocsigenserver 

Open the browser and go try to interpret. And if you are too lazy, then you can frolic with me: sorokdva.net