Opt out of NFS in the cloud

    Sorry for the long silence - a lot of work, big updates are coming. In the meantime, a little about a not very large, but very noticeable change for our customers.

    We refuse to host kernel modules on NFS. (And not only modules, but customers will notice exactly a change in the storage location of the modules).

    How it should have worked

    Client virtual machines are loaded using our cores (that is, kernel code is stored outside the virtual machine). Kernels need modules in the process. / lib / modules is mounted by NFS, the kernel itself determines which directory to load which modules from, it’s easy for us to update them, and it’s easy for the client to get access.

    As it turned out

    Firstly, NFS balls are mounted later than the network initialization (this is obvious) and after mounting all the other lines in fstab. Even cooler - in the debian / ubuntu family, they are mounted by default asynchronously, so that we get a race condition with rc.local running.

    Bottom line: pre-up scripts on interfaces do not work as expected, non-standard file systems from fstab are not mounted as expected. Additionally, NFS is not the most reliable service (especially considering the bug # 538000), in other words, it is inconvenient.

    How to solve this problem

    The modules are now located on the ISO'shoku, connected to all virtual machines as a separate disk / dev / xvdp. Modules are mounted immediately after mounting the root ('/') and make it easy to perform all subsequent operations (pre-up scripts, non-standard file systems, etc.).

    The mount line (fstab) looks the same for everyone:
    / dev / xvdp / lib / modules iso9660 ro 0 0
    

    By the way, this disk is not paid by customers.

    Why in read only?

    Firstly, as already mentioned, it is one for all. I don’t want your neighbor to attach you a special version of iptables, for example. Secondly, modules (like the kernel) are controlled by our control system. In the near ... well, in the near future we will add the ability to choose which core to use, but the core will be from our list. The reason is simple - there are a lot of not very stable kernels under xen. For example, debian suffers from this. Alas. Moreover, some of the bugs will not be visible right away - and we really wanted to avoid the need to answer questions about the stability of the cores that we did not choose. Xen's paravirtualization implies kernel cooperation during many operations (such as migration) - and abandoning this cooperation in the middle of the process may render the machine inoperative.

    Can you load your modules?

    Yes you can. Despite the fact that / lib / modules is mounted in readonly, you can load modules using insmod from anywhere. The only “but”, the kernels we sometimes update, in this case, the modules we supply are also updated. And you have to rebuild the modules yourself. Fortunately, updates are always obvious and “accidentally broken” will not happen.

    Why iso9660?

    This is the file system used on CD (and ISO-images) - for it the most natural is read only mode. If we used ext3, we would be tempted to remount it in rw (this could not have been done and a lot of errors would have appeared). The second reason - iso9660 is ignored by OS installers during installation (no one will try to create a partition table there).

    How to get away from NFS to ISO

    Simple script:

     sed -i /109.234.152.2/d / etc / fstab
     echo "/ dev / xvdp / lib / modules iso9660 ro 0 0" >> / etc / fstab
     umount / lib / modules
     mount / lib / modules
    


    No reboot required.

    On new machines, the modules are mounted exactly as needed.

    Of course, you don’t have to go over, we will maintain the current configuration on NFS as long as required (until the last machine with the old module system). Those who switched to the new system can safely cut out any traces of NFS - it is no longer needed.

    Also popular now: