Developers in the struggle for the effectiveness of the programmer, team, team

    Hello.

    Today we would like to discuss one very important aspect of effective work - reuse.

    This will, of course, be about code.

    A routine that I do not want to write twice, and even more so three times. Infrastructure, which has to be written from project to project. Brilliant, which simply expresses a great idea and which is a sin not to let others use it.

    Yes, and let's admit - we all love to write “frameworks”, which we use later and others use. This adds a bit of courage to our daily activities.

    And here the author catches himself on the fact that he is doing wrong, starting to describe the situation - everyone already understands what it is about. And you need to describe the technology and process. So let's just dwell on the abstract idea that suddenly there was a lot of code that could be useful to colleagues and it would be nice to make it easily accessible. We also take into account the second, much less pleasant, idea that in “mature” projects you can find a large number of copy-paste code, which significantly reduces maintainability. And the work on such projects is hardly pleasant to call. And something needs to be done with this.

    And we decided to do something about it.


    The path we traveled turned out to be not so difficult, we can advise you to repeat it for others, because the time spent pays for itself. And since at some point this path became interesting and even fun, we decided to share our experience.

    Well, enough preface.

    So, today we will tell you in detail:
    1. How to deploy a corporate nuget server ( we will omit the basics of nuget , assuming that most readers know what it is).
    2. How to deploy a corporate symbol server integrated with nuget.
    3. Why is one nuget server not enough to build an efficient process.
    4. We also outline in advance the topics for the second part, which we will gladly produce if this topic finds its readers:
    5. What is SandCastle and what it is eaten with.
    6. How to configure continuous integration to automate everything we have done.
    7. A few thoughts on the (self) organization.


    We deploy the corporate nuget server.

    So nuget is great and fun. This is not just a new tool that improves productivity, it is a new approach, a new way of thinking. Almost as healthy as a healthy diet and morning exercises. So you need to use.

    If we are talking about deploying a corporate server, there are two installation options:
    1. Easy tape rollout without UI and buns. For this option, it is enough to create an MVC application and install the nuget server by downloading the nuget package (such a recursion).
    2. The second option is created for a little less lazy, but it is with UI and goodies. For it, you need to download the sources and compile them according to the instructions from the same page. After that, by pressing Ctrl + F5, we should see a (almost) complete analogue of the official tape.




    Actually, that's all - the process is extremely simple. However, there are several points that are not described in the documentation that have appeared with us and which may be useful to others:

    1. IIS express . The option of working on IIS express using localdb is difficult to consider as production ready for a number of reasons. Therefore, we moved our newborn to a fully functional IIS server. Fortunately, there are no difficulties with this. Just go to the properties tab of the “Website” project and uncheck “Use IIS Express” in the “Web” section. Visual Studio will immediately respond to such changes - it will offer to create a Virtual Directory and change the Connection Strings in web.config from localdb to SQL express.



    2. Database Migrations. Nuget uses Entity Framework to access data and actively uses the Database Migrations mechanism. In the deployment guide, there is step number 4 - creating a database using dances with sabers around the procedure for opening tabs in the studio. In fact, everything is simpler. It is enough just to specify the name of the project in which migrations are stored when updating through the console. And no dancing:



    3. Security settings . The manual does not describe the points regarding security settings (although they were in the outdated manual ). Apparently, it is assumed that everything is set by the unconditional admin and everything will spin under his own account. If this is not so, then we need:

    • a. Give write permissions to the “AppData” folder of the application. Nuget packets will be added there. We give rights to the account under which the AppPool applications are spinning. In case the ApplicationPoolIdentity is in the IIS settings, and not a specific account, we give rights to the IIS_IUSRS group.
    • b. By the same principles, we give rights to access the database in SQL.
    • c. After registration, you can give yourself the same role as an administrator. To do this, after standard registration through the UI, insert the record into the dbo.UserRoles table, looking at the Id of the role in the dbo.Roles table. After that, the address “<server path> / admin” will allow you to access the admin panel with all sorts of goodies and the glimpse bookmarklet .




    4. GalleryOwner . It makes sense to change the “Gallery.GalleryOwner” item in web.config for real settings. Letters will be sent from him during registration. And you can completely disable this newsletter. And be sure to write down the current SiteRoot.



    5. There is a getStats function in the stats.js file . At the time of this writing, its invocation led to an error on the main page.



    The reason is that in the local version of nuget, statistics are disabled . At the moment, it is intended only for the cloud version.

    You can create your own implementation of IStatisticsService or just disable this call. We have chosen the second path.

    That, in fact, is all. Now we have a nuget server with a gallery and packages .

    But what if a bug crept into our package (dll from it)? Or does the developer just want to understand what our “DoStuff” function does? Obviously, the library needs to be debited. But it’s not so easy when we have only dll at our disposal. That is why nuget is able to collect not only library packages, but also symbol packages .

    And we are going to learn how to build our own symbol server

    As a solution to support debugging, we decided to use SymbolSource , since it is so good friends with nuget that it supports it as a default symbol server .

    Here we will make a small digression and tell you how it all works.

    When creating a nuget package through the nuget pack command, we can specify the “–Symbols” flag. As a result, nuget creates, in addition to the standard package with the library, the package <package_name> .symbols.nupkg, which will contain a pdb file and source code. Then this package can be placed on the SymbolSource server, which will index the pdb file and place the project sources in the internal directory. Next, in Visual Studio, access to the symbol server is configured, and if necessary, it will download the pdb file and sources from the server. This is how debugging becomes possible.

    But since we didn’t dare to put the libraries into a public feed, then there’s nothing to say about the sources. In principle, SymbolSource makes it possible to create a private feed. But he also supports the Community version, which allows you to deploy your internal server. For a number of reasons, we decided to go the last way.

    And then the adventure began ...

    According to the authors, the task of deploying your symbol server is almost as simple as deploying your nuget feed - we create an MVC application, download a nuget package , give rights to the App_Data and Data folders - everything works.

    Oh yes, there is one additional requirement - you must install Debugging Tools for Windows and write the path to them in web.config:



    Actually, this is what we did - we downloaded version 1.3.2 of SymbolServer, installed, configured, saw a magic screen.



    Diagnostics launched - all is well. We set up Visual Studio exactly according to the instructions .

    But Debug doesn't work ...

    The problem was found quite quickly - the service did not save the source code into the “Data” directory when loading the symbol package. Having tinkered with the IIS settings for some time, we realized that the problem was different, and started debugging.

    Exactly. Debugging a SymbolSource server using a SymbolSource server.

    The cause of the problem was also found quite quickly, and at first it seemed even funny. The ManagedSourceExtractor class has the IsTemporaryCompilerFile method, which returns, as the name implies, a sign of whether a certain file is a temporary compiler file. And in this method there was an extra operator of logical negation. That is, this method reported that the files that were not them were the tempo ones. So one extra character broke the entire server.

    Well, what can you do ... They laughed, forked the source code , corrected it, and gathered to make a pull request. And here we find that a similar request already exists, and created a long time ago. Yes, and the package version 1.3.2 has been released for two months already, but the version of the package with the fix has not been released so far. Now this was not so funny. More precisely, not at all funny.

    For a while, we even thought about whether to use this server at all or to look for another one. But the positive previous experience of use ... And the nuget developers made it the default server, which means they trust. And the package version 1.3.1 works fine. As a result, we decided to run everything on the corrected source. To do this, we additionally copied the setting from “Web.config.transform” to “Web.config”, since the transform is not performed during the local assembly:



    Now it’s ready and also works as it should. And then it's time to think about

    Why is one nuget-server not enough to build an effective process (beta).

    And it’s not enough for a very simple reason: nuget, as it is, is a server available to the entire company.

    This means that you cannot put packages into it that may turn out to be unstable. But when the development of final projects required a change in the assemblies placed in nuget, then to make changes so that it worked right away, the task from the category of “write code without bugs”.

    There are many different tricks and squats to work with one server somehow. For example, replace dlls with handles in the “packages” folder of the final project, and publish to nuget only when everything is ready. But we are doing this all to make life easier, and not vice versa. Therefore, we decided to approach the issue in more detail. After a little thought, we came up with something.

    The following is a beta version of the process that we are only testing and comprehending. Therefore, it would be great to hear the opinions of readers in the comments.

    So, we decided that we would have three nuget tapes.
    1. Local feed . In fact, this is just a network folder (in Visual Studio, when configuring sources, you can specify not only the server address, but also the UNC path to the folder). Packets are collected in this tape for every commit in VCS. No one except the developers of the reusable source code configures this tape.
    2. Test feed . This role logically falls on the SymbolSource server, which has its own feed (namely a feed, not a gallery) for packages, where a regular nuget package is automatically placed when publishing a symbol package. Publishing to this feed is performed when changes to the base code are completed and you can start using it. All developers of the company and build-machines collecting test versions of final products have a source for this tape.
    3. Release Package Server . This server was the one that we deployed at the beginning of the article - beautiful, with a UI and other goodies. The most important bun is the ApiKey requirement. The local tape does not require it, as does the SymbolSource tape (more precisely, SymbolSource requires, but does not verify). And there they are not needed. Everything is more serious here, so it makes sense to make sure that not everyone can publish. And if someone could, it would be clear who it is. Publication on this server occurs only when the base code is tested both on its own and in working with end products. Source is configured on this server for all developers, build machines that assemble test versions of end products, and build machines that assemble release versions of end products.

    It turns out, on the one hand, that build machines that build release versions of end products will never assemble a project if it refers to a test version of the package.

    On the other hand, the order in which the servers are specified in the Package Sources settings in Visual Studio (Tools-> Library Package Manager-> Package Manager Settings) sets the priority for downloading packages. Therefore, for example, the developers of the base code have access to all three feeds and the order in the settings is set as release feed, test feed, local feed.

    In Visual Studio, it looks something like this:



    By the way, this is where the argument in favor of local server deployment becomes quite significant - independence from internet access and greater speed when working on the local network.

    “What if one of the servers crashes?” You ask. Well, it doesn’t matter. After all, the build process of the “framework” is one for all tapes. That is, if a package version 1.2.3.4 is declared release, then it lies in all three tapes. And access to it, for example, the developer will always have.

    On the one hand, such a process is not very simple, but on the other hand it turned out to be very natural, so you won’t have to think about its complexity often - it just falls on a standard workflow.

    Well, well thought out. Now you can return to technology and think about Continuous Integration, which automates the whole process.

    But the article has already dragged on.


    Therefore, we will say goodbye and leave some material for the second part. And here we would like to ask our readers. If the topic seemed interesting to you - let us understand it, leave comments. They will help us make the second part more useful and informative, and also motivate her to release it as soon as possible.

    Thank you for your attention.

    UPD: as it usually happens, exactly the day after the article was added, an update to the SymbolSource.Server.Basic package (version 1.3.3) was released, in which the bug mentioned in the section on deploying the symbol server was fixed.

    Also popular now: