Experience in creating frameworks

I have been working in a company that has been developing custom accounting systems for quite some time. About the same amount of time there is a technology department - a team of people who develop tools and frameworks for their comrades, and those in turn create systems for customers. It can be said briefly that we (technologists) unify the process of writing business logic using our own ORM, unite designers and programmers through a UML editor with code generators, and provide various UI controls that allow users of end applications to work productively.
In this article I want to share my experience regarding the process of creating a technological layer.


Why do we need frameworks?


There are several reasons why the company contains several people who do not bring direct profit, here, in my opinion, the most important thing:
1. Reuse of ideas and code. The technology department collects best practices as various projects are completed and applies them to all teams. In this case, there is no need to arrange each one’s connections with each other in order to be aware of what's new and useful for the neighbor. After all, a neighbor can immediately share with technologists and not bother much about universalization, and technologists have the opportunity to inform everyone else (a list of improvements that hit the next version of technological products is periodically formed).
2. Unification of decisions. Programmers sometimes change jobs. To deal with the negative aspects of such situations, the technology provides some unified methods for solving typical problems. Thus, the number of bicycles is reduced and the time for a new person to enter the project is reduced.

Points of growth or growing technology


A little about how we are emerging a new technological direction. From time to time, the platforms used change and you have to create technology for these new products.

Kindergarten

Due to the fact that our product is not a complete application, any subject area for a prototype project (1-3 pcs.) Is selected first. This application serves only to verify the solutions and approaches that we plan to implement. For fun, the last such system we used was for keeping domestic cats in the apartment. A feature of prototypes is that the code is rather mixed and it is not clear where the technological layer is, and where the applied logic is already starting. On this "cat" project, all the necessary components, generators, user interface elements are being worked out. Only technologists work on the project.

School time

When it comes time to try out the technology for something more serious, a “sacrificial” project is selected from the commercial ones. As a rule, it has a 1.5 - 2 times longer period that can be set aside for development than a typical project (everyone understands that there are risks, because we use the technology for the first time and the company itself pays for the technological work, not customer). A feature of such a project is that the technological code is initially in solution, and as it is developed, it is divided into technological projects and application code (which depends on the subject area). Both technologists and applied programmers are working on the project. It should be understood that the basic approaches are still being debugged and not everything in the technology is 100% ready, so the project will be with inborn “bicycles” that complicate the support.

Higher education

At this stage, we believe that the technology is ready for widespread use. We pass into the normal "operating mode". Applied developers no longer need direct assistance from technologists. Technological assemblies are delivered in compiled form. Finalization tasks and error messages are transferred to the TFS-technology project.

Professional Development

From time to time, serious refinements of the technology are required, as a rule, this is adding functionality, for example, a general reporting subsystem, authorization subsystem, logging subsystem, etc. ... We try to use something like Feature driven development: the goal is to realize the required feature entirely, and for users, it appears already in finished form. As a rule, such innovations immediately become popular with all projects that are actively developing. It's funny that sometimes, at the end of the development of another feature, an applied programmer comes and asks to implement just that.

Retirement

As new directions develop, the old ones gradually come to naught. Projects are closed, there are units of projects that require support. Together with them, the technology "retires", goes into Bug driven development mode. New features are no longer added, only fixing old bugs that accumulate over the years. Some bugs are fixed by crutches in the application project, since it is no longer practical to make changes to the technology. Changes in technology should always be made with an eye to all projects that depend on it. One wrong move and problems are guaranteed both to people on the application project, and after them and the technologists.
If you still need to rewrite something, then for a technology with many occurrences in projects:
• We collect the requirements that the old functionality met (as a rule, after a series of improvements and small corrections of complete and up-to-date documentation for the programmer, there are no more)
• We plan work (we look at the budget again and evaluate the appropriateness)
• We write tests to fix the condition. We will determine from them that what worked before will not break.
• We write functionality
• We check the quality and compliance with the stated requirements

The technical side of the issue


Now a little about techniques and techniques. It will be about the Microsoft .NET Framework, because our solutions are built on this platform.

Extension points

The technology should not limit programmers, but give advantages over a solution from scratch, and sometimes completely turnkey solutions. In order for the programmer to be able to show his personality where the technology is “too standard” we use the following approaches:
• Inheritance. There is a base class that implements standard behavior and is located in the technological assembly. The code generated for the application project is inherited from the base class and in special methods involves changing the standard behavior.
• Events. Technology classes contain a series of events, the processing of which allows you to change the behavior. It should be remembered that an event can be subscribed many times.
• Delegates. Similar to events, but there can only be one implementation. This prevents unwanted operation several times.
• App.config. Various little things get into the configuration file. As a rule, these are some flags and paths (whether it is necessary to write debugging information, connection strings to services, etc.).
• DI. This option involves overriding the whole module.

Event logging

From the technology layer, there is rarely normal access to the user interface. As a rule, if something went wrong, it indicates an error either in technology (yes, they exist and we understand it), or in the application code (in the ideal case, you can tell the application programmer explicitly what he is doing wrong and in what place). In any case, you need to somehow signal this. We use Log4net. To highlight technological messages, you can use either a separate logger or fix the level of errors, for example, Warn, to the technology.

XML comments

XML comments in process code are crucial. It is necessary to prescribe them for public classes, methods and properties. This will allow you to generate help (http://sandcastle.codeplex.com/) and provide tips in application projects thanks to IntelliSense. Do not forget to indicate in the project settings whether to generate an xml file with comments.

Breaking changes

It is extremely rare that such a change in technology occurs that requires the intervention of an application programmer to update the technology layer. If such an update can be predicted, then you need to warn in advance, for example, using the Obsolete attribute:
[Obsolete (“This function will be removed in the next version, use the function”)]
Sometimes you have to go on a lead and apply the rule: “it’s better to add 1 time, than 2 times to update ", ie if the method signature is used in 1000 places in each application project, changing it in technology is not a good idea. The most important rule: always a detailed description of possible problems with the update - warned, then armed.

Documentation

For technological assemblies, the availability of documentation is extremely important. To store information, we use a Wiki engine with write access from technologists. To keep the information up to date, 2 rules apply:
1. Did the job - write it in the Wiki!
2. Answered the question - write in the Wiki!
Description levels:
1. For myself (how it works)
2. For programmers of applied projects (how to use methods and classes)
3. For users of systems (how to use components)

Framework project structure

Sometimes it is required to minimize the number of assemblies issued for projects. This allows you to deal with incorrect updates when you need to update several assemblies at once, otherwise something will not work and minimizes the risk that something will be forgotten to be installed by the customer. However, different projects need slightly different functions. What can be applied?
1. Links to cs-files between projects (work with projects is a little complicated)
2. ILMerge (this can be entrusted to the Build server, but debugging is difficult)
3. Assemblies in resources (with a separate assembly loading mechanism)

Signing assemblies or Strong name

Another iron rule: all technological assemblies must be signed. This gives us the following advantages:
1. Signed assemblies are uniquely identified in the GAC
2. You can sign only if all dependent assemblies are signed (application assemblies cannot be signed if process assemblies are not signed)
3. Some degree of protection (in order to change the code, it will be necessary to rebuild all application assemblies)
If it happens that the code from the assembly is lost, then signing can be done by rebuilding the assembly. Reassembly is performed in this way (to specify a file with a key, look for a special parameter):
1. ildasm.exe sampleA.exe / source /out:sampleA.il
2. ilasm.exe sampleA.il / exe /out:sampleA.exe

Versioning Strategy

The more often a product is released, the less it seems to users that programmers do nothing. We tested 2 versions of the release of technological product versions:
1. The latest builds are the best.
Everything that is placed in SourceControl is considered proven and application programmers can take any server build of technological assemblies. At the same time, technologists must carefully check their changes before each Check-in.
Pros:
• Instant publication of changes
Cons:
• Each Check-In requires extensive verification with a lot of different functionality that may suffer
• Errors can easily penetrate the customer’s version
2. Periodic release of a stable version
Release of a version of technological assemblies on a periodic basis.
Pros:
• More reliable integration verification
Cons:
• There is a risk of losing control of the code in the middle of the iteration
• It takes time to check everything again, no instant corrections
During the release of the 1-2-day version, all technologists are distracted by checking and correcting the errors found. This is the option we are currently using.

Assembly versions

There are 2 strategies for designating assembly versions:
1. Changeable versions of assemblies
• Updating application projects only with recompilation of the project (inability to send a patch in technology directly to the customer)
• Some problems will be visible during compilation (relevant for complex dependencies between assemblies)
• Problem with type serialization ( serialized data stored by the customer can cause problems when updating versions of assemblies)
2. Fixed versions of assemblies
• Updating a simple substitution of assemblies
• Problems will be visible at startup because recompilation during the upgrade is not required
• You can only send the corrected assembly to the customer as a last resort
• The release date of the assembly as a conditional version for identification
Our team settled on the second option because often it is necessary to check new technological assemblies on applied projects, and there is no desire or time to recompile them. We have server builds configured in Team Foundation Server and only the server build can be transferred for implementation.

Branching: Pros and Cons

We considered 2 approaches to organizing the storage of source code:
Branches for adding features, branches for implementations
Plus:
• You can make a small change directly to the desired version
Minus:
• You need to perform many code merges
Single version without branches
Plus:
• No merging
Minus:
• Version , which is in a disassembled state, can not be quickly repaired.
We settled on a simpler second option. We try not to make long-term changes without Check-ins, but also Check-ins should not ruin the performance of our solutions.

External assemblies

A few words about OpenSource in a commercial product: if you wish, you can find OpenSource solutions with a good license for solving any desired task. However, when making such a decision, you should think hard about how the changes will be made there:
1. Edit everything locally
Plus:
• You can do anything
Minus:
• When updating the official version, you will have to merge your changes yourself
2. Send updates to the team supporting the
Plus project :
• An additional check on the quality of corrections
Cons:
• The risk that support will cease
• M.b. long check and release fixes
• Corrections can be rejected for one reason or another.
Sometimes you have to make a choice in favor of a paid solution only because the policy of making changes (at least fixing errors) is fixed in the license for which money is paid.

Obfuscation

How to protect the code that is available from both application and technology assemblies from disassembling?
You can obfuscate technological assemblies before transferring to a project, but then we get:
Plus:
• Application programmers will not stick their nose into the technological code and ask silly questions about this
Minuses:
• Debugging complexity for application developers
• Possible instability during development
• Obfuscation and the check will be performed twice. The
second option is to obfuscate the whole finished application with all the technological assemblies.
Pros:
• Obfuscation during the build phase of the application
• The ability to debug an application by entering the framework code. Verification of the application is performed once.
Minus:
• Application programmers are involved in the protection of technological assemblies.
We use the latter option.

Testing

I will not talk about unit tests - any layer needs this, not just the technological one. However, technological code should always be written taking into account performance requirements and work flawlessly in multi-threaded mode.
Testing of technological projects, as a rule, is impossible without the final application, which describes the subject area. In some cases, in order to understand exactly where the error is hidden, you have to create your own separate application and try to repeat the necessary conditions. Only this option helps to separate the "tips" from the application code.

Conclusion


I can note that working on the technology is a very fascinating and, in some places, fun activity. The main thing is not to get carried away too much and sometimes communicate with end users, trying to understand their needs. Thanks to everyone who read this long monologue to the end.

Also popular now: