October 15, 2014 at 15:09IIS as an Edge Web Server (now haproxy)
In this article I want to describe not the most typical scenario, which, however, has the right to life.
The fact is that we use IIS as a proxy for other company web servers. I will tell you how it was implemented and what difficulties I had to face.
Problem statement
Let us examine the example of the YouTrack server. It is represented by the unsightly srv-youtrack-01.local.domain and is located on a web server inside the company. The task is to provide access to it from the Internet by the beautiful name yt.company.ru. In this case, https must be used.
Implementation
To get started, we need to install the URL Rewrite component . This can be done using the web platform installer, as well as manually. After installing it, we’ll see a new
URL rewrite shortcut in IIS Manager .
Using this tool, you can create a rewrite address rewrite rule.
When creating a rule, you must specify the server URL (without the prefix http: // - its IIS will automatically add) to which proxying will occur. As a result, we will get a rule that is editable. It does not apply to all requests, but only to those that meet the criteria that we can configure. First, we check the URL for compliance f pattern, and then in the course are checking on other criteria.
I must say right away that there are two ways: the first way is to create a set of rules with different URL patterns for different resources on the same IIS site; and the second one is to create a site for each proxied resource and make one rule in each of them. Realizing that the first path is more Jedi, I nevertheless chose the second path - though not so beautiful, but I do not risk writing the wrong regular expression for one site to break the entire routing. Therefore, the URL pattern is defaulted everywhere to me "(. *)".
So, I create the site yt.company.ru with binders for port 80 and 443 and the obligatory indication of the host name so that IIS knows which site I am accessing. About getting and installing a certificate for 443 let me not mention. I’ll only draw attention to the fact that you don’t need to configure the service to use https - there is no one to encrypt from the inside of the network, and external requests will be connected via ssl to the edge server, which will proxy requests within the network via an insecure channel.
As long as the mandatory requirement is to use https, we will proxy only those requests that come to port 443, for which we will create a simple condition. When you create it, a drop-down list of possible options is offered.
Ok, now all the yt.company.ru requestsproxied to the internal server with the unsightly name srv-youtrack-01.local.domain transparent to the user.
However, all yt.company.ru requests are cut off with a 403 error, which is not very nice. To solve this problem, you can either create index.html with a redirect, or another URL Rewrite rule, in which we select the permanent redirect to the URL we need in the "action" field.
It should be noted that the rules for the site are applied in order, so first you need to arrange the rule with a condition, and then the rule without conditions. At the same time, since the second rule applies to all URLs without exception, for the first rule it is necessary to check (leave the box checked) “Stop processing subsequent rules”.
After manipulating the graphical interface, a web.config is created at the root of our site, which contains all the created rules. Therefore, if you need to proxy another site, then you do not need to repeat these manipulations; you can simply copy web.config and change the required URL in it, or use the graphical interface to change the rules after copying. Moreover, you can don’t use the interface at all, but write right there - to whom you like.
Pitfalls
When you click on the Agile boards tab, YouTrack generates a URL of the form yt.company.ru/rest/agile/Overview-0/sprint/Iteration+24 . Further, when switching between sprints yt.company.ru/rest/agile/Overview-0/sprint/Iteration%252023?q=. When switching to these URLs, IIS began to return 404 errors to me. This indicated that requests were not proxied. At the same time, transitions between saved queries of the form yt.company.ru/issues/IT?q=%23 {Current + work} + Assigned + to% 3A + me + updated% 3A + {This + week} worked out quite correctly.
The experiments with adding a question mark in the middle of the problematic URL ended with the fact that I began to receive a 404 error already from the YouTrack server, and not IIS. This gave me the idea that IIS for some reason (hi, Microsoft) interprets the URL and this needs to be fixed.
The problem with the plus sign in the middle of the address was solved by adding the requestFiltering allowDoubleEscaping = "true" parameter :
But after that, switching between sprints still did not work. It turned out that IIS considers such requests unsafe. This check also had to be disabled:
This is what web.config turned out after all the manipulations:
Results.
Probably, the solutions that I found are not optimal and instead of resolving everything in a row, it was necessary to carefully prescribe the rules that were suitable in a particular case. But now it works. I will be glad to hear your thoughts and suggestions.
Thus, absolutely all web servers that need external access are proxied in our organization, among which there are nginx, and apache, and svn, and gitlab, and exchange web access.
The main problem that makes me look for a solution is that NTLM authentication, which is so necessary for many Microsoft services, does not work through a proxy. I don’t want to use a dead TMG product, so now I'm trying to figure out a new Windows Server 2012 R2 service called Web Application Proxy looking at nginx and apache at the same time, which, it seems, also do not yet know how to proxy NTLM.
http://www.ifinity.com.au/Blog/EntryId/60/404-Error-in-IIS-7-when-using-a-Url-with-a-plus-sign-in-the-path
stackoverflow .com / questions / 2831142 / asp-net-4-url-limitations-why-url-cannot-contain-any-3f-characters
Big update:
In the comments I was advised to try haproxy. Having visited the site, I searched the ntlm page and found “full HTTP keep-alive for better support of NTLM and improved efficiency in static farms”, which gave me confidence.
After several days of active fuss in the console, I mastered this wonderful tool and now I no longer need IIS as a proxy server. I think it's not worth writing a separate article about this, so I decided to update the topic.
This whole thing works quite simply and out of the box:
1. It is installed from backports using apt-get (I prefer debian)
2. A config is written. Settings of proxied applications are slightly corrected
3. iptables switches to a new proxy.
On the second point I will dwell in more detail.
I added the config to the defaults section
The last item is needed so that the host name is passed to the backend, the rest - “like everyone else”.
Next, frontends for 80 and 443 ports are created, which will listen and decide which backend to send the request to, depending on some conditions. And I have only one condition - the host name has arrived.
With https it's a bit more complicated. A neighboring topic came to the rescue , in the comments of which it was recommended to use SNI. And used it
It turned out to be very simple! First of all, certificates are generated for all backends - they will be given to customers. I use the PKI from Microsoft, so I had to tinker a bit with generating requests, issuing and transferring these certificates to proxies. By the way, the use of * .company.name is allowed, but I decided that it was somehow not very solid, especially with such a small number of backends. After the certificates are ready, you need to write them stupidly in the line as in the example above, and then write the rules for backends - the certificates will be slipped in order.
The design using sni is so simple that you don’t even have to explain. True, it turned out that most android email clients do not know how (or do not want) sni, and send requests to port 443 without specifying a host name. No problem! For such cases there is
(
By the way, I did not check which certificate is slipped in this case) Well, the backends are described further. With http, everything is trivial:
Here it.company.name is the host name that will be transferred to srv-web-01. I need this because IIS on this server uses host name authentication.
For https, these are the designs
Here you can unload SSL by specifying port 80 - then traffic between the client and proxies will be encrypted, but will not be inside the network. And you can continue to use https ( verify none means "do not find fault with the certificate"). However, it should be understood that the client still receives the certificate that we entered when creating the frontend. If you need to palm off the certificate of the destination server, then you can use the method described in the topic that I indicated above.
Another point: I want to beautifully redirect http to https for some servers. To do this, I created special backends with the _r postfix , which carefully throw the unsuspecting user to https.
I didn’t consciously remove the commented-out line - such an option was originally used, but it redirected me to the root of the site, which is very inconvenient if the user clicked on the long link http: //site.company.name/lib/doc/Russian%20 letters%20в% 20 titles.docx, and he was thrown to the main page without any hope of finding his document. It is likely that he will close it and try to follow the link again, but again he won’t get anything and will be very upset. To prevent this from happening, the redirect scheme https construct helps us , which gently redirects the user, substituting the entire URL.
Details on all the subtleties of configuration on the documentation page cbonte.github.io/haproxy-dconv/configuration-1.5.html#4.2
Thanks for attention. I hope someone will find my experience useful.
The fact is that we use IIS as a proxy for other company web servers. I will tell you how it was implemented and what difficulties I had to face.
Problem statement
Let us examine the example of the YouTrack server. It is represented by the unsightly srv-youtrack-01.local.domain and is located on a web server inside the company. The task is to provide access to it from the Internet by the beautiful name yt.company.ru. In this case, https must be used.
Implementation
To get started, we need to install the URL Rewrite component . This can be done using the web platform installer, as well as manually. After installing it, we’ll see a new
URL rewrite shortcut in IIS Manager .
Using this tool, you can create a rewrite address rewrite rule.
When creating a rule, you must specify the server URL (without the prefix http: // - its IIS will automatically add) to which proxying will occur. As a result, we will get a rule that is editable. It does not apply to all requests, but only to those that meet the criteria that we can configure. First, we check the URL for compliance f pattern, and then in the course are checking on other criteria.
I must say right away that there are two ways: the first way is to create a set of rules with different URL patterns for different resources on the same IIS site; and the second one is to create a site for each proxied resource and make one rule in each of them. Realizing that the first path is more Jedi, I nevertheless chose the second path - though not so beautiful, but I do not risk writing the wrong regular expression for one site to break the entire routing. Therefore, the URL pattern is defaulted everywhere to me "(. *)".
So, I create the site yt.company.ru with binders for port 80 and 443 and the obligatory indication of the host name so that IIS knows which site I am accessing. About getting and installing a certificate for 443 let me not mention. I’ll only draw attention to the fact that you don’t need to configure the service to use https - there is no one to encrypt from the inside of the network, and external requests will be connected via ssl to the edge server, which will proxy requests within the network via an insecure channel.
As long as the mandatory requirement is to use https, we will proxy only those requests that come to port 443, for which we will create a simple condition. When you create it, a drop-down list of possible options is offered.
Ok, now all the yt.company.ru requestsproxied to the internal server with the unsightly name srv-youtrack-01.local.domain transparent to the user.
However, all yt.company.ru requests are cut off with a 403 error, which is not very nice. To solve this problem, you can either create index.html with a redirect, or another URL Rewrite rule, in which we select the permanent redirect to the URL we need in the "action" field.
It should be noted that the rules for the site are applied in order, so first you need to arrange the rule with a condition, and then the rule without conditions. At the same time, since the second rule applies to all URLs without exception, for the first rule it is necessary to check (leave the box checked) “Stop processing subsequent rules”.
After manipulating the graphical interface, a web.config is created at the root of our site, which contains all the created rules. Therefore, if you need to proxy another site, then you do not need to repeat these manipulations; you can simply copy web.config and change the required URL in it, or use the graphical interface to change the rules after copying. Moreover, you can don’t use the interface at all, but write right there - to whom you like.
Pitfalls
When you click on the Agile boards tab, YouTrack generates a URL of the form yt.company.ru/rest/agile/Overview-0/sprint/Iteration+24 . Further, when switching between sprints yt.company.ru/rest/agile/Overview-0/sprint/Iteration%252023?q=. When switching to these URLs, IIS began to return 404 errors to me. This indicated that requests were not proxied. At the same time, transitions between saved queries of the form yt.company.ru/issues/IT?q=%23 {Current + work} + Assigned + to% 3A + me + updated% 3A + {This + week} worked out quite correctly.
The experiments with adding a question mark in the middle of the problematic URL ended with the fact that I began to receive a 404 error already from the YouTrack server, and not IIS. This gave me the idea that IIS for some reason (hi, Microsoft) interprets the URL and this needs to be fixed.
The problem with the plus sign in the middle of the address was solved by adding the requestFiltering allowDoubleEscaping = "true" parameter :
But after that, switching between sprints still did not work. It turned out that IIS considers such requests unsafe. This check also had to be disabled:
This is what web.config turned out after all the manipulations:
Results.
Probably, the solutions that I found are not optimal and instead of resolving everything in a row, it was necessary to carefully prescribe the rules that were suitable in a particular case. But now it works. I will be glad to hear your thoughts and suggestions.
Thus, absolutely all web servers that need external access are proxied in our organization, among which there are nginx, and apache, and svn, and gitlab, and exchange web access.
The main problem that makes me look for a solution is that NTLM authentication, which is so necessary for many Microsoft services, does not work through a proxy. I don’t want to use a dead TMG product, so now I'm trying to figure out a new Windows Server 2012 R2 service called Web Application Proxy looking at nginx and apache at the same time, which, it seems, also do not yet know how to proxy NTLM.
References
http://www.ifinity.com.au/Blog/EntryId/60/404-Error-in-IIS-7-when-using-a-Url-with-a-plus-sign-in-the-path
stackoverflow .com / questions / 2831142 / asp-net-4-url-limitations-why-url-cannot-contain-any-3f-characters
Big update:
In the comments I was advised to try haproxy. Having visited the site, I searched the ntlm page and found “full HTTP keep-alive for better support of NTLM and improved efficiency in static farms”, which gave me confidence.
After several days of active fuss in the console, I mastered this wonderful tool and now I no longer need IIS as a proxy server. I think it's not worth writing a separate article about this, so I decided to update the topic.
This whole thing works quite simply and out of the box:
1. It is installed from backports using apt-get (I prefer debian)
2. A config is written. Settings of proxied applications are slightly corrected
3. iptables switches to a new proxy.
On the second point I will dwell in more detail.
I added the config to the defaults section
mode http
balance roundrobin
option redispatch
http-send-name-header Host
The last item is needed so that the host name is passed to the backend, the rest - “like everyone else”.
Next, frontends for 80 and 443 ports are created, which will listen and decide which backend to send the request to, depending on some conditions. And I have only one condition - the host name has arrived.
frontend http
bind *:80
#Define hosts
acl host_yt hdr(host) -i yt.company.name
acl host_ar hdr(host) -i ar.company.name
acl host_portal hdr(host) -i portal.company.name
acl host_crm hdr(host) -i crm.company.name
acl host_git hdr(host) -i git.company.name
acl host_mail hdr(host) -i mail.company.name
...
use_backend yt if host_yt
use_backend ar if host_ar
use_backend crm_r if host_crm
use_backend git_r if host_git
use_backend mail_r if host_mail
With https it's a bit more complicated. A neighboring topic came to the rescue , in the comments of which it was recommended to use SNI. And used it
frontend https
bind *:443 ssl crt /etc/ssl/tfs.cer.ipk.pem crt /etc/ssl/yt.cer.ipk.pem crt /etc/ssl/crm.cer.ipk.pem crt /etc/ssl/git.cer.ipk.pem crt /etc/ssl/mail.cer.ipk.pem
use_backend tfs if { ssl_fc_sni tfs.company.name }
use_backend yt if { ssl_fc_sni yt.company.name }
use_backend crm if { ssl_fc_sni crm.company.name }
use_backend git if { ssl_fc_sni git.company.name }
use_backend mail if { ssl_fc_sni mail.company.name }
It turned out to be very simple! First of all, certificates are generated for all backends - they will be given to customers. I use the PKI from Microsoft, so I had to tinker a bit with generating requests, issuing and transferring these certificates to proxies. By the way, the use of * .company.name is allowed, but I decided that it was somehow not very solid, especially with such a small number of backends. After the certificates are ready, you need to write them stupidly in the line as in the example above, and then write the rules for backends - the certificates will be slipped in order.
The design using sni is so simple that you don’t even have to explain. True, it turned out that most android email clients do not know how (or do not want) sni, and send requests to port 443 without specifying a host name. No problem! For such cases there is
default_backend mail
(
By the way, I did not check which certificate is slipped in this case) Well, the backends are described further. With http, everything is trivial:
backend it
server it.company.name srv-web-01
backend ar
server ar.company.name srv-web-01
Here it.company.name is the host name that will be transferred to srv-web-01. I need this because IIS on this server uses host name authentication.
For https, these are the designs
backend yt
server yt.company.name srv-youtrack-01:80
backend tfs
server tfs.company.name tfs:443 ssl verify none
Here you can unload SSL by specifying port 80 - then traffic between the client and proxies will be encrypted, but will not be inside the network. And you can continue to use https ( verify none means "do not find fault with the certificate"). However, it should be understood that the client still receives the certificate that we entered when creating the frontend. If you need to palm off the certificate of the destination server, then you can use the method described in the topic that I indicated above.
Another point: I want to beautifully redirect http to https for some servers. To do this, I created special backends with the _r postfix , which carefully throw the unsuspecting user to https.
backend tfs_r
#redirect location https://tfs.company.name code 301
redirect scheme https
I didn’t consciously remove the commented-out line - such an option was originally used, but it redirected me to the root of the site, which is very inconvenient if the user clicked on the long link http: //site.company.name/lib/doc/Russian%20 letters%20в% 20 titles.docx, and he was thrown to the main page without any hope of finding his document. It is likely that he will close it and try to follow the link again, but again he won’t get anything and will be very upset. To prevent this from happening, the redirect scheme https construct helps us , which gently redirects the user, substituting the entire URL.
Details on all the subtleties of configuration on the documentation page cbonte.github.io/haproxy-dconv/configuration-1.5.html#4.2
Thanks for attention. I hope someone will find my experience useful.