September 24, 2012 at 13:39The atypical story of a home provider
The story dates back to 2004, when affordable Internet access in cities with a population of less than a million was a rarity. In those days, “home” networks in which “providers” existed were very popular — usually students who installed a satellite dish and set up a server to distribute traffic. I also wanted to participate in this ...
2004
It will be about the city of N-sk, in which a little more than half a million people live. I will not go into the basic part of the organization of access in detail - everything was standard: after entering the university I wanted to earn money and gain experience in administration. Connected to the local house network, in which there were about 150 people. I already had ADSL at home, installed a satellite dish, bought the necessary equipment, connected to the traffic tariff from a satellite provider (PlanetSky). Since he was well acquainted with Linux, the server was organized under Debian. I used a free solution as a billing solution - Neon Internet Billing System.
2005 year
The house network grew, a couple more nearby networks were connected. Therefore, the number of subscribers has grown - I had to refine billing. Instead of a traffic tariff, it became more profitable for a satellite operator to take a conditional unlimited for a few megabits (SatGate). For the end consumer, prices ranged from 30 kopecks to the ruble depending on the time of day (for comparison, local providers offered from 3 to 7 rubles). Acceptance of payment was still carried out on the basis of the principle came with money - they put it on the account.
2006 year
The home network continued to grow. Several networks were also connected over the air. The most expensive conditional unlimited at the satellite provider has ceased to cope - it was decided to buy a dedicated band. The choice fell on RuSat, as the most affordable. Due to this, there was a great opportunity to install the ADSL + antenna + server set in other remote networks of the city - that point "flopped" in the same dedicated band. So a few more networks were connected. There are even more subscribers, connection points too. According to the previous scheme, it was impossible to accept payment. I finalized billing - an automated process of generating access cards was organized. Cards were printed on thick paper with a color laser printer, laminated, sheets were cut, corners were rounded. In place of the password was attached a scratch panel.
2007 The
subscriber base has become even larger. Expansion of the channel by a satellite operator was required. RuSat refused, because they had already allocated the entire band on the satellite. It was decided not to resort to satellite providers, but to install a server in the data center and conclude an agreement with the teleport - to get your own band on the satellite. This already allowed connecting customers throughout Russia. No sooner said than done. The first connections went, but in this situation the card went wrong. After a couple of months, the TV men came to the satellite and drove everyone away, because they were ready to pay much more. In addition, local providers began to launch the first unlimited - 128 kbit / s honest unlimited for 1000 rubles. You won’t compete with satellite ping and prices.
One of the local federal Ethernet providers allowed to conclude several contracts for one address, and also allowed to "raise" a PPPoE session with a login from another address. The picture in my head developed quickly. I sent 30 friends to conclude ten contracts. As a result, I got 300 PPPoE connections with a speed of 128 kbps each. It remained to somehow unite them into one large channel.
I rented a physical server in a data center in Moscow and wrote a module for the Linux kernel based on tun / tap, which added a virtual network interface to the system. One such interface was on my server, the other on the Moscow server, all client traffic was directed to them. Every first network packet that got into this interface was sent over the first PPPoE connection from one server to another, every second packet was sent over the second PPPoE connection and so on cyclically. Thus, I got an unlimited channel a little more than 30 Mbps (taking into account the overhead tunnel).
On the way, I met three major problems:
The first is packet shuffling. That is, the packets left the Moscow server in the order 1, 2, 3, 4, 5, 6 ... and on my server, because of the shaper of the local provider, they arrived at different times and the order could be this: 3,2,5,1 , 6.4 ... As a result, many users received the same 128 kbit / s when downloading, since Windows TCP stack does not digest such a sequence. A packet sorter was written into the module for the kernel, which accumulated a small buffer from the packets and sent them to the clients in the desired order.
The second problem was that the standard Linux shaper with a large number of static rules, a large amount of traffic and a large band overlap (rules were added at 800 Mbps, and the channel was only 30 Mbps) started to shape incorrectly. I had to write a module for Linux tc that dynamically added / removed rules in a shaper on a Moscow server, depending on its activity on the network interface.
The third problem was the monthly ratio of outgoing / incoming traffic on the Moscow server - at that time they required 1 to 3. For deviations from this ratio downwards - they required an extra charge of 30 rubles for each gigabyte of excess, and the traffic there was measured in terabytes. Since the server was just a gateway, the traffic ratio was 1 to 1. Torrents turned out to be a simple solution. Popular content was uploaded to the server and distributed in the console torrent client. Terabytes of side-by-side traffic on popular country trackers were provided.
As a result, we got great prices for the end user: 256 and 512 kbps of honest unlimited for 500 and 700 rubles, respectively. In this light, 128 kbit / s on average for 1000 rubles from local operators looked unattractive. I also had traffic and conditionally unlimited tariffs.
2008 year
In the city, I made serious competition to local providers - they launched an active offensive through the authorities. In addition, over the year of my work, the provider through whom my channel worked, began to notice that among its customers there are a couple of dozen people who have a dozen contracts, and each contract pumps the maximum possible volume around the clock, which is a decent fraction of their traffic. At the same time, there was a big drop in unlimited prices in the city. All these circumstances forced me to stop the activity.
UPD: By popular demand, in the near future the article will be supplemented with detailed information, photographs, charts and diagrams. Thanks for the feedback!
2004
It will be about the city of N-sk, in which a little more than half a million people live. I will not go into the basic part of the organization of access in detail - everything was standard: after entering the university I wanted to earn money and gain experience in administration. Connected to the local house network, in which there were about 150 people. I already had ADSL at home, installed a satellite dish, bought the necessary equipment, connected to the traffic tariff from a satellite provider (PlanetSky). Since he was well acquainted with Linux, the server was organized under Debian. I used a free solution as a billing solution - Neon Internet Billing System.
2005 year
The house network grew, a couple more nearby networks were connected. Therefore, the number of subscribers has grown - I had to refine billing. Instead of a traffic tariff, it became more profitable for a satellite operator to take a conditional unlimited for a few megabits (SatGate). For the end consumer, prices ranged from 30 kopecks to the ruble depending on the time of day (for comparison, local providers offered from 3 to 7 rubles). Acceptance of payment was still carried out on the basis of the principle came with money - they put it on the account.
2006 year
The home network continued to grow. Several networks were also connected over the air. The most expensive conditional unlimited at the satellite provider has ceased to cope - it was decided to buy a dedicated band. The choice fell on RuSat, as the most affordable. Due to this, there was a great opportunity to install the ADSL + antenna + server set in other remote networks of the city - that point "flopped" in the same dedicated band. So a few more networks were connected. There are even more subscribers, connection points too. According to the previous scheme, it was impossible to accept payment. I finalized billing - an automated process of generating access cards was organized. Cards were printed on thick paper with a color laser printer, laminated, sheets were cut, corners were rounded. In place of the password was attached a scratch panel.
2007 The
subscriber base has become even larger. Expansion of the channel by a satellite operator was required. RuSat refused, because they had already allocated the entire band on the satellite. It was decided not to resort to satellite providers, but to install a server in the data center and conclude an agreement with the teleport - to get your own band on the satellite. This already allowed connecting customers throughout Russia. No sooner said than done. The first connections went, but in this situation the card went wrong. After a couple of months, the TV men came to the satellite and drove everyone away, because they were ready to pay much more. In addition, local providers began to launch the first unlimited - 128 kbit / s honest unlimited for 1000 rubles. You won’t compete with satellite ping and prices.
One of the local federal Ethernet providers allowed to conclude several contracts for one address, and also allowed to "raise" a PPPoE session with a login from another address. The picture in my head developed quickly. I sent 30 friends to conclude ten contracts. As a result, I got 300 PPPoE connections with a speed of 128 kbps each. It remained to somehow unite them into one large channel.
I rented a physical server in a data center in Moscow and wrote a module for the Linux kernel based on tun / tap, which added a virtual network interface to the system. One such interface was on my server, the other on the Moscow server, all client traffic was directed to them. Every first network packet that got into this interface was sent over the first PPPoE connection from one server to another, every second packet was sent over the second PPPoE connection and so on cyclically. Thus, I got an unlimited channel a little more than 30 Mbps (taking into account the overhead tunnel).
On the way, I met three major problems:
The first is packet shuffling. That is, the packets left the Moscow server in the order 1, 2, 3, 4, 5, 6 ... and on my server, because of the shaper of the local provider, they arrived at different times and the order could be this: 3,2,5,1 , 6.4 ... As a result, many users received the same 128 kbit / s when downloading, since Windows TCP stack does not digest such a sequence. A packet sorter was written into the module for the kernel, which accumulated a small buffer from the packets and sent them to the clients in the desired order.
The second problem was that the standard Linux shaper with a large number of static rules, a large amount of traffic and a large band overlap (rules were added at 800 Mbps, and the channel was only 30 Mbps) started to shape incorrectly. I had to write a module for Linux tc that dynamically added / removed rules in a shaper on a Moscow server, depending on its activity on the network interface.
The third problem was the monthly ratio of outgoing / incoming traffic on the Moscow server - at that time they required 1 to 3. For deviations from this ratio downwards - they required an extra charge of 30 rubles for each gigabyte of excess, and the traffic there was measured in terabytes. Since the server was just a gateway, the traffic ratio was 1 to 1. Torrents turned out to be a simple solution. Popular content was uploaded to the server and distributed in the console torrent client. Terabytes of side-by-side traffic on popular country trackers were provided.
As a result, we got great prices for the end user: 256 and 512 kbps of honest unlimited for 500 and 700 rubles, respectively. In this light, 128 kbit / s on average for 1000 rubles from local operators looked unattractive. I also had traffic and conditionally unlimited tariffs.
2008 year
In the city, I made serious competition to local providers - they launched an active offensive through the authorities. In addition, over the year of my work, the provider through whom my channel worked, began to notice that among its customers there are a couple of dozen people who have a dozen contracts, and each contract pumps the maximum possible volume around the clock, which is a decent fraction of their traffic. At the same time, there was a big drop in unlimited prices in the city. All these circumstances forced me to stop the activity.
UPD: By popular demand, in the near future the article will be supplemented with detailed information, photographs, charts and diagrams. Thanks for the feedback!