How much are clouds for the people now and where to apply them

On a long weekend, the noble dons had absolutely nothing to do except go to visit the very wealthy moles and count.

It was decided to calculate whether the clouds are so expensive on the example of Azure and Office 365, why so many people are running there, what are the pros and cons. What to read, where to dig, where is the warehouse for shovels.

Just before the new year, a holivar happened - Microsoft released Azure TCO Calculator, and VMware said "you're lying," which can be read in detail here.


TL / DR - It turns out a little expensive anyway, but there are plenty of pluses. A server made of matches and acorns, with software on a wooden leg and an amateur lover comrade V. , instead of a system administrator, will come out of course cheaper.

The text came out chaotic, because one was planned, and another came out.

Theory

T1. Introduction
T2. Basic (starting) configuration.
T3 What services can be transferred or reserved in the cloud at the initial stage and what is the use of it.
T4 The resulting configuration.
T5 Training courses that must be completed to achieve the resulting configuration.
T6 Pricing for virtual machines.
T7 Estimated cost for virtual disks.

T1. Introduction
If the office of a young dynamically developing company has its own software-defined data center (1), a number of problems arise, the main of which is that most of the determinability of this data center is determined in the well-known yellow program, to which there are no normal methods tests, no methodologies, except for one product on G with recommendations of the same quality and the same production (2). But let's not talk about sad things, especially in a virtualization environment.

During the operation of something larger (and most importantly - in the licensed field), expenses for everything appear:

- Electricity (including UPS).
- cooling (maintenance of air conditioners).
- security (fireproof doors, extinguishing - Novec 1230, ACS).
- Connection / Client Access License (CAL) for each sneeze.

Of course, the costs should include the hardware itself, and spare parts, the cost of operation, a system for reporting failures (monitoring of all systems as a whole).

If you assemble systems not from matches and acorns, then it’s worth looking at the cost of additional support purchase - NBD (Next business day) can cost quite beyond humanity.

A plus is the need for a person who arrives “right now”, who does not drink, does not smoke, is on duty on holidays.

And this I have not yet moved to the issue of licensing and configuring applications at startup, and the issue of supporting the service outside of startup.

Of course, if you already have a separate team and everything is licensed and purchased, the budget has been formed, then there is no subject of reasoning “now”. Or if you reacted to the idea of ​​licensing without interest.

If everything is not so simple, then you should think (and calculate) this: - let's say, after 2-3 years (the depreciation of existing equipment, including batteries in UPSs, freon in air conditioners) we decided to calculate how much we would get up to moving to the cloud, what risks does this add / remove, and how to live with it.

Options for deploying all kinds of clouds now ... it’s easier to list:

- Own undercloud - 1..10 servers in a VMWare / Hyper-V / KVM-openstack cluster (3)
- a hybrid structure with the ability to back-up to the cloud.
- full cloud deployment. It seems that the same Aplle does not live on its data centers.

T2. Basic (start-up) configuration.
For example, take a small company whose main tasks (business) are far from IT. What services are they deployed?

- AD and everything is near (certificates, trusts, federation is possible)
- Exchange (including some kind of antispam if they do not use cloud) - including with remote access (OWA + office)
- File server
- remote access to systems (Anyconnect, RRAS, OpenVPN and so on)

- accounting and other accounting
- Miscellaneous consultant plus other help systems (TechExpert), HR systems such as E-Staff Recruiter (4) and so on.
- the main working tool (from car dealerships to inventory and cash registers)
- Internet access
- access and control systems, other utility systems (UPS, skud, monitoring, video surveillance (CCTV - Closed Circuit Television), backup, etc.).

T3 What services can be transferred or reserved in the cloud at the initial stage and what is the use of it
? Personally, I think that it is easiest to drag at least AD into the Azure cloud as a service, and possibly email. The file server is not sure. Dragging the file server under a big question because of the communication channels in the first place, and the issue of organizing backup in the second.

Here's the price tag and mnogabukaf about Azure AD - The main disadvantages after the first reading
- It is not clear how to manage and replicate with the main DC.
- It’s completely not clear how to backup.
- according to this table (above) the price per month is not clear.
- the update model is not clear, the same WSUS for virtual machines, how to tie it to the policy of the main organization. And whether it is necessary.

Of course, there is a free trial of Azure Active Directory Premium for one month .

But this is only a month, and then what? No, for full-fledged training and self-training, you need a subscription to some kind of cloud-based mini laboratory. Of course, there is Azure Spark (or whatever it is called), there is training (6).

There are courses, but with a subscription everything is not obvious. There is Visual studio enterprise with MSDN (NFR) - but not free.

Email is easier -migration to office 365 is described, for example, here .

T4 Resulting configuration
Resulting configuration - hybrid, Hybrid:
- One DC on the ground (on-premise), one in a virtual machine in Azure and one service in Azure
- A small file server in Azure. It could be great, of course, but then you will have to deploy something like RDP Azure, and not to mention pre-training systems in the press, construction and so on (all kinds of Adobe Indesign, Synchro, Autodesk Revit, and other NX / Unigraphics with 50 gig files).
- Backups on the ground (on-premise)
- Duplicate mailboxes (2 pieces) in office365, in another mail domain, but somehow synchronized with the ground.
- Communication with Azure - VPN or ExpressRoute (but this is not for the Russian Federation)
times, Two .

T5 Training courses that must be completed to achieve the resulting configuration.
Everything is very bad. The page in Russian in white says "Getting Started with Free Online Courses" - but on a click it switches to Managing Infrastructure with Microsoft Azure - Getting Started, where there is a free watch button on pluralsight. Personally, it would be more convenient for me to watch on Youtube, but no. Sadness and trouble.
But for example, Azur channel9 lies partially on youtube, and you can download videos.

True, there is Udemy - also in English.

So in Russian with training it’s still sad, not every Momkinthe architect will master. But there is a solution - e-books (EPUB, MOBI, PDF) for Windows Azure in Russian!
one , two .

Windows Azure internal device ( in Russian ).

Here you also need to look - A selection of the best online courses on Microsoft Azure .

There are (in Russian) meetings from COMPAREX, a couple of times a year, for example,
once , twice . In general, you should often look at
blogs.technet.microsoft.com/rutechnews and azurenotes.tech/ServiceNotes .

The problem is there is no single course / pill.

T5.-2. Docking with the ground
Everything is simple. ICND1 + ICND2 + CCNA Security + LiftMeUp + Wendell Odom
And a separate new topic - Azure NSG - Network Security Group (NSG) and Azure Network Security Groups (NSG) - Best Practices and Lessons Learned .

T6 Pricing
Everything is very bad. Just got to this point, as already laid out. New Year's Holivar: Microsoft released Azure TCO Calculator, and VMware said “you're lying” - see further (7).

Truth, as usual, is somewhere on a different plane. For a somewhat developed infrastructure, you would need to have a vSphere Distributed Switch - and this is a vSphere Enterprise Plus license, the price tag for which is completely equine, and in general it might be worth considering the purchase of MS Server 2016 Datacenter.

Moreover, this cost estimate has nothing to do with attempts to estimate the total (including electricity and air conditioning) cost of ownership of the equipment on the ground, and this is not to mention the risks listed in the article “9.5 Rules for Secure IT Business in Russia”.

You can count "very roughly." Let a physical server with 50 (virtual) machines consume 500 watts, 10 watts per virtual machine, 240 watts / day (plus cooling 30%, but we won’t take it yet), 7 kWh per month, 5 r / kWh , 35 rubles per month, 420 per year.

The cost of licenses is more complicated. Let's say we have 40 cores (2 * 20) and we are licensed under Win Datacenter:

$ 6.155 for the first 16 cores (6155 * 60 = 369300, rounded 370)
www.microsoft.com/ru-ru/cloud-platform/windows-server -pricing

plus (40-16 = 24 cores not licensed, packs of 2 cores = 12 packs
- 9EA-00128 WinSvrDCCore 2016 SNGL OLP NL 2Lic CoreLic = 40000 * 12 = 480000.
Total 370 + 480 = 850 t / for 5 years, 170 / year, 3400 per machine

For, conditionally, 5 years each (per year, and 50 machines per physical host - 3246.32 per virtual machine.
Sample .

Of course, we would also have to charge for disk costs and server amortization (the same 5 years). Let the server stand ... HP DL360 Gen9 server - from 250t, no drives. For a year - 50,000, per machine - 1000.

Drives - 697574-B21 Gen8 HP 1.2T 10K 2.5 SAS costs about 20t.r., 60 GB of capacity (for the role only AD) will cost 20/20 - 1000r, but it takes 2 disks for 5 years, taking into account raid10 - coefficient so the price is 4, and 60 GB will cost 4000 rubles.

Total - 400 + 3400 + 1000 + 4000 - 8800 per year per 5 years.

A B2S machine will cost $ 52 per month), an S6 disk (64 GB) - another 5, and other expenses that the calculator considers are worth $ 58-60 a month, or (tadam) $ 720 or (course 60) 433200 / year .

So Azure costs almost 5 times more expensive than its own ground server.

BUT, there are also methods against Kostya Saprykin. For example, if we need fault tolerance, then we will have to license all the cores on the second same server, and store another copy there (via vsan or Storage Spaces) - which simply doubles the cost of the solution. And if we have done everything on top of Vmware, and even on top of vSphere Enterprise Plus, and even purchased vSan separately, then the price tag will be even more horse-drawn, and if the disks are in a disk shelf (and not just crammed into a server), then it would be necessary to calculate the depreciation of the shelf, and this is if it is directly connected, without a switch for a lot of money (10G cards in the latest intel platform go 2 pieces out of the box).

All this apart from the fact that the neighboring (spare) machine should be ready to accept the load, i.e. have a reservation (in the case of 2 cars) 50%, 3 - 30% and so on. All this apart from CAL (although I did not look at the CAL topic in Azure). And all this per 50 virtualoks per host.

Addon:

We also need to compare with Amazon as a whole, and at the same time compare the value of money. In the sense that we buy a server right away (and get CAPEX) or on credit (at a terrible interest in the Russian Federation), licenses can come right away (see above about credit), or maybe by installments for 3 years (if not 1 server of course ), etc. CAPEX / OPEX - the topic seems to be simple, but no .

That is, you need to drive a million + server licenses into CAPEX, which, given 10% profitability, will give another 100,000 a year in lost profits, and if you take it on credit, then everything is very bad, whoever took it for business knows how much.

The conclusion

should be considered individually in each case, taking into account the costs of licensing, depreciation, reservation of load (unused capacity) and the cost of servicing money (money price / lost profit / lending). Electricity, air conditioning, the cost of using (renting) the area, of course, can also be calculated, as well as the reserved capacity and depreciation of the UPS.

T7 Pricing for virtual disks
Let's try counting for disk shelf / space in Azure. Let's start with the introductory ones: we don’t have IOPS, we just need about 9-10 TB. Let's take the simplest shelf — the HPE MSA 1040, with 10G, or rather, the HPE MSA 1040 2-port 10G iSCSI Dual Controller SFF Storage (E7W04A)

Configuration:

1 MSA 1040/1050 SFF 2U (WxDxH: 48.26 x 49.5 x 8.9 cm )
2 HPE MSA 1040/1050 1GbE iSCSI 2-port controller controller
(here I made a mistake first, I needed E7W04A, but I considered E7W02A - 1GbE iSCSI)
11 1.8TB 2.5 '' SFF hard drive (SFF) SAS 10K 12G 512e Hot Plug DP for MSA2040 / 1040

We will collect from this R10 and reserve 1 disk on Hot spare. We get 10 working disks, R10 = 1.8 * 5 = 9 Tb. For such volumes, collecting R5 is no longer a very good solution (a long rebuild, the risk of releasing another 1 disk during a rebuild), so you can also assemble a brake R6 (10-2 = 8) = 14.4 Tb. For further conversion of the price per terabyte, the coefficient will be 14.4 / 9 = 1.6.

Price (itelon) - Price: ~ $ 12,807.00 ~ 738,067.41 (price mistakenly taken from E7W02A - 1GbE iSCSI, instead of E7W04A)

Configurator on stss for the same configuration (E7W04A, 11 disks) - writes that the price will be 994,375.00 rubles . You can take a million. But maybe HPE will give us a discount.

The cost of the support subscription is not indicated here (see below), so we will change the disks ourselves and pray that the controllers and power supplies do not die. And so - we prepare 20-30% per year of the purchase price, for 5 years - 100% (doubled, yeah).

According to personal experience, over 5 years - all disks will change, well, they usually do not live for more than 3-4 years (although 36 GB tell disks saw live, last year, and colleagues say that their disks can live for 5-6 years. Okay, okay - but now I have no statistics on disks purchased in 2012). This means that there will be not 11, but 22 disks, and the price tag will be
Price: ~ $ 20 848.00 ~ 1 201 470.24

But these are expensive disks (SAS 10K), let's arrange crolo:
11 2TB 2.5 '' Hard Drive (SFF) NL -SAS 7.2K 12G 512e Hot Plug DP for MSA2040 / 1040
Price: ~ $ 11 531.00 ~ 664 531.53
22 disks -
Price: ~ $ 18,296.00 ~ 1,054,398.48

(Note: 1.4 million went to STSS for 10G config , including HP Service for 3 years. 9x5 work schedule. Start of repair the next business day at the installation site. (NBD, on- site) . The disks in the calculation are the HDD 2000GB SAS 12G 7200rpm Hot Plug SFF 512e (1 year warranty).

That's the price of 18,500 / 1,100,000 (we rounded up a bit) and will be repelled. For 10 Tb R10 and 16 R6 for 5 years. year (excluding electricity, air conditioning and other OPEX / CAPEX) we get per year - Depreciation - 3700 / 220.000.
(Note. The calculation above was performed for a 1G controller. For 10G it will be 1.400.000 / 5 = 280.000. 27% more expensive).

This, I note, is not the easiest and cheapest storage with 10G interfaces. If you do not rest on SAN, but stay on NAS / iSCSI, that is, say the relatively cheap Qnap 12 Bay NAS / iSCSI IP-SAN, Intel Skylake Core i5 3.6GHz Quad Core, 16GB RAM, 10G-ready (TVS-1282-i5- 16G-US). But then again, part of these storage systems is not 10G-ready (you need to buy two separate network), you need to look at how many disk controllers and power supplies are there.

Price
list Azure azure.microsoft.com/en-us/pricing/details/storage
azure.microsoft.com/en-us/pricing/details/storage/blobs

Let's start with BLOB
Western Europe, LRS, hot level - The first 50 TB per month 1.23 RUB - FOR GB.
Terabytes (* 1024) = 1259.52 per month, 10 TB - 12600 per month, 151200 per year. However, cheaper.

Let's look at unmanaged disks of the Standard class
azure.microsoft.com/en-us/pricing/details/storage/unmanaged-disks

LRS - 2.82 RUB per GB, 28900 per 10 TB per month, 346 thousand per year (versus 280 per calculation above)

Calculator for Western Europe / LRS / file storage - says about 38,400.00 ₽ for 10 Tb per month, plus for 10t to 10t operations (I don’t know how relevant it is) -
Subtotal - 58 087.50 ₽, or almost 700,000 a year.
Three times more expensive than its storage R10, or 5 from R6
DORAGO. Saving on personnel with such a volume of tasks is not observed.
With such a difference in price tag - too.

On the other hand, MS provides 2-3 replicas of storage and guaranteed availability of services, plus migration, if necessary, to another data center, plus efficiency. On a standard storage system you can’t just buy “another shelf and drives for a million” - you need a budget, approvals, etc., plus bring, mount and run. See above about CAPEX / OPEX

Conclusion:

When calculating without taking into account the cost of money and the speed of scaling and migration - Azure goes more expensive than its mini-cloud. As soon as the opportunity arises and the need for flexible volume management and risk management (see 9.5 of the rules) - Azure can be quite comparable in price with the classic solution, which brings the issue of choice into the political field, for example, the need to add an AWS, Azure Stack and let's sayIBM Bluemix

The opinion of colleagues about the disks and the cost, I’ll just give you:

Over 13 system years on the hitach (both have already left the warranty), no more than a quarter of the disks flew out. On a more recent hit HUS110, 1-2 flights (well, I don’t remember how many there were, from 2 to 4) to the shelf in 4 years. That is less than 10%. In the first two years of 3PAR's life, exactly zero out of 72 disks flew out in it.

Of other calculations. Of course, the calculation of TCO in my household, alas, is a napkin (but I'm working on it), but the approximate, by sight, cost of a terabyte is R5-10K (yes, from poverty, from greed, in configurations from 3 + 1 to 5 + 1) is $ 3K for 5 years, i.e. 600 a year or 50 a month.

According to my estimates, this our VSAN gives about the same figure, but already in the R5-SSD variant. Subject to:

- Advanced type licenses
- additional 10 Gbit cards and DA cables to ToR switches
- Replaces a pair of 10G ToR switches
- Mix Use SSD at the cache level

- Deduplication + compression 2: 1, because the license is Advanced and All-flash (according to estimates there are 3 : 1 should come out, but it doesn’t hurt to get laid). If you believe how Veeam deduplicates backups, there 4: 1 can happen.
- R5 3 + 1. It appeared in the recent version of VSAN and only for Advanced and All-flash. Now you have a choice for each machine network R10 (1 + 1), R5 (3 + 1) or R6 (4 + 2), as once in LeftHand. Well, that is, it is a little more complicated, there Failures to Tolerate and all that, but the idea is that.