June 2, 2019 at 15:04The history of the assembly of the "village supercomputer" of spare parts from eBay, Aliexpress and a computer store. Part 3
Good day, Khabrovites! I will continue my story of assembling a "supercomputer in the village."
Link to the 1st part of the story
Link to the 2nd part of the story The
third part I will begin by expressing sincere gratitude to my friends who supported me in difficult times, motivated, helped with money by sponsoring this rather expensive business for a long time and even helped with the purchase of components because boundaries in cases where I could not buy them directly myself. For example, if a company selling server parts in the USA or Canada simply did not send to Russia. Without their long and regular help, my successes would have been much more modest.
So, thanks to their requests, I decided and started an account on Youtube, bought an old Lumia 640 smartphone that I use exclusively as a video camera and started shooting informative videos, both about assembling the “village supercomputer” and about other aspects and projects of my village life.
Playlist "Village Supercomputer":
Interested spoilers can familiarize themselves, although of course it is better to do this while reading my story, or even after it.
The second part of my story was interrupted by the fact that I connected a disk array to the Tesla K20M, GT 610 and M.2 NVE SSD system. By the way, what else is this Dell board good for - it has a built-in “disk shelf” and only 6 devices and RAID is not “the most sophisticated in the world”, but unlike its more professional external counterparts, it misses the TRIM command on the SSD. What is also important if you intensively use non-professional server SSDs.
By the way, there is also one interesting and important point about this board. The chipset heatsinks are low with small fins. This works well when the board is in the original rack, where powerful turbines blow it along. But when using the board, you must separately remove the plastic sticker from the one near the radiator expansion slots, and the far one should be replaced with any suitable radiator from the chipset of the old motherboard with large fins, because the chip below it heats up most on the board.
Having removed the video card from the system, I began to assemble the frame for my server, in the test version everything was on electrical tape, matchboxes and other plastic supports, but for the full use of 24/7/365 this option did not seem acceptable to me. It was necessary to make a normal frame from an aluminum corner. The aluminum corners from Leroy Merlin were sent to me, which were sent to me by a friend from the Moscow Region; in their nearby town they simply were not sold anywhere at all!
In addition to the corners, the design used screws for the countersunk head M5 and nuts, screws M3 and nuts, small furniture corners, aluminum rivets for 5 mm holes, a riveter, a hacksaw for metal, a screwdriver, a 5.0 mm drill for metal, a file, a Phillips screwdriver, cable ties and hands growing not from the ass.
Corners were used to attach the board to the frame and some other elements. This of course added a few heights to the whole system, because the board was raised high enough above the bottom plane of the frame, but I decided that this was permissible for me. I didn’t fight for every gram of weight, a millimeter of height, after all, it’s not a BCM of an airplane where the standard is “15 G on 3 axes, impacts up to 1000 G and vibration”.
The board is installed, the risers are screwed, the adapter with the SSD M.2 is screwed.
The board, SSD, risers and Tesla are installed in their places. DC-DC has not yet been screwed into place and it hangs on wires off-screen. This is server version 1.0, still on one Tesla K20M.
Here DC-DC is already mounted on the frame, there it is there a small scarf on the side of the motherboard under the “tails” of power.
And this system is already assembled, a top view. Above Tesla one more time from the corners where SSD steam is screwed side by side, above them is an HDD basket, and on top of it on the closing frame there is a 850 W PSU Thermaltek modular. The PSU is fashionable, gaming, with RGB backlight that I turned off so that it would not blink like a Christmas tree. The only powerful modular PSU at that time in stores in the nearby city.
Side view of version 1.0 server.
View of the server from the "front end". I made connectors and rivers for disks on one side, as in server systems, so that for all manipulations it was not necessary to twist the whole system back and forth. On the “cut-out bar", a screw with two USB 2.0 is screwed that I connected instead of the card reader, and the adapter board on M.2 is screwed to its lower part.
Here it is shown how DC-DC and the board are fixed, the same corners about which I spoke.
View from the other side, how the GPGPU-riser, which is EdgeSlot, is fixed.
The highest angle riser with additional power for the GPGPU that I bought through Shopotam from America.
The machine was assembled, the OS was installed, the driver was configured CUDA Toolkit ...
Here's a short video about her.
In this form, the system with one Tesla K20M 5 GB worked for half a year, while my friend-astronomer considered his tasks. Then he went on vacation and suddenly on eBau there were server Tesla K20X 6 GB for 6000 r / b, would be a sale from a data center in England. And we decided to assemble the second version of the “supercomputer” already on the 3x Tesla K20X.
Tesla was bought, the second motherboard was bought exactly the same, they just decided to save on delivery and chose eVau delivery. Which drove her to SPAIN and there she betrayed to some completely left type. A dispute was opened on e-Wow, I was supported by a seller from the USA and the money was returned, and already the third payment went to me with the usual expensive but reliable USPS. Other spare parts also arrived and here is a video about the beginning of the assembly of the "village supercomputer" 2.0.
Video about spare parts for this same "car".
Board launch and some features.
Then he began to assemble the framework of the second version of the server.
Tesla K20X arrived, the first video.
An informative video about Tesla K20X, about the design of the card and its cooling system, and a bummer with a water block from the GTX 780 Ti.
The continuation of the video about Tesla K20X, scanned her board on a scanner, if anyone suddenly needs it.
Front side with GPU chip.
Backside.
As you can see, the Tesla K20, although similar in general terms to the GTX 780 GTX 780 ti GTX TITAN on the GK110 Kepler GPU, is nevertheless not compatible with the board and the cooling system. If I have a Quadro K5200 K6000 GK110 Kepler then I will also compare its board with the Tesla K20 board, but so far I don’t have the above Quadro.
And here is the continuation of server 2.0 assembly
Again 1U coolers with snails and other things that are required for a server with more power than the first. By the way, I had to disassemble the first server to assemble the second, while my friend did not have an urgent need to consider it.
A bit of cable management ...
And the second Tesla is set in its place.
And here I was struck by an insulting bummer. It turned out that the system does not pull 3 pieces of Tesla K20. When starting the BIOS, this is such an error and that’s all, the third Tesla does not work in any way. Even the BIOS update to version 2.8.1 did not help, after which the board from Dell DCS 6220 turned into Dell C6220 2.8.1 I turned on and off various options in BIOS, even tried to tape part of the contacts on Tesla with tape to make them 8x - nothing helped. I had to put up and stop at the configuration of 2 Tesla K20X + NVE SSD. By the way, in version 2.0 of the server all SATA drives live in one Chinese basket with 6 bays. Now there is a pair of Samsung 860 EVO 500 Gb + 4 terabyte Seigate. I bought Samsung on Ali for 3600 apiece. OEM wheels, but I'm happy.
Now "supercomputer 2.0" is fully assembled and ready to go.
In other matters, spare parts bought for the second system arrived and I picked up the first one, here about this video.
And I suggest readers to vote what to do with the first board? What to collect interesting on its basis? Or if someone wants to buy it like Tesla K20M and K20X with or without cooler-snails - I'm ready, write.
Here is such a story, I hope it turned out to be interesting and useful to dear readers.
PS: Those who had the patience to read through to the end - subscribe to my YouTube channel, comment, like / dislike - this will motivate me to publish and shoot new informative videos.
Link to the 1st part of the story
Link to the 2nd part of the story The
third part I will begin by expressing sincere gratitude to my friends who supported me in difficult times, motivated, helped with money by sponsoring this rather expensive business for a long time and even helped with the purchase of components because boundaries in cases where I could not buy them directly myself. For example, if a company selling server parts in the USA or Canada simply did not send to Russia. Without their long and regular help, my successes would have been much more modest.
So, thanks to their requests, I decided and started an account on Youtube, bought an old Lumia 640 smartphone that I use exclusively as a video camera and started shooting informative videos, both about assembling the “village supercomputer” and about other aspects and projects of my village life.
Playlist "Village Supercomputer":
Interested spoilers can familiarize themselves, although of course it is better to do this while reading my story, or even after it.
The second part of my story was interrupted by the fact that I connected a disk array to the Tesla K20M, GT 610 and M.2 NVE SSD system. By the way, what else is this Dell board good for - it has a built-in “disk shelf” and only 6 devices and RAID is not “the most sophisticated in the world”, but unlike its more professional external counterparts, it misses the TRIM command on the SSD. What is also important if you intensively use non-professional server SSDs.
By the way, there is also one interesting and important point about this board. The chipset heatsinks are low with small fins. This works well when the board is in the original rack, where powerful turbines blow it along. But when using the board, you must separately remove the plastic sticker from the one near the radiator expansion slots, and the far one should be replaced with any suitable radiator from the chipset of the old motherboard with large fins, because the chip below it heats up most on the board.
Having removed the video card from the system, I began to assemble the frame for my server, in the test version everything was on electrical tape, matchboxes and other plastic supports, but for the full use of 24/7/365 this option did not seem acceptable to me. It was necessary to make a normal frame from an aluminum corner. The aluminum corners from Leroy Merlin were sent to me, which were sent to me by a friend from the Moscow Region; in their nearby town they simply were not sold anywhere at all!
In addition to the corners, the design used screws for the countersunk head M5 and nuts, screws M3 and nuts, small furniture corners, aluminum rivets for 5 mm holes, a riveter, a hacksaw for metal, a screwdriver, a 5.0 mm drill for metal, a file, a Phillips screwdriver, cable ties and hands growing not from the ass.
Corners were used to attach the board to the frame and some other elements. This of course added a few heights to the whole system, because the board was raised high enough above the bottom plane of the frame, but I decided that this was permissible for me. I didn’t fight for every gram of weight, a millimeter of height, after all, it’s not a BCM of an airplane where the standard is “15 G on 3 axes, impacts up to 1000 G and vibration”.
The board is installed, the risers are screwed, the adapter with the SSD M.2 is screwed.
The board, SSD, risers and Tesla are installed in their places. DC-DC has not yet been screwed into place and it hangs on wires off-screen. This is server version 1.0, still on one Tesla K20M.
Here DC-DC is already mounted on the frame, there it is there a small scarf on the side of the motherboard under the “tails” of power.
And this system is already assembled, a top view. Above Tesla one more time from the corners where SSD steam is screwed side by side, above them is an HDD basket, and on top of it on the closing frame there is a 850 W PSU Thermaltek modular. The PSU is fashionable, gaming, with RGB backlight that I turned off so that it would not blink like a Christmas tree. The only powerful modular PSU at that time in stores in the nearby city.
Side view of version 1.0 server.
View of the server from the "front end". I made connectors and rivers for disks on one side, as in server systems, so that for all manipulations it was not necessary to twist the whole system back and forth. On the “cut-out bar", a screw with two USB 2.0 is screwed that I connected instead of the card reader, and the adapter board on M.2 is screwed to its lower part.
Here it is shown how DC-DC and the board are fixed, the same corners about which I spoke.
View from the other side, how the GPGPU-riser, which is EdgeSlot, is fixed.
The highest angle riser with additional power for the GPGPU that I bought through Shopotam from America.
The machine was assembled, the OS was installed, the driver was configured CUDA Toolkit ...
Here's a short video about her.
In this form, the system with one Tesla K20M 5 GB worked for half a year, while my friend-astronomer considered his tasks. Then he went on vacation and suddenly on eBau there were server Tesla K20X 6 GB for 6000 r / b, would be a sale from a data center in England. And we decided to assemble the second version of the “supercomputer” already on the 3x Tesla K20X.
Tesla was bought, the second motherboard was bought exactly the same, they just decided to save on delivery and chose eVau delivery. Which drove her to SPAIN and there she betrayed to some completely left type. A dispute was opened on e-Wow, I was supported by a seller from the USA and the money was returned, and already the third payment went to me with the usual expensive but reliable USPS. Other spare parts also arrived and here is a video about the beginning of the assembly of the "village supercomputer" 2.0.
Video about spare parts for this same "car".
Board launch and some features.
Then he began to assemble the framework of the second version of the server.
Tesla K20X arrived, the first video.
An informative video about Tesla K20X, about the design of the card and its cooling system, and a bummer with a water block from the GTX 780 Ti.
The continuation of the video about Tesla K20X, scanned her board on a scanner, if anyone suddenly needs it.
Front side with GPU chip.
Backside.
As you can see, the Tesla K20, although similar in general terms to the GTX 780 GTX 780 ti GTX TITAN on the GK110 Kepler GPU, is nevertheless not compatible with the board and the cooling system. If I have a Quadro K5200 K6000 GK110 Kepler then I will also compare its board with the Tesla K20 board, but so far I don’t have the above Quadro.
And here is the continuation of server 2.0 assembly
Again 1U coolers with snails and other things that are required for a server with more power than the first. By the way, I had to disassemble the first server to assemble the second, while my friend did not have an urgent need to consider it.
A bit of cable management ...
And the second Tesla is set in its place.
And here I was struck by an insulting bummer. It turned out that the system does not pull 3 pieces of Tesla K20. When starting the BIOS, this is such an error and that’s all, the third Tesla does not work in any way. Even the BIOS update to version 2.8.1 did not help, after which the board from Dell DCS 6220 turned into Dell C6220 2.8.1 I turned on and off various options in BIOS, even tried to tape part of the contacts on Tesla with tape to make them 8x - nothing helped. I had to put up and stop at the configuration of 2 Tesla K20X + NVE SSD. By the way, in version 2.0 of the server all SATA drives live in one Chinese basket with 6 bays. Now there is a pair of Samsung 860 EVO 500 Gb + 4 terabyte Seigate. I bought Samsung on Ali for 3600 apiece. OEM wheels, but I'm happy.
Now "supercomputer 2.0" is fully assembled and ready to go.
In other matters, spare parts bought for the second system arrived and I picked up the first one, here about this video.
And I suggest readers to vote what to do with the first board? What to collect interesting on its basis? Or if someone wants to buy it like Tesla K20M and K20X with or without cooler-snails - I'm ready, write.
Here is such a story, I hope it turned out to be interesting and useful to dear readers.
PS: Those who had the patience to read through to the end - subscribe to my YouTube channel, comment, like / dislike - this will motivate me to publish and shoot new informative videos.