Habraclans
About a month ago I published an article about habracottas on the habr . A by-product of this article turned out to be a dump of the user’s pages, and I wanted to extract some more information from it. Articles on user analysis , articles , comments , karma regularly appear on the habr, but I did not find a single article in which to analyze the habits. Therefore, I built a graph of habrainweights and looked at some of its characteristics.
Let me remind you that the pages were downloaded in January 2016, so everything that happened after (registration of new users, removal of old users, changes in karma) was not taken into account. Removing all read-only and deactivated users from the list of downloaded users, we get 79870. As far as I know, this number approximately corresponds to the actual number of users (plus or minus a thousand). Further, to get a graph without holes, we had to add 955 read-only users and 382 deactivated (these are the users who invited someone or were invited, but were drunk from the hub or transferred to RO for one reason or another). As a result, we get a graph at 81,207 vertices.
It is worth noting that getting a list of Habr users is not easy. Most were obtained a couple of years ago, when lists of hub subscribers were still available. But now there are no such lists, so the usernames for 2015 and 2016 were extracted from articles, comments, pages of already known users, subscriber lists, user lists from given cities, countries, and I also cut frequently encountered prefixes of user names (of the form Alex * , admin *, Captain *, etc.) and made several thousand requests on the Habr search page. I also added active users on giktaymsa and megamind, so if you are not on my list, you are well encrypted.
So, we have a directed graph with 81207 vertices and 20195 arcs. As you can see, only about 20 thousand users were registered on invites from other users, the rest either registered before invites (more than 40 thousand) or were invited by UFOs. Let's
call habraklanweak connection component of this directed graph. It is worth noting that these components, generally speaking, are not trees, since one person can receive invites several times. Therefore, we have loops: for example, @ tangro invited @ Milla, and @ Milla invited @ tangro; loops: for example, @ aavezel invited himself; vertex, which includes several arcs: the user @ shara was invited 6 times (@ Deeman, @ myagi, @ homm, @ Azya, @ veveve, @ shifttstas). Although all these are more likely exceptions, in general the graph looks like a forest.
In our column there are 61021 habraclan. The size distribution is as follows:
Let's look at the biggest components.
Below are pictures of these 20 graphs. Green circles - users with positive karma, red - with negative, blue - with zero, gray - read-only or deactivated users. The area of the circle is proportional to the karma modulus (if this number is greater than 1). All pictures are links to a large version.
Let's also look at the "heights" of the habraclans. If we throw away the negligible number of graphs with cycles, then dag_longest_path_length (G) gives the following result.
The longest chain is: @ Garyan invited @ Andrey_Rogovsky, who invited @ DmitryGushin, who invited @ Uncle_Sam, who invited @ RootHell, who sent an invite @ alexey_qwe, who invited @ Doom2, who called for @ Odnoklassniki_ru and who finally invited @ DarkDefender.
The analysis coincides with the expectation that most habraclans are small and with a small "height".
Now remember that users have karma. Compound summation gives that there are at least 450323.4 units of positive karma on the habr . (By the way, 10,579 habra users have more than or equal to 10 karma, so theoretically this article can gain 10,578 pluses.)
Let's see which habraclans have the largest karma reserves.
Below are pictures of graphs that have not met before.
Also, for some users, the country is indicated on the page in the "From" field. The top countries by users can be found on the hub itself, but it was interesting for me to look at the invites in which the inviter and the invitee are in different countries. Such invites characterize the “geographical” connectivity of the habrasociety.
At first I wanted to build the so-called. chord diagram , but did not find an easy way to do this in python, so I give the upper left corner of the corresponding matrix. (If someone tells me how to build the diagram, I’d be grateful.) The blue the cell in the picture, the more the logarithm of the number of invites from country 1 to country 2. The
connection between Russia, Ukraine, Belarus, the USA and Germany is noticeable.
Another piece of information that is not related to invites, but is easily extracted from user pages, is the date of registration and the date of the last appearance. The following table shows how many users registered in a given year and how many of them appeared on the hub from January 1, 2015 (otherwise, we think that the user has ceased to be habraactive).
Same thing in chart form.
We see that half of the users registered in 2007 and 2008, as well as many old-timers are active.
That's all. A table with source data and a script for drawing graphs are available on the github . A raw data archive is available upon request.
Let me remind you that the pages were downloaded in January 2016, so everything that happened after (registration of new users, removal of old users, changes in karma) was not taken into account. Removing all read-only and deactivated users from the list of downloaded users, we get 79870. As far as I know, this number approximately corresponds to the actual number of users (plus or minus a thousand). Further, to get a graph without holes, we had to add 955 read-only users and 382 deactivated (these are the users who invited someone or were invited, but were drunk from the hub or transferred to RO for one reason or another). As a result, we get a graph at 81,207 vertices.
It is worth noting that getting a list of Habr users is not easy. Most were obtained a couple of years ago, when lists of hub subscribers were still available. But now there are no such lists, so the usernames for 2015 and 2016 were extracted from articles, comments, pages of already known users, subscriber lists, user lists from given cities, countries, and I also cut frequently encountered prefixes of user names (of the form Alex * , admin *, Captain *, etc.) and made several thousand requests on the Habr search page. I also added active users on giktaymsa and megamind, so if you are not on my list, you are well encrypted.
So, we have a directed graph with 81207 vertices and 20195 arcs. As you can see, only about 20 thousand users were registered on invites from other users, the rest either registered before invites (more than 40 thousand) or were invited by UFOs. Let's
call habraklanweak connection component of this directed graph. It is worth noting that these components, generally speaking, are not trees, since one person can receive invites several times. Therefore, we have loops: for example, @ tangro invited @ Milla, and @ Milla invited @ tangro; loops: for example, @ aavezel invited himself; vertex, which includes several arcs: the user @ shara was invited 6 times (@ Deeman, @ myagi, @ homm, @ Azya, @ veveve, @ shifttstas). Although all these are more likely exceptions, in general the graph looks like a forest.
In our column there are 61021 habraclan. The size distribution is as follows:
| Component Size | Number of components |
|---|---|
| more than 1001 | one |
| 101-1000 | 6 |
| 11–100 | 436 |
| 2-10 | 3110 |
| one | 57468 |
| No. | The size | Root Peaks |
|---|---|---|
| one | 1027 | @ Davekeinz (sent 412 invites - more than anyone else on the hub, also in this component @ Mithgol, which sent 78 invites) |
| 2 | 584 | @ Mudhoney (sent 242 invites) @ valemak |
| 3 | 316 | @ XaocCPS (sent 65 items) |
| four | 272 | @ Alaunquirie (invited @ BarsMonster, who invited 73 users) @ kip |
| five | 189 | @ Deeman @ homm @ DorBer @ myagi @ Azya @ maovrn @ fil9 @ yoihj |
| 6 | 106 | @ Rossomachin |
| 7 | 104 | @ Garyan |
| eight | 97 | @ Kukutz (Yandex.Component) |
| 9 | 90 | @ Eosunknown |
| ten | 85 | @ Cigulev @ tyr |
| eleven | 80 | @ Mdevils |
| 12 | 80 | @ Nuzgul |
| 13 | 77 | @ Ni404 @ tronix286 @ Rembish |
| 14 | 77 | @ Tigger |
| 15 | 76 | @ Gaidar |
| sixteen | 70 | @ Auren |
| 17 | 69 | @ Saltommeister |
| 18 | 68 | @ Kalan |
| nineteen | 68 | @ Alisadenisova |
| 20 | 67 | @ Horsev |
| The length of the longest chain | Number of components |
|---|---|
| 9 | one |
| 7 | 2 |
| 6 | eleven |
| five | 39 |
| four | 125 |
| 3 | 479 |
| 2 | 2888 |
| one | 57468 |
The analysis coincides with the expectation that most habraclans are small and with a small "height".
Now remember that users have karma. Compound summation gives that there are at least 450323.4 units of positive karma on the habr . (By the way, 10,579 habra users have more than or equal to 10 karma, so theoretically this article can gain 10,578 pluses.)
Let's see which habraclans have the largest karma reserves.
| No. | Total karma | Root Peaks |
|---|---|---|
| one | 6184.4 | @ Mudhoney @ valemak |
| 2 | 5333.7 | @ Davekeinz |
| 3 | 4720.8 | @ XaocCPS |
| four | 3587.1 | @ Alaunquirie @ kip (@ BarsMonster here) |
| five | 2464.5 | @ Deeman @ homm @ DorBer @ myagi @ Azya @ maovrn @ fil9 @ yoihj |
| 6 | 2390.1 | @ Horsev (@ PapaBubaDiop and @ Milfgard here) |
| 7 | 1984.9 | @ Cigulev @ tyr (@ Zelenyikot here) |
| eight | 1780.2 | @ Ni404 @ tronix286 @ Rembish |
| 9 | 1606.1 | @ Eosunknown |
| ten | 1526.9 | There is no root, and it all starts with the @ tangro - @ Milla loop |
| eleven | 1319.3 | @ Kit |
| 12 | 1304.1 | @ Ocelot |
| 13 | 1299.5, | @ Auren |
| 14 | 1104.5 | @ Kalan |
| 15 | 1009.1 | @ Rossomachin |
| sixteen | 985.5 | @ Easy_john |
| 17 | 932.3 | @ Assuri |
| 18 | 871.7 | @ Sourcerer |
| nineteen | 845.2 | @ LukaSafonov |
| 20 | 838.6 | @ Mdevils |
Another piece of information that is not related to invites, but is easily extracted from user pages, is the date of registration and the date of the last appearance. The following table shows how many users registered in a given year and how many of them appeared on the hub from January 1, 2015 (otherwise, we think that the user has ceased to be habraactive).
| 2006 | 3091 | 909 |
| 2007 | 19433 | 5511 |
| 2008 | 22031 | 6348 |
| 2009 | 6032 | 3094 |
| 2010 | 6826 | 3345 |
| 2011 | 9341 | 6355 |
| 2012 | 5841 | 4160 |
| 2013 | 4029 | 2819 |
| 2014 | 2684 | 2100 |
| 2015 | 1473 | 1473 |
| Total | 80781 | 36114 |
We see that half of the users registered in 2007 and 2008, as well as many old-timers are active.
That's all. A table with source data and a script for drawing graphs are available on the github . A raw data archive is available upon request.