Analysis of articles Habrahabr and Geektimes

At first there was an idea to compile a frequency dictionary of the words Habrahabr and Geektimes, but then I found the beautiful: Detailed analysis of Habrahabr using the Wolfram Language (Mathematica) (I recommend that you click on the link before viewing this article), saw the difference in the results and decided to do the same for articles with Habrahabr and geektimes. The review did not include articles with podcasts and custom design (habrahabr.ru/article). The analysis was 170,000 articles. All images are clickable. Wolfram Notebook code on github . Dataset on Yandex.Disk . UPD: regarding the rating calculation - I, too, was mistaken in the calculation. Here in the discussion OsipovRoman writes that the differences are small.

Data processing results

Hub analysis

Distribution of the number of hubs in which the article is posted:

The largest hubs by the number of articles:

If we consider only unique articles (relating to only one hub):

I didn’t do the graph of connections, because I did not collect the list of hubs separately.

Number of Articles by Time

In the caption to the pictures only Habrahabr is mentioned, but Geektimes is also implied.

Number of posts per month:

In a year:

In the hub "Mathematics":

Hub "Cosmonautics":

Hub "Habrahabr":

The number of images (videos) used in posts versus time

And in separate hubs:

Clouds of keywords and individual hubs

Here the WordCloud function, whatever value you pass to the WordOrientation attribute (Random, {- Pi / 4, Pi / 4}), painted everything by default:

Hub "Mathematics":

Hub "Programming":

Java hub:

Hub "Open source":

Hub "Machine Learning":

Sites referred to in articles

We remove Habrahabr as a source of links:

In the hub "Mathematics" (without Habrahabr as a source of links):

Hub "Development for iOS":

Hub ".NET":

Codes that result in articles

Without SomeCode (if no programming language is specified):

In the hub "Algorithms":

In the hub "Programming":

In the hub "Configure Linux":

In the hub "Machine Learning":

Word Frequency

In the hub "Development for iOS"

In the hub "Development for Android":

Frequency of using the names of operating systems in the hub "Open source":

And on Habrahabr / Geek magazine:

Rating and number of views of posts, as well as the probability of reaching their specific values

The average post rating on Habrahabr / Geektimes is 25.6067, and the average number of views is 13487.2.

Expectation: {25.6067, 13487.2}
Standard deviation: {35.9361, 28783.9} The

probability that the post will gain a certain rating:

The likelihood that the post will gain a certain number of views:

The dependence of the rating and number of post views on the time of publication

The dependence of the rating of the post on its volume

The average post size on Habrahabr / Geektimes is 5199 characters.

The probability that a post with a volume not exceeding a specified number of characters will gain a rating of at least a given:

Speaking of word frequencies. Before using Wolfram on a Jupyter Notebook using pymorphy2 libraries, nltk built word clouds by year, but for fewer articles. She took 50 of the most common words in the article (excluding stop words), and then combined the dictionaries for all articles for a certain year. Clouds built in Tagul. KAPV is a word cloud for 2006. For 2016:

Posts with the maximum number

Images: " An overview of email clients for Android, or how I chose a mailer "
Comments: " How to distribute invites on Google+ "
Rating: " Making a private monitor from the old LCD monitor "
Number of tags: " Information and technological tools for the practical survival of social communities in a disconnected environment Internet in 2014 "
Views:" Hack Wi-Fi in ... 3 seconds " By
video count:" DUMP-2016: video of all reports in one post. Free. Without SMS "
Number of links:" Pseudoscience and crooks. Fake scientific journals "
Text: "Create a clone Bird Flappy - Zombie Bird "