Twitter is following you, anonymous

Often you hear that large corporations like Google, Facebook or LG are watching every bunch of unfortunate users . And having examined some of their software, I have to admit that yes - they follow.

Yesterday I investigated the latest (at the time of this writing) official Twitter iOS client version 5.13.1 . Among other things, it was interesting for me to look at the traffic between the client and the server. Since the Twitter client uses certificate pinning , its traffic is quite difficult to intercept using Fiddler2 and similar programs. Therefore, I wrote a definecommand for GDB that clings to a breakpoint in [NSURLConnection start]and dumps into the console the fields that interest me NSURLConnectionimmediately before calling thisstart. Such a simple trick in general allows you to see what exactly the program sends to the server before this “what exactly” is encrypted for further transmission via HTTPs.

And you know what I saw? The Twitter client, in addition to its usual requests (user registration, authorization, tweet, etc.) often ... no, not so ... very often sent requests to a mysterious URL to the POST server api.twitter.com/1.1/jot/client_event. These requests went to the server for almost any reason. I started the client - the request went. I clicked the “Sign in” button (I have not logged in yet, I just clicked the button) - the request has gone. The login screen got out, the “Username” field got the focus - the request went. I turned off the client - the request went ... I certainly was surprised and climbed to understand.

Typical request forapi.twitter.com/1.1/jot/client_eventit looked like this (for obvious reasons, I missed some fields): Having



looked at a couple of dozens of such queries, I roughly understood the general structure:

URL: https://api.twitter.com/1.1/jot/client_event
Method: POST
Headers:
  X-Twitter-API-Version: 5
  Authorization: OAuth ... la-la-la, regular OAuth authentication signed with a secret key (without quotes)
                 "GgDYlkSvaPxGxC4X8liwpUoqKwwr3lCADbz8A7ADU & fgFuHBbtKUk1ykYzwEKaYeNg28H0XEvztfVQCEu8GA"
  X-Twitter-Client: Twitter-iPhone
  X-Twitter-Client-Version: 5.13.1
  Geolocation: my coordinates, damn it (but if you turn off the Location Service - this field disappears from the requests)
  Accept-Language: en
  X-Twitter-Client-DeviceID: ID of my device
  X-Client-UUID: unique UUID (generated once at the first start of the client, does not change further) 
  User-Agent: Twitter-iPhone / 5.13.1 here is the iOS version (Apple; iPhone3,1 ;;;;;; 1)
Body (of course URL-encoded):
  lang = en & log = ... here is an interesting JSON (see below) ... & send_error_codes = 1

Of course, the most interesting thing here was in JSON in the request body. In all requests, it was an array of elements of the following form:

[
  ...
  {
   "event_name": event name
   "ts": UNIX timestamp,
   "format_version": 2,
   "_category _": "client_event",
   "client_version": "Twitter-iPhone \ 5.13.1 here is the version of iOS (Apple; iPhone3,1 ;;;;; 1)"
  }
  ...
]

For example, when I clicked the “Sign in” button (I have not logged in yet, I just clicked on the button), a request with something like this JSON was sent to the server:

[
  {
   "event_name": "iphone: login ::: impression",
   "ts": 1385678390003,
   "format_version": 2,
   "_category _": "client_event",
   "client_version": "Twitter-iPhone \ 5.13.1 iOS \ 6.1.3 (Apple; iPhone3,1 ;;;;; 1)"
  }
]

I watched other requests (there were a lot, huh), and it seems that the field event_name, ts, format_version, _category_and client_versionare mandatory. Each JSON element had them, in each request. But besides them, additional fields came across. For example, when the Username field received focus on the login screen, the following JSON was left in the request:

[
  {
   "_category _": "client_event",
   "event_name": "iphone: ddg: digits_iphone_signup_1337 :: experiment",
   "client_version": "Twitter-iPhone \ 5.13.1 iOS \ 6.1.3 (Apple; iPhone3,1 ;;;;; 1)",
   "ts": 1385678389354,
   "experiment_key": "digits_iphone_signup_1337",
   "format_version": 2,
   "version": 12,
   "bucket": "uncheck_follows"
   }
]

Probably the field experiment_key, versionand bucketprovide some additional information about what and how I am doing. Well, what would the guys on Twitter understand. What kind of information is it and what these magical meanings are 12and uncheck_follows, I don’t know yet, but maybe I’m going to study this issue more carefully.

That's all. At the end of the article, something smart should be said, but at four in the morning nothing comes to mind. Unless I really like how Twitter, through its application, monitors almost every of my movements and knows in detail what, when and (given the title Geolocation) where I do with their program. Up to which button I clicked and when. This is probably some kind of necessary and useful experiment to collect my user experience, not for nothing that in some queries there is a fieldexperiment_key.

And I don’t like that this application with its endless "spy" requests eats up my precious (literally) GPRS traffic. Well, at least I'm not in roaming, otherwise with these Twitter experiments you can stay without pants for a couple of days ... well, I blurted out something: “I don’t like it, I don’t like it ...” I’ll probably go to bed.

Good Friday everyone :)

PS Sorry if the title seemed yellow to someone. Thank you all for the opinions in the comments, it turns out an interesting discussion. At the moment, the points of view of the commentators are divided: someone does not like such a meticulous and detailed tracking from the official Twitter client. Others say that they say it's okay, sites for example have long been collecting statistics on user actions, and applications too. Mmmm ... I don’t even know. In the case of sites, the user has at least a theoretical opportunity to compete for their privacy (using the same Ghosteryeg). And applications from the App Store have closed sources, HTTPs, some certificate pinning do not have a “do not follow me” checkmark. It turns out that there are almost no chances to prevent the collection of statistics on the actions of an ordinary user. Although it may really be okay, it’s just that I’m already old and behind the times, and I would have to get used to the brave new world. But still I hope not.