Social network on Android over the weekend - Part II (server)

  • Tutorial

Summary of the first part


In response to the ongoing boom of mobile social applications, my friends and I decided to get together in a mini-hackathon and write another social network on Android in order to outline a circle of common issues and offer a skeleton from which everyone can make something new and original. In the first part, we looked at the client interface, network requests, the friends graph, and image processing.
In this article, we will briefly talk about uploading photos to the cloud storage, delivering push notifications and queues of asynchronous tasks on the server.

Content


Introduction
Register
Contact Sync
Upload Photos
Push Notifications
Asynchronous Task Queues
Conclusion

Introduction


The server side of the application performs the functions of registering users, synchronizing the contact list and managing the friends list, uploading and post-processing photos, managing and issuing comments / likes, sending push notifications. Consider these issues in more detail.

registration


When registering, the user is required to specify a name and phone number, and also optionally select an avatar. Because user identification is carried out by the contact book, then an important aspect is the verification of the specified phone, so we added SMS verification . You can choose your service for sending SMS from this article .

Contact Sync


To build a graph of friends on the server, the contact lists of users are kept and compared with the phone numbers of users specified during registration. All contact lists are stored in hashed form. Phone numbers should be brought back to normal form, which uses the libphonenumber library from Google.

Code 1. Normalization example in libphonenumber
String strRawPhone = "8-903-1234567";
PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
try {
  PhoneNumber swissNumberProto = phoneUtil.parse(swissNumberStr, "RU");
} catch (NumberParseException e) {
  System.err.println("NumberParseException was thrown: " + e.toString());
}
System.out.println(phoneUtil.format(swissNumberProto, PhoneNumberFormat.E164));
//Результат: +79031234567


It is worth noting one nuance - the country code is determined in the ISO-3166 format relative to the user's device, i.e. even if the phone numbers of other countries are in my contact book, then when normalizing these numbers, you must use the country code of the “home” SIM card of my device - RU.

Phones are mapped in one of two cases:
  • When registering a new user, his phone is compared with existing contact lists
  • Also, each time the application is launched, the contact list is re-sent to the server to identify new contacts.

For the described scenario, two tables are created on the server database - one for the contact list itself and one for the list of confirmed friends (the friends graph itself). Such a scheme allows you to modify existing contacts without violating the previously formed edges of the friends graph.
Code 2. Database schema - contacts and friends
db / schema.rb
  create_table "contacts", force: true do |t|
    t.string   "public_id"
    t.string   "contact_key"
    t.datetime "created_at"
    t.datetime "updated_at"
  end
  create_table "friends", force: true do |t|
    t.string   "public_id_src"
    t.string   "public_id_dest"
    t.integer  "status"
    t.datetime "created_at"
    t.datetime "updated_at"
    t.string   "contact_key"
  end



Upload photos


We chose two options as a photo storage: a free AWS S3 free tier account as the primary and our own server as a backup (for example, in case of exceeding the request limit in the free S3 account).
Figure 1. Uploading Images to AWS S3
image

Before downloading, the client requests a temporary public link from the server with write permissions, downloads directly from this link to S3, and then reports to the server about the successful download. To work with AWS S3 we used aws-sdk gem . Before work, you need to create an account in AWS Web Services (at the time of development it was possible to create a free test account for 5GB and 20,000 requests) and get a key pair ACCESS_KEY / SECRET_ACCESS_KEY
Code 3. Request a public link in aws-sdk
lib / s3.rb
require 'aws-sdk'
class S3Storage
...
def self.get_presigned_url(key)
    s3 = Aws::S3::Resource.new(
      :access_key_id => APP_CONFIG['s3_access_key_id'],
      :secret_access_key => APP_CONFIG['s3_secret_access_key'],
      :region => APP_CONFIG['s3_region'])
    obj = s3.bucket(APP_CONFIG['s3_bucket']).object(APP_CONFIG['s3_prefix'] + "/" + key)
    obj.presigned_url(:put, acl: 'public-read', expires_in: 3600)
end
...


After the client reported the successful upload of the photo, our server downloads it in asynchronous mode, makes two thumbnails using rmagick gem and saves it back to the cloud. Thumbnails are used to facilitate traffic on a mobile device when viewing images in the stream.
Code 4. An example of creating thumbnails in rmagick
lib / uploader.rb
require 'aws-sdk'
require 'open-uri'
require 's3'
class Uploader
  @queue = :upload
  def self.perform(img_id)
...
  image = Image.where(image_id: img_id).first
  image_original = Magick::Image.from_blob(open(image.url_original).read).first
  image_medium = image_original.resize_to_fit(Image::MEDIUM_WIDTH, medium_height)
  image_medium.write( filepath_medium ){self.quality=100}
...
  end
end


After the uploaded photos are processed, a push notification is sent to all subscribers.

Push notifications


When uploading new photos or adding comments, push notifications are sent to the user's subscribers in real time. The most popular and fairly simple way to deliver push notifications in Android is GCM - Google Cloud Messaging . Before using the service, you need to register your project in the developer console , get the API key and Project Number . The API key is used to authorize the application server for requests to GCM, it is added to the header of HTTP requests.

On the client side, the unique identifier of the notification recipient is PushID, which is obtained by accessing the GCM server directly through the GoogleCloudMessaging SDK from the Android device, and you must specify the previously obtained ProjectID . The received PushID is sent to our application server and subsequently used for delivery of notifications.
Fig 2. The sequence of registration of the new PushID
image

Code 5. Example of registering a new PushID (client)
class MainActivityHandler
    public void registerPushID() {
        AsyncTask task = new AsyncTask() {
            @Override
            protected Object doInBackground(Object[] params) {
                String strPushID = "";
                try {
                    if (gcm == null) {
                        gcm = GoogleCloudMessaging.getInstance(activity);
                    }
                    strPushID = gcm.register(Constants.PUSH_SENDER_ID);
                    Log.d(LOG_TAG, "Received push id = " + strPushID);
                } catch (IOException ex) {
                    Log.d(LOG_TAG, "Error: " + ex.getMessage());
                }
                return strPushID;
            }
            @Override
            protected void onPostExecute(Object res) {
                final String strPushID = res != null ? (String) res : "";
                if (!strPushID.isEmpty()) {
                    UserProfile profile = new UserProfile();
                    profile.pushid = strPushID;
                    Log.d(LOG_TAG, "Sending pushId " + strPushID + " to server");
                    ServerInterface.updateProfileRequest(activity, profile,
                            new Response.Listener() {
                                @Override
                                public void onResponse(String response) {
                                    Photobook.getPreferences().strPushRegID = strPushID;
                                    Photobook.getPreferences().savePreferences();
                                    Log.d(LOG_TAG, "Delivered pushId to server");
                                }
                            }, null);
                }
            }
        };
        task.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
    }


The connection between the application server and GCM can be done in two ways - through XMPP and HTTP . The first option is asynchronous (allows you to send multiple messages without waiting for confirmation from the previous ones), and also supports two-way upstream / downstream communication . HTTP only supports synchronous downstream requests, but allows multiple notifications to be sent at once.
Figure 3. Push notification delivery sequence
image


Code 6. Example of sending push notifications (HTTP)
lib / push.rb
require 'net/http'
class PushSender
  def self.perform(id, event, msg)
    user = User.where(id: id).first
    http = Net::HTTP.new('android.googleapis.com', 80)
    request = Net::HTTP::Post.new('/gcm/send',
      {'Content-Type' => 'application/json',
       'Authorization' => 'key=' + APP_CONFIG['google_api_key']})
    data = {:registration_ids => [user.pushid], :data => {:event => event, :msg => msg}}
    request.body = data.to_json
    response = http.request(request)
  end
end



Asynchronous task queues


To speed up interaction with the client, some tasks on the server are performed in the background. In particular, it is sending push notifications, as well as scaling images. For such tasks, we chose resque gem . The list of queuing solutions and a brief description can be found here . We chose resque for its ease of installation and configuration, support for persistence using the redis database, and the presence of a minimalist web interface. After starting the rails server, you must separately run the resque queue processor in the following way:
QUEUE=* rake environment resque:work

After that, queuing new tasks is carried out in the following way (For example, sending push notifications)
Code 7. Example of queuing a task
app / controllers / image_controller.rb
#Crop and save uploaded file
def create
  img_id = request.headers['imageid']
  ...
  Resque.enqueue(Uploader, img_id)
  ...
end

lib / uploader.rb
require 'aws-sdk'
require 'open-uri'
require 's3'
class Uploader
  @queue = :upload
  def self.perform(img_id)
...
    author = User.where(id: image.author_id).first
    if (author != nil)
      followers = Friend.where(public_id_dest: author.id.to_s, status: Friend::STATUS_FRIEND)
      followers.each do |follower|
        data = {:image_id => img_id, :author => JSON.parse(author.profile), :image => image}
        PushSender.perform(follower.public_id_src, PushSender::EVENT_NEW_IMAGE, data)
      end
    end
  end
end



Conclusion


Work on the application was carried out without the goal of generating commercial profit and solely for its own interest, as well as to strengthen teamwork skills. The format of our meetings was similar to a weekend hackathon, every day we tried to implement a specific application module. We will be glad if you have comments or suggestions for improving the project, and also plan to continue similar hackathons, so if you are a beginner backend / web / Android developer and you have an interest in participating in this format of offline meetings in Moscow or remotely , then write to us through any communication channels.
This is us
image

PS I would like to note that writing a new social network is not a difficult task and, if you wish, is available even to a novice Android developer. Instead of your own backend, you can use ready-made solutions from Google Apps Engine or Heroku . The development of the concept, operational support, and network scaling is much more difficult due to the growing number of users. Perhaps we will address these issues in future articles.

github
Android client
Server on ruby ​​on rails

Good luck to everyone and have a good week!

Also popular now: