Caching in Android, Telegram for groups, improved callback, multicast, showlist and other innovations



    A fairly large update that fixes errors in the android client, improving the security of obtaining a user ID, sending one push to a group of users in the channel, as well as an API for working with user lists.
    Today in the rubric:
    1. We use PushAll and Telegram when developing in a team
    2. Icons gobbled up my traffic - help!
    3. Why do I see only a piece of them?
    4. Emoji killers!
    5. How to combine a thousand iterations of sending notifications into one (multicast)
    6. We steal personal data. We get a list of users subscribed to the channel.
    7. Subscribe to push notifications from your friends (vulnerability)
    8. We are not waiting for all notifications to be sent, we are doing everything in the background.


    I will describe in the style of the problem and its solutions

    Notification icons cropped


    I was approached with a problem that on Android 4. * Icons are cropped. More precisely, on older versions there was no automatic fitting of icons.

    It was solved simply:

    if(icon != null){
        Resources res = getApplicationContext().getResources();
        int height = (int) res.getDimension(android.R.dimen.notification_large_icon_height);
        int width = (int) res.getDimension(android.R.dimen.notification_large_icon_width);
        int realwidth=icon.getWidth();
        int realheight=icon.getHeight();
        if(realwidth>realheight){
            height=height * realheight / realwidth;
        }else{
            width=width * realwidth / realheight;
        }
        icon=Bitmap.createScaledBitmap(icon, width, height, true);
    }
    


    Above is the algorithm for adaptively changing the icon for the device. At that, filtering is used, therefore, in parallel with this, we also improved the quality of displaying icons on newer versions of Android.

    The application downloaded 100 MB of traffic per month in the form of icons


    I already noticed this problem, and I thought that it was very serious. You can also include the following problems in it:
    1. Long delivery of notification when downloading images via unstable connection
    2. Lack of icon with poor connection
    3. Lack of icon in power saving mode.

    I want to tell you more about the latter. The fact is that Android, in new versions, in power saving mode, disables the background transfer of all non-system applications. This leads to the fact that the notification arrives, but we cannot load the icon.

    Solution: create a caching algorithm.

    At first, I struggled for a long time for the algorithm to use standard HTTP cache facilities.

    connection.setUseCaches(true);
    


    I even found a solution to override the caching algorithm, as There was no standard one, and it seemed to work, with one exception - if there is no network, the icon will still not load. That is, Android blocked exactly all HTTP requests, even if there is a cache check.

    Then I wrote my own method, which checks for the presence of the md5 file name in the cache folder. And if he is, then he uses it. The network is not used at all. This is a fairly tough cache, that is, to change the image, you must always change its name, otherwise the old one will load.

    public Bitmap getBitmapFromURL(String strURL) {
        Bitmap myBitmap;
        BitmapFactory.Options options = new BitmapFactory.Options();
        options.inPreferredConfig = Bitmap.Config.RGB_565;
        File file = new File(cacheDir, md5(strURL));
        FileInputStream finput=null;
        if (file.exists()) {
            try {
                finput = new FileInputStream(file);
                myBitmap=BitmapFactory.decodeStream(finput, null, options);
                finput.close();
                return myBitmap;
            } catch (IOException e){
                return BitmapFactory.decodeResource(
                        getResources(), R.drawable.gcm_cloud, options);
            }
        }
        try {
            URL url = new URL(strURL);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            //connection.setUseCaches(true);
            connection.setDoInput(true);
            connection.connect();
            if (connection.getContentLength() < 524288){
                InputStream input = connection.getInputStream();
                FileOutputStream output = new FileOutputStream(file);
                int bufferSize = 1024;
                byte[] buffer = new byte[bufferSize];
                int len = 0;
                while ((len = input.read(buffer)) != -1) {
                    output.write(buffer, 0, len);
                }
                output.close();
                finput = new FileInputStream(file);
                myBitmap = BitmapFactory.decodeStream(finput, null, options);
                finput.close();
            }else{
                myBitmap=BitmapFactory.decodeResource(
                    getResources(), R.drawable.gcm_cloud, options);
            }
            return myBitmap;
        } catch (IOException e) {
            //e.printStackTrace();
            return null;
        }
    }
    public static final String md5(final String s) {
        final String MD5 = "MD5";
        try {
            // Create MD5 Hash
            MessageDigest digest = java.security.MessageDigest
                    .getInstance(MD5);
            digest.update(s.getBytes());
            byte messageDigest[] = digest.digest();
            // Create Hex String
            StringBuilder hexString = new StringBuilder();
            for (byte aMessageDigest : messageDigest) {
                String h = Integer.toHexString(0xFF & aMessageDigest);
                while (h.length() < 2)
                    h = "0" + h;
                hexString.append(h);
            }
            return hexString.toString();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        }
        return "";
    }
    


    Above, I described the final algorithm for generating a bitmap from a link with a cache check and MD5 function that I found on the network. I think the code is simple and there is no need for comments.

    The algorithm also uses a picture size limit of 512 kilobytes.



    Bottom line: when subscribing to 20 channels, the user does not load more than 1-5 megabytes per day. Notifications also come much faster, and icons are displayed even if there is no network, of course, provided that the channel does not use new icons every time.

    Anyone could embed a callback link on their site and “link” their devices to other people's accounts


    It is strange that we did not think of this before, but it is. We have added new protection methods.

    The new address is: ADDRESS? Pushalluserid = ID & time = UNIXTIME & sign = SIGNATURE.

    That is, in the GET, the “pushalluserid” parameter with the user ID will be passed to you, as well as the parameters for verification.
    To verify the signature, use md5 ($ key. $ Pushalluserid. $ Time. $ IpAddress).

    Where ipAddress:
    $ ipAddress = $ _SERVER ['REMOTE_ADDR'];
    Where $ key is the key of your channel. Where $ time is UNIXTIME.

    You can set the required verification level yourself, for example, after checking the time and IP, consider the key valid for 1 minute. Also make sure that the time on your server is accurate. If you wish, you can consider the link valid in general for 5 seconds.
    Backward compatibility also remains to work. That is, after the introduction of this security method, your current applications continue to work. But you need to implement protection on your side as soon as possible.

    As a result, we have verification and security implemented without additional requests. The main thing is to admit your mistakes and correct them in time.

    Why is it the whole day through RSS that the same post is being pushed ?!



    In general, something like this turned out even ominously a bit.
    What happened? Yes, here's what,



    Tinkoff inserted emoji into the post. The same thing happened with one more channel. Since utf8_unicode_ci is on the server, and it is three-byte, mysql chopped posts by emoji. The problem has 2 solutions, it is to use utf8mb4 or cut emoji.
    The first takes time, applied the second crutch:

    function removeEmoji($text) {
        $clean_text = "";
        // Match Emoticons
        $regexEmoticons = '/[\x{1F600}-\x{1F64F}]/u';
        $clean_text = preg_replace($regexEmoticons, '', $text);
        // Match Miscellaneous Symbols and Pictographs
        $regexSymbols = '/[\x{1F300}-\x{1F5FF}]/u';
        $clean_text = preg_replace($regexSymbols, '', $clean_text);
        // Match Transport And Map Symbols
        $regexTransport = '/[\x{1F680}-\x{1F6FF}]/u';
        $clean_text = preg_replace($regexTransport, '', $clean_text);
        // Match Miscellaneous Symbols
        $regexMisc = '/[\x{2600}-\x{26FF}]/u';
        $clean_text = preg_replace($regexMisc, '', $clean_text);
        // Match Dingbats
        $regexDingbats = '/[\x{2700}-\x{27BF}]/u';
        $clean_text = preg_replace($regexDingbats, '', $clean_text);
        return $clean_text;
    }
    


    It was not possible to send notifications to a group of users, only one at a time


    This problem has been brewing for a long time. At the moment, one of the channels uses unicast to send to several people, its volumes are not so large, but the new partner will need to send more than 10 thousand notifications at a time, so multicast has become a priority.

    Features are:
    1. Filter - taken into account (unicast does not, which allows you to notify the user through the filter)
    2. You can select 1,000 users in an array at a time. (of course, such volumes must be sent via POST)
    3. You don’t need to do additional notification and forward all the data, send lid (push ID in the log) and new messages will be assigned to it, moreover, you must do this, otherwise it will be difficult for you to track statistics and work with the notification in the future


    Also below I will describe at the same time the ability to take the task to the "background" mode. That is, you do not need to wait until the system sends all the data to Google and works with the telegram (which takes a very long time). You just give a request and get the push ID with which you continue to work.

    $ch = curl_init()
    //цикл?
    if($i==0)
    $data=array(
        "type" => "multicast",
        "id" => "1",
        "key" => "7db783637a21ed5a8af94513239dada7",
        "text" => "Тестовое сообщение",
        "title" => "Заголовок",
        "background" => 1
      );  //первая итерация
    else
    $data=array(
        "type" => "multicast",
        "id" => "1",
        "key" => "7db783637a21ed5a8af94513239dada7",
        "lid" => $lid,
        "background" => 1
    ); //остальные итерации
    $data['uids']=json_encode(array_slice($alluids, $i*1000, 1000, true)); //отрезаем 1000 записей для передачи на сервер.
    curl_setopt_array($ch, array(
    CURLOPT_URL => "https://pushall.ru/api.php",
    CURLOPT_POSTFIELDS => $data,
      CURLOPT_SAFE_UPLOAD => true,
      CURLOPT_RETURNTRANSFER => true
    ));
    $return=curl_exec($ch); //получить ответ или ошибку
    $lid=json_decode($return,true)['lid'];
    //закрываем цикл
    curl_close($ch);
    


    The code above allows.
    1. 1. Complete tasks quickly by sending them to the background
    2. 2. Send several thousand notifications with the same ID in a loop
    3. 3. Use Keep-Alive for faster communication.


    Background - available today.
    Multicast is still in development and is being tested, it will be available in a week.

    How to get user lists?


    Earlier, I described the showlist method. Now it has a parameter - subtype = users
    When using this parameter, you get a list of all users, and if you add uid = user ID, you can select a separate one.

    You get all the data as when viewing the list through the site.


    Telegrams for groups, business and development


    After adding the telegram integration function, many wrote to me with requests to work with groups.

    Why is this needed?
    • It turns out there are online stores that discuss orders in general chat and coordinate actions. It’s convenient for them when a notification is sent from their store to the general chat.
    • There are development groups who want to receive general chat, commits, bugs, updates or pushes from their systems and then discuss this in general chat.


    You only need to add the @PushAllGrBot bot to the general chat, write / getid and register the received ID in the channel settings.
    At the same time, your subscribers will also be able to receive notifications on their devices in parallel with the telegram.

    Results, what awaits us next?


    We fixed many mistakes, made many new features and optimizations. In a week we are planning to release the native iOS version. We are actively integrating with new sites and services, and we are also waiting for many volunteers in our catalog who can integrate the system on their own.

    New composite filters, rating and feedback systems for channels, and much more will soon be available to users. The applications will have the ability to customize the sound of notifications and vibration.
    Ask questions about new features, offer new features that you need.
    Have a good day!

    Also popular now: