Import Dictionary in Lingualeo.com

RELAXART February 3, 2016 at 13:16

Import Dictionary in Lingualeo.com

From the sandbox

Background

There are several solutions for importing words into Lingualeo.com :

Applications for browser or phones:
Adding words to the site.

The disadvantages of these methods are that you can enter words only one at a time. We need an implementation that will add a few words at a time.

Should there be an API?

The official api could not be found. BUT! There are extensions for the browser, and this is a great way to find the internal api for the service.
We go into Google Chrome and stomp on the service website. At the bottom of the page, applications and extensions are offered. Choosing for Chrome. The browser installs the extension in the Extensions folder (I don’t specify the full path, it depends on the OS). Inside the directory there will be several folders with hashes in the form of names. Choose the latest by date. Inside you can find the config.js file - it contains all the paths to the project API. We are interested in only three of them:

/ api / login
/ gettranslates
/ addword

What to write on?

Chose python since installed by default on most OS. We take modules that do not require additional installation. We implement a solution for working with api.

service.py

import urllib
import urllib2
import json
from cookielib import CookieJar
class Lingualeo:
    def __init__(self, email, password):
        self.email = email
        self.password = password
        self.cj = CookieJar()
    def auth(self):
        url = "http://api.lingualeo.com/api/login"
        values = {
            "email": self.email,
            "password": self.password
        }
        return self.get_content(url, values)
    def add_word(self, word, tword, context):
        url = "http://api.lingualeo.com/addword"
        values = {
            "word": word,
            "tword": tword,
            "context": context,
        }
        self.get_content(url, values)
    def get_translates(self, word):
        url = "http://api.lingualeo.com/gettranslates?word=" + urllib.quote_plus(word)
        try:
            result = self.get_content(url, {})
            translate = result["translate"][0]
            return {
                "is_exist": translate["is_user"],
                "word": word,
                "tword": translate["value"].encode("utf-8")
            }
        except Exception as e:
            return e.message
    def get_content(self, url, values):
        data = urllib.urlencode(values)
        opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(self.cj))
        req = opener.open(url, data)
        return json.loads(req.read())

The original solution was for text files only. Each word should be on a new line. But in the process of writing code, I decided to add an implementation for Kindle, as periodically it is also necessary to take words from him.

handler.py

class Word:
    text = '';
    context = '';
    def __init__(self, text):
        self.text = text
class Base(object):
    data = []
    def __init__(self, source):
        self.source = source
    def get(self):
        return self.data
    def read(self):
        raise NotImplementedError('Not implemented yet')
class Kindle(Base):
    def read(self):
        conn = sqlite3.connect(self.source)
        sql = 'select word, usage from words LEFT JOIN LOOKUPS ON words.id = LOOKUPS.word_key where words.lang="en" GROUP BY word ORDER BY word;'
        for row in conn.execute(sql):
            if isinstance(row[0], unicode):
                word = Word(row[0])
                if isinstance(row[1], unicode):
                    word.context = row[1]
                self.data.append(word)
        conn.close()
class Text(Base):
    def read(self):
        f = open(self.source)
        for word in f.readlines():
            self.data.append(Word(word))
        f.close()

And the implementation of the script itself for export / import of words:

export.py

import handler
import config
import service
import sys
email = config.auth.get('email')
password = config.auth.get('password')
export_type = sys.argv[1]
if export_type == 'text':
    word_handler = handler.Text(config.sources.get('text'))
elif export_type == 'kindle':
    word_handler = handler.Kindle(config.sources.get('kindle'))
else:
    raise Exception('unsupported type')
word_handler.read()
lingualeo = service.Lingualeo(email, password)
lingualeo.auth()
for word_dto in word_handler.get():
    word = word_dto.text.lower().encode('utf-8')
    translate = lingualeo.get_translates(word)
    if translate["is_exist"]:
        print "Already exists: " + word.strip()
    else:
        context = word_dto.context.encode('utf-8')
        lingualeo.add_word(word, translate["tword"], context)
        print "Add word: " + word.strip()

Launch and installation

Download the code from github. Create the config.py file from config.py.dist. We write the path to the file with words. In the case of kindle to sqlite base inside kindle.

python export.py text #Для текстовых файлов
python export.py kindle #Для kindle

Source code

GitHub: lingualeo.export .

Tags:

Import Dictionary in Lingualeo.com