Online implementation of localStorage

I want to share how Safari's private mode led to the development of a simple key-value store on Node.js with backup, access to data from certain domains and password protection from writing and cleaning the store.



It all started with the fact that they gave me the task of implementing a test order in a web application that is built in via an iframe in one popular resource.

The problem was solved and worked as follows:

  1. an unauthorized user clicks on the store (link "_blank");
  2. test goods are displayed in a new window, and in the iframe we redirect the user to the test user profile and wait for the purchase data to appear in localStorage;
  3. after making a purchase, data about it is stored in localStorage (amount, quantity, store, purchase time and number of bonuses)
  4. in the iframe, when test purchase data appears in localStorage, we display information in the “purchase history” block;

Everything worked in most browsers, and even in IE11, but not in Safari, whose security policy (better known as porno-mode) did not allow access to localStorage data of the same domain inside the iframe and outside (in a new window).

It is necessary to store intermediate data somewhere, to attract the developers backend to create an API for storing data for this task, I did not receive permission, all I had to do was find some kind of online storage, with the ability to create a token for each user.

Searches led me to keyvalue.xyz, it allows you to create a key, write and read data. And so I started for each user who decided to try a test order, create a token and pass it in the url parameters to a new window, then with a successful test order, we write the data to the repository, and already in the iframe periodically requested the data until it appears.


Everything worked, but here a message came from the tester, this time she said that the test order did not work with adblock turned on. So it is, in the adblock console I wrote that the request to the resource is blocked. I turned to the developers of the service with a request to make a mirror, they did not answer, I tried to make a mirror through nginx (proxy_pass), it also did not help, most likely due to the cloudflare filter.


It was not pleasant, it was necessary to get out of the situation.

I decided to write a simple key = storage value like localStorage, with backup, access from a specific domain, password protection from writing, and a convenient library for working with it.

Development


Writing a simple rest api using express with Node.js is not difficult, I chose MongoDB to store data, because there is no rigid structure and you can change the structure of a document with just one line of code in the scheme and of course that mongodb can work with large documents size (100-200GB).

It does not make sense to talk in detail about the development, it is very simple and most of us have already used the express framework.

Let's start with the basic storage requirements:

  1. Token Creation
  2. Token Update
  3. Retrieving value from storage
  4. Retrieving the entire storage
  5. Data recording
  6. Delete item
  7. Storage Cleanup
  8. Get a list of backups
  9. Restore storage from backup

The token scheme is quite simple, as follows:

const TokenSchema = new db.mongoose.Schema({
  token: { type: String, required: [true, "tokenRequired"] },
  connect: { type: String, required: [true, "connectRequired"] },
  refreshToken: { type: String, required: [true, "refreshTokenRequired"] },
  domains: { type: Array, default: [] },
  backup: { type: Boolean, default: false },
  password: { type: String },
})

Extra options:
tokenUsed to access storage
connectProperty for associating storage with a token
refreshTokenToken update in cases
when you need to update a token or a token is
highlighted somewhere, for example in git commit
domainsArray of domains, access to the repository that is allowed.
Origin HTTP header is used for verification
backupIf set to true, then every 2 hours, the
entire storage will be backed up,
that is, several backups are always available during the day,
to which you can roll back
passwordSetting a password for writing and deleting

POST request handler / create
  app.post('/create', async (req, res) => {
    try {
      // Additional storage protection data
      const { domains, backup, password } = req.body
      // New unique uuid token
      const token = uuid.v4()
      // A unique identifier for connecting the token to the storage 
      // as well as using it you can update the token
      const connect = uuid.v1()
      // Default
      const tokenParam = {
        token: token,
        connect: connect,
        refreshToken: connect
      }
      // The list of domains for accessing the repository
      if (domains) {
        // If an array is passed, store it as it is
        if (Array.isArray(domains)) tokenParam.domains = domains
        // If a string is passed, wrap it in an array
        if (typeof domains === 'string') tokenParam.domains = [domains]
        // If passed boolean true, save the host
        if (typeof domains === 'boolean') tokenParam.domains = [req.hostname]
      }
      // Availability of backup
      if (backup) tokenParam.backup = true
      // If a password is sent, we save it
      if (password) tokenParam.password = md5(password)
      // Save to db
      await new TokenModel.Token(tokenParam).save()
      // Sending the token to the client
      res.json({ status: true, data: tokenParam })
    } catch (e) {
      res.status(500).send({ status: false, description: 'Error: There was an error creating the token' })
    }
  })


Examples of requests will be given using the axios library, unlike the curl command, its code is quite concise and understandable.

axios.post('https://storage.hazratgs.com/create', {
  domains: ['example.com', 'google.com'],
  backup: true,
  password: 'qwerty'
})

As a result of execution, we get a response with a token, which can be used to write and read data from the storage:

{
  "status":  true,
  "data":{
    "token": "002cac23-aa8b-4803-a94f-3888020fa0df",
    "refreshToken": "5bf365e0-1fc0-11e8-85d2-3f7a9c4f742e",
    "domains": ["example.com", "google.com"],
    "backup": true,
    "password": "d8578edf8458ce06fbc5bb76a58c5ca4"
  }
}

Writing data to the repository:

axios.post('https://storage.hazratgs.com/002cac23-aa8b-4803-a94f-3888020fa0df/set', {
  name: 'hazratgs',
  age: 25,
  city: 'Derbent'
  skills: ['javascript', 'react+redux', 'nodejs', 'mongodb']
})

Retrieving an item from storage:

axios.get('https://storage.hazratgs.com/002cac23-aa8b-4803-a94f-3888020fa0df/get/name')

As a result, we get:

{
  "status":  true,
  "data": "hazratgs"
}

You can see other examples on the project page on GitHub. .

You can clone the repository and deploy the repository for yourself, detailed instructions in the project repository.

For convenience, a JavaScript library and a Python library are available.
The repository itself is available at storage.hazratgs.com

Library example


And so we already have a repository and a library for working with it, let's install the library:

npm i online-storage

Import into the project:

import onlineStorage from 'online-storage'

I want to note, since we do localStorage implementation online, in the project code, by default we return an object, not a class, in order to work with one data source throughout the project, if you need several objects, you can import the OnlineStorage class itself and create as many objects as you like based on it.

Create a token:

onlineStorage.create()

I must say that onlineStorage is an asynchronous method, like almost all methods of an onlineStorage object, so the best option would be to use async / await syntax.

After creating the token, it is written to the token property and then substituted if necessary, for example, writing data:

await onlineStorage.set({
  name: 'hazratgs',
  age: 25,
  city: 'Derbent'
  skills: ['javascript', 'react+redux', 'nodejs', 'mongodb']
})

reading data:

const order = await onlineStorage.get('name') // hazratgs

property removal:

await onlineStorage.remove('name')

Now we can safely say that we have an online implementation of localStorage and even more, because localStorage works only with strings, and our storage works, can store strings, numbers, objects and logical types.

Conclusion


As a result, we have a data store with backup, access from certain domains and password protection, but I want to say that this store, like localStorage, is not safe and is not intended to store the main data of the application, on which the operability of the project, user data and many more that can harm your application.

Use it only for irrelevant and public data, as in my example, for transferring test purchase data from a window to an iframe.

Problems can arise due to the fact that the token, like all javascript we transmit to the client and it will not be difficult for him to receive the data of the entire storage, is not a problem of our storage specifically, any api-keys transferred to the client become public and although we have some protection in the face of working with certain domains and passwords, all this can most likely be circumvented.

In order to hide the token, of course, you can write a wrapper over api on your server, but this is already so, it’s easier to set up your own database.

Please do not scold much, this is my first publication and contribution to open source.
I will be very grateful in helping to eliminate vulnerabilities, tips, pull requests.

Also popular now: