Online Database, data structures service, dynamic object-relational projection (Dynamic ORM)
This article will focus on creating a working prototype of an online database. And some services are reviewed analogues from Google and Yahoo.
Recently, services for working with online databases have begun to develop more actively. Such services include platforms such as GData (Google Data API) and Yahoo! Query Language (YQL).
The indicated services of online access to the database allow better control and operation of data received from different platform services, and most importantly, these are ready-made services for the use of which it is enough to study the documentation and examples.
Google’s online data access service allows you to:
- request, enter and manage data programmatically to create applications and mashups;
- allows you to view and update the contents of spreadsheets in the form of Google data channels ("GData");
- the client can request a list of user tables, as well as present, edit or delete content in an existing table;
- apply autofiltration and other table functions in the application.
For convenient work with the above services, Google has developed its own SQL-like query language Google Visualization API Query Language.
Query language for retrieving data from Yahoo's information arrays! .. SQL-like query language for working with web services from Yahoo! .. Query results can be obtained in XML or JSON formats.
Here is what Jonathan Trevor, YQL Execute Development Manager says:
“Developers can now call many services at the same time and operate with many data sources, which will make their applications more powerful and diverse. Received from Yahoo! data can be transcoded in real time from one format to another. "
YQL makes it easy to manipulate data using SQL-like syntax. For example, you want to select photos in Paris, France that have a Creative Commons Attribution (4) license:
select * from flickr.photos.info where photo_id in (select id from flickr.photos.search where woe_id in (select woeid from geo.places where text = 'paris, france') and license = 4)
data tables can be created and stored using the service "Open Table". The service is quite interesting, but the description is still very small.
The above services will allow you to access and perform online manipulation of data tables, which is vital for many applications. Having analyzed the above services and a number of other similar services, we have implemented a similar service for working with data tables. However, it has a specific distinguishing feature. Let's take a closer look at our development directly from the inside.
API documentation www.hivext.ru/index.php/ Structures
One of the extremely necessary for flexible and convenient development of applications based on the platform is the service of structures. A distinctive feature of our development lies in the object approach in the implementation of such a service. In order to provide all the benefits of an object-oriented approach to building applications, proven over the years, the service was designed with the maximum “tilt” in the direction of OOP (Object-Oriented Design). Due to this, the structure service allows developers to create their applications in the OOP style, allows you to manipulate and manage the types and objects of developed applications.
The structure service is designed to determine types and create instances of types (objects). The approach to creating your objects is quite simple and familiar. To create a collection, a set of any objects, it is initially necessary to determine the type of future objects. Defining a type means defining the fields that future objects should contain. In turn, defining a type field means specifying the name of the field and the simple base type of the field. There are predefined simple types in the structure service, based on which developers can create their more complex types.
List of simple types that are predefined in the structure service
Based on the above types, developers can create their own more complex types. Let's look at an example. Let's say we need to create an online bookstore. One of the mandatory objects of our application will be a product-book, which we sell. Define a type for our book objects. To determine the type, the DefineType method (appid, session, type, fields) is used. For
our books, we will call the method with the following
DefineType parameters (appid, session, “book”, {price: ”float”, name: ”string”, description: ”string (500) ”, author:” string (100) ”})
Thus, we have defined a book type that has price fields- the price of the book, name - the name of the book, description - the description of the book, author - the author of the book.
Further, we can safely create instances of this type, i.e. objects are books. To do this, use the
CreateObject (appid, session, type, data) method. For
our book type, call the method with the following
CreateObject (appid, session, “book”, {price: 10.5, name: ”Web Application Development, description:” about designing and developing complex one-page application sites ”, author:” Know-it-all I.N. ”})
If the object is successfully created, we will get the identifier of the new object in the response.
Now we have the first object of type book. Next, in the same way, we create the set of objects necessary for our online store.
CreateObject (appid, session, “book”, {price: 8, name: ”Development of desktop applications”, description: ”about designing and developing simple desktop applications”, author: ”Vestka EV”}})
CreateObject ( appid, session, “book”, {price: 12, name: ”Mobile application development”, description: ”about designing and developing applications for mobile phones”, author: “Know-it-all I.N.”})
And so on. Everything is pretty simple!
Next, you need to display formatted lists of books available in our store to users. This is made even easier. To select many objects, two methods of service structures are used.
GetObjects (appid, session, type, from, count)
GetObjectsByCriteria (appid, session, type, criteria, from, count)
The GetObjects method is intended for straight-line sequential selection of objects. It returns lists of objects that are selected according to their sequence of creation. In other words, lists of objects are sorted in ascending order of identifiers (id). The from and count parameters indicate which index to take and how much, these are optional parameters.
Consider an example. We call the method with the following parameters GetObjects (appid, session, “book”)
As a result, we get the following answer.
{response: 0, book: [
{id: 1, price: 10.5, name: "Development of web applications", description: "about designing and developing complex one-page website applications", author: "Know-it-all I.N."},
{id: 2, price: 8, name: ”Development of desktop applications”, description: ”about designing and developing simple desktop applications”, author: ”Vesto EV”},
{id: 3, {price: 12, name: ”Mobile development applications ”, description:” about designing and developing applications for mobile phones ”, author:” Know-it-all I.N.}}
]} The
operation of the GetObjects method is quite simple and transparent.
Consider the operation of the GetObjectsByCriteria method. A distinctive feature of this method relative to the previous method is the ability to set search criteria for objects. This method allows you to flexibly generate lists of displayed objects.
You can create search criteria using two different syntaxes. Either using SQL similar commands or using JSONQL syntax. You do not need to explicitly indicate which syntax for generating the criteria you are using. The method automatically determines which syntax the developer uses. Let's look at an example of the same query in SQL and JSONQL syntaxes. For example, we get a list of books whose price is less than 11.
SQL search criteria: price <11
GetObjectsByCriteria (appid, session, “book”, “price <11”)
JSONQL search criteria: {price: {'<': 11}}
GetObjectsByCriteria (appid, session, “book”, “{price: {'<': 11}}”)
We get a list of books whose authors are Know-It-All or Know-It-All.
SQL search criteria: author author like '% Know-it-all%' or author like '%
Know-nothing %' GetObjectsByCriteria (appid, session, “book”, “author like '% Know-it-all%' or author like '%
Know-nothing %'”) JSONQL criterion search: {author: {like: ['% Know-it-all%', '% Know-it-all%']}}
GetObjectsByCriteria (appid, session, “book”, {author: {like: ['% Know-all%', '% Know-nothing% ']}})
You can use both SQL and JSONQL syntax. Choose what is more convenient for you. If everything is clear and familiar with SQL syntax, then JSONQL is a unique development that was invented when developing the Hivext platform. Let us dwell in more detail on JSONQL. The syntax for generating criteria using JSONQL is very flexible and has several variations that are similar in effect. For a more complete understanding of the syntax of such criteria, consider in detail the principles for the formation of these search criteria. Since the service of structures is implemented on the basis of the Hibernate library, Hibertnate criteria form the basis of the search criteria for objects. Let's look at the implementation using the last example, namely, we get a list of books whose authors are Know-it-All or Know-It-All. Hibertnate uses the following syntax to formulate a criterion
Criterion criterion = Restrictions.or (Restrictions.like (“author”, ”% Know-it-all%”), Restrictions.like (“author”, ”% Know-it-all%”));
Transform the specified expression into JSONQL syntax. The operation of unification according to the logic OR is determined by the array [..., ...]. Thus
[{like: {author: '% Know-it-all%'}}, {like: {author: '% Know-all%'}}]
Everything is pretty simple and transparent. Record practically repeats Hibernate syntax. Let's look at similar possible action options for the formation of the criterion.
{like: [{author: '% Know-it-all%'}, {author: '% Know-it-all%'}]}
Another option
{like: {author: ['% Know-it-all%', '% Know-it-all%']}}
As you can see, the last option is simplified by the number of characters and readable. However, there is another option, which, according to the author, is even more readable. Just swap the condition and field name
{author: [{like: '% Know-it-all%'}, {like: '% Know-all%'}]}
{author: [{like: '% Know-all%'}, {like: ' % Dunno% '}]}
{author: {like: ['% Dunno% ','% Dunno% ']}}
The last option is the most readable, however, all 6 of the last considered options are working and perform the same action. Such freedom in the formation of search criteria gives the formation of the criterion great flexibility. Let's look at the AND operation.
We get a list of books whose authors are Know-it-All and Know-It-All. Hibertnate syntax is as follows
Criterion criterion = Restrictions.and (Restrictions.like (“author”, ”% Know-it-all%”), Restrictions.like (“author”, ”% Know-it-all%”));
In JSONQL, the AND join operation is determined by a sequence of single-level objects {...}, {...}
Thus, we get the JSONQL syntax:
{author: {like: '% Know-it-all%'}}, {author: {like: '% Know-
nothing %'}} {author: {{like: '% Know-it-all%'}, {like: '% Know-all%'}}}
Having such an extensive set of possible JSONQL options, as well as the ability to query using standard SQL, we get a very flexible method for querying objects using criteria.
Also in the structure service, there are other methods for managing types and objects.
All methods of service structures can be found in the documentation: hivext.ru/index.php/ Structures
For convenient development and testing of service methods, for testing query criteria, there is a developer console code.hivext.ru/development/APIConsole/#Data.Base
The console has a list of all existing service methods. You can try the console in real work when creating your types and objects.
One of the main points in the server implementation of the structure service is the dynamic mapping of the created objects into the database tables. When determining types, when changing the structure of types on the server, classes are automatically generated that describe the types defined by the developer. This allows for simple mapping of objects into tables, as well as for interconnecting objects. All principles of the implementation and construction of service structures cannot be described in one article. I’ll add that the service also has caching, transactions, and lazy loading of objects, and much more ...
Service structures will allow you to conveniently and flexibly determine the object-oriented structure of the developed application. The specified service has a complete set of methods for creating, editing, deleting and selecting the necessary types and instances of these types (objects). Also, the structure service, thanks to the ORM used, allows you to abstract from the DBMS and conveniently work with objects. In combination with the access rights service, it allows you to define the OO architecture of the application and define clear access rights for subjects of access to objects.
In the next article, it is planned to consider the access rights service, which is also one of the main and critical services for developing full-fledged applications.
We hope that the habrasociety with understanding will approach the analysis of these services. Constructive criticism regarding the implementation of such services and the syntax of working with services is welcome.
Structural service documentation hivext.ru/index.php/ Structures
Recently, services for working with online databases have begun to develop more actively. Such services include platforms such as GData (Google Data API) and Yahoo! Query Language (YQL).
The indicated services of online access to the database allow better control and operation of data received from different platform services, and most importantly, these are ready-made services for the use of which it is enough to study the documentation and examples.
Google Base API + Google Data Table API
Google’s online data access service allows you to:
- request, enter and manage data programmatically to create applications and mashups;
- allows you to view and update the contents of spreadsheets in the form of Google data channels ("GData");
- the client can request a list of user tables, as well as present, edit or delete content in an existing table;
- apply autofiltration and other table functions in the application.
For convenient work with the above services, Google has developed its own SQL-like query language Google Visualization API Query Language.
Yahoo! Query language
Query language for retrieving data from Yahoo's information arrays! .. SQL-like query language for working with web services from Yahoo! .. Query results can be obtained in XML or JSON formats.
Here is what Jonathan Trevor, YQL Execute Development Manager says:
“Developers can now call many services at the same time and operate with many data sources, which will make their applications more powerful and diverse. Received from Yahoo! data can be transcoded in real time from one format to another. "
YQL makes it easy to manipulate data using SQL-like syntax. For example, you want to select photos in Paris, France that have a Creative Commons Attribution (4) license:
select * from flickr.photos.info where photo_id in (select id from flickr.photos.search where woe_id in (select woeid from geo.places where text = 'paris, france') and license = 4)
data tables can be created and stored using the service "Open Table". The service is quite interesting, but the description is still very small.
The above services will allow you to access and perform online manipulation of data tables, which is vital for many applications. Having analyzed the above services and a number of other similar services, we have implemented a similar service for working with data tables. However, it has a specific distinguishing feature. Let's take a closer look at our development directly from the inside.
Hivext Structure Service
API documentation www.hivext.ru/index.php/ Structures
One of the extremely necessary for flexible and convenient development of applications based on the platform is the service of structures. A distinctive feature of our development lies in the object approach in the implementation of such a service. In order to provide all the benefits of an object-oriented approach to building applications, proven over the years, the service was designed with the maximum “tilt” in the direction of OOP (Object-Oriented Design). Due to this, the structure service allows developers to create their applications in the OOP style, allows you to manipulate and manage the types and objects of developed applications.
General characteristics of the service
- Service and data are available on any platform and device where HTTP requests are supported (you can also directly access the database from Javascript, Actionscript and other client programming languages);
- Object-relational projection (ORM) of created objects in database tables;
- Service and data are available on any platform and device where HTTP requests are supported;
- Unlimited database volumes for applications;
- Work with the database at the level of types and objects, flexible data management through service methods.
The structure service is designed to determine types and create instances of types (objects). The approach to creating your objects is quite simple and familiar. To create a collection, a set of any objects, it is initially necessary to determine the type of future objects. Defining a type means defining the fields that future objects should contain. In turn, defining a type field means specifying the name of the field and the simple base type of the field. There are predefined simple types in the structure service, based on which developers can create their more complex types.
List of simple types that are predefined in the structure service
- string / String - length 255 characters (default type)
- string (N) / String (N) - N any integer (from 1 to 10,240 characters)
- text / Text is a text data type that can contain a maximum of 65,000 characters.
- mtext / Mtext - text data type, can contain a maximum of 16000000 characters.
- short / short
- int / int / integer / integer
- long / long
- bool / bool / boolean / boolean
- float / float
- double / double
- char / char
- byte / byte
Based on the above types, developers can create their own more complex types. Let's look at an example. Let's say we need to create an online bookstore. One of the mandatory objects of our application will be a product-book, which we sell. Define a type for our book objects. To determine the type, the DefineType method (appid, session, type, fields) is used. For
our books, we will call the method with the following
DefineType parameters (appid, session, “book”, {price: ”float”, name: ”string”, description: ”string (500) ”, author:” string (100) ”})
Thus, we have defined a book type that has price fields- the price of the book, name - the name of the book, description - the description of the book, author - the author of the book.
Further, we can safely create instances of this type, i.e. objects are books. To do this, use the
CreateObject (appid, session, type, data) method. For
our book type, call the method with the following
CreateObject (appid, session, “book”, {price: 10.5, name: ”Web Application Development, description:” about designing and developing complex one-page application sites ”, author:” Know-it-all I.N. ”})
If the object is successfully created, we will get the identifier of the new object in the response.
Now we have the first object of type book. Next, in the same way, we create the set of objects necessary for our online store.
CreateObject (appid, session, “book”, {price: 8, name: ”Development of desktop applications”, description: ”about designing and developing simple desktop applications”, author: ”Vestka EV”}})
CreateObject ( appid, session, “book”, {price: 12, name: ”Mobile application development”, description: ”about designing and developing applications for mobile phones”, author: “Know-it-all I.N.”})
And so on. Everything is pretty simple!
Next, you need to display formatted lists of books available in our store to users. This is made even easier. To select many objects, two methods of service structures are used.
GetObjects (appid, session, type, from, count)
GetObjectsByCriteria (appid, session, type, criteria, from, count)
The GetObjects method is intended for straight-line sequential selection of objects. It returns lists of objects that are selected according to their sequence of creation. In other words, lists of objects are sorted in ascending order of identifiers (id). The from and count parameters indicate which index to take and how much, these are optional parameters.
Consider an example. We call the method with the following parameters GetObjects (appid, session, “book”)
As a result, we get the following answer.
{response: 0, book: [
{id: 1, price: 10.5, name: "Development of web applications", description: "about designing and developing complex one-page website applications", author: "Know-it-all I.N."},
{id: 2, price: 8, name: ”Development of desktop applications”, description: ”about designing and developing simple desktop applications”, author: ”Vesto EV”},
{id: 3, {price: 12, name: ”Mobile development applications ”, description:” about designing and developing applications for mobile phones ”, author:” Know-it-all I.N.}}
]} The
operation of the GetObjects method is quite simple and transparent.
Consider the operation of the GetObjectsByCriteria method. A distinctive feature of this method relative to the previous method is the ability to set search criteria for objects. This method allows you to flexibly generate lists of displayed objects.
You can create search criteria using two different syntaxes. Either using SQL similar commands or using JSONQL syntax. You do not need to explicitly indicate which syntax for generating the criteria you are using. The method automatically determines which syntax the developer uses. Let's look at an example of the same query in SQL and JSONQL syntaxes. For example, we get a list of books whose price is less than 11.
SQL search criteria: price <11
GetObjectsByCriteria (appid, session, “book”, “price <11”)
JSONQL search criteria: {price: {'<': 11}}
GetObjectsByCriteria (appid, session, “book”, “{price: {'<': 11}}”)
We get a list of books whose authors are Know-It-All or Know-It-All.
SQL search criteria: author author like '% Know-it-all%' or author like '%
Know-nothing %' GetObjectsByCriteria (appid, session, “book”, “author like '% Know-it-all%' or author like '%
Know-nothing %'”) JSONQL criterion search: {author: {like: ['% Know-it-all%', '% Know-it-all%']}}
GetObjectsByCriteria (appid, session, “book”, {author: {like: ['% Know-all%', '% Know-nothing% ']}})
You can use both SQL and JSONQL syntax. Choose what is more convenient for you. If everything is clear and familiar with SQL syntax, then JSONQL is a unique development that was invented when developing the Hivext platform. Let us dwell in more detail on JSONQL. The syntax for generating criteria using JSONQL is very flexible and has several variations that are similar in effect. For a more complete understanding of the syntax of such criteria, consider in detail the principles for the formation of these search criteria. Since the service of structures is implemented on the basis of the Hibernate library, Hibertnate criteria form the basis of the search criteria for objects. Let's look at the implementation using the last example, namely, we get a list of books whose authors are Know-it-All or Know-It-All. Hibertnate uses the following syntax to formulate a criterion
Criterion criterion = Restrictions.or (Restrictions.like (“author”, ”% Know-it-all%”), Restrictions.like (“author”, ”% Know-it-all%”));
Transform the specified expression into JSONQL syntax. The operation of unification according to the logic OR is determined by the array [..., ...]. Thus
[{like: {author: '% Know-it-all%'}}, {like: {author: '% Know-all%'}}]
Everything is pretty simple and transparent. Record practically repeats Hibernate syntax. Let's look at similar possible action options for the formation of the criterion.
{like: [{author: '% Know-it-all%'}, {author: '% Know-it-all%'}]}
Another option
{like: {author: ['% Know-it-all%', '% Know-it-all%']}}
As you can see, the last option is simplified by the number of characters and readable. However, there is another option, which, according to the author, is even more readable. Just swap the condition and field name
{author: [{like: '% Know-it-all%'}, {like: '% Know-all%'}]}
{author: [{like: '% Know-all%'}, {like: ' % Dunno% '}]}
{author: {like: ['% Dunno% ','% Dunno% ']}}
The last option is the most readable, however, all 6 of the last considered options are working and perform the same action. Such freedom in the formation of search criteria gives the formation of the criterion great flexibility. Let's look at the AND operation.
We get a list of books whose authors are Know-it-All and Know-It-All. Hibertnate syntax is as follows
Criterion criterion = Restrictions.and (Restrictions.like (“author”, ”% Know-it-all%”), Restrictions.like (“author”, ”% Know-it-all%”));
In JSONQL, the AND join operation is determined by a sequence of single-level objects {...}, {...}
Thus, we get the JSONQL syntax:
{author: {like: '% Know-it-all%'}}, {author: {like: '% Know-
nothing %'}} {author: {{like: '% Know-it-all%'}, {like: '% Know-all%'}}}
Having such an extensive set of possible JSONQL options, as well as the ability to query using standard SQL, we get a very flexible method for querying objects using criteria.
Also in the structure service, there are other methods for managing types and objects.
All methods of service structures can be found in the documentation: hivext.ru/index.php/ Structures
Console
For convenient development and testing of service methods, for testing query criteria, there is a developer console code.hivext.ru/development/APIConsole/#Data.Base
The console has a list of all existing service methods. You can try the console in real work when creating your types and objects.
One of the main points in the server implementation of the structure service is the dynamic mapping of the created objects into the database tables. When determining types, when changing the structure of types on the server, classes are automatically generated that describe the types defined by the developer. This allows for simple mapping of objects into tables, as well as for interconnecting objects. All principles of the implementation and construction of service structures cannot be described in one article. I’ll add that the service also has caching, transactions, and lazy loading of objects, and much more ...
Service structures will allow you to conveniently and flexibly determine the object-oriented structure of the developed application. The specified service has a complete set of methods for creating, editing, deleting and selecting the necessary types and instances of these types (objects). Also, the structure service, thanks to the ORM used, allows you to abstract from the DBMS and conveniently work with objects. In combination with the access rights service, it allows you to define the OO architecture of the application and define clear access rights for subjects of access to objects.
In the next article, it is planned to consider the access rights service, which is also one of the main and critical services for developing full-fledged applications.
We hope that the habrasociety with understanding will approach the analysis of these services. Constructive criticism regarding the implementation of such services and the syntax of working with services is welcome.
Structural service documentation hivext.ru/index.php/ Structures