Intersystems DeepSee. Simple and fast data visualization
- Tutorial
As the BI system, our organization selected DeepSee, which is part of the InterSystems Cache database management system package. In part, this choice was predetermined by the fact that we use this DBMS to store data, so using integrated BI is the best solution.
DeepSee is an OLAP cubes that "look" at the selected table of the selected area, such cubes can be filtered by all fields of the original table, which allows you to view data from anywhere. For example, one of the tasks was to display information about the births; DeepSee allowed creating tables based on cubes showing how many children were born in the last year, dividing them into groups by sex, weight, body length, mothers age and place of birth. Also, for each parameter, you can set the conditions for DrillDown - failing in the parameter, so instead of the year we can see how many were born in a particular month, day, hour.
Sounds good, right? The way it is!
However, as always in any barrel of honey there is a fly in the ointment, in the case of DeepSee, this is the design in which the data is presented:
In this form, it’s not worth anyone to show analytics, they will either take us for terrible conservatives or for people who have no taste.
Fortunately, I found salvation on github, a completely open product called DeepSeeWeb (DSW) - a site created using jQuery and AngularJS. And if pure DeepSee works inside the DBMS, i.e. to get to it, you need to go to the Caché management portal, then DeepSeeWeb works as a simple client to the server using REST requests, which are described in the MDX2JSON package installed in Caché, but nothing prevents us from writing our own requests, for this the main thing is to master “local” language - Caché ObjectScript (COS). Each request is:
"Declaration" in the default handler class (class file with code in COS), for example REST.cls. The following XDATA section is created in this class:
And in it we can determine the reaction of the server to client requests, each request is declared as follows:
where Url is the actual name of the request, http: // <our server> / <application name> / Url
in Url you can also pass arguments arg1, arg2 ... then the request will take the form
http: // <our server> / <application name> / Url / arg1 / arg2
then the request method (POST, GET, DELETE and etc.) is indicated, and Call ends - the function that will actually process our request.
Many should be interested in what kind of <application name> we embed in our request. In order to have access to REST functions, it is necessary to create a Web application in the DBMS.
Here we must give it the very name by which the server will understand what exactly we want from it and define a handler class for REST requests.
But, perhaps, enough theory, more details on this issue can be returned next time. And considering that there is very little information about DeepSee in the p-space, the next time it will certainly be.
Let's continue about DeepSeeWeb.
Let's look at it step by step.
Main menu:
The site uploaded all the dashboards previously created in DeepSee (old design) and brought them to us in the Metro style, preserving the original structure, which was in the folders, there. The wrench icon at the bottom of the screen does not slip away from attentive eyes, this is a transition to the setting mode for the appearance of the menu tiles.
You can change the color, icon, title and color of the text. However, the most interesting is the ability to output to the data tile one of the graphs that is on the selected dashboard.
After getting acquainted with the menu, let's move on and go inside the dashboard.
Each widget of the old design has its own analogue in the new one, the connection is due to the type of widget, when requesting a list of all widgets, the client receives their name, type and request to receive data, then compares the types existing on the client with the received ones and displays them. Widgets themselves are “drawn” using the Highcharts library , this is a very powerful tool for visualizing data, for development it provides dozens of various chart'ov - graphs, linear, column, circular, time lines, as well as many methods for customizing them, this Libraries are just a huge API that allows you to do anything you like with charts.
And now, as soon as you get acquainted with Highcharts, you will certainly want to see some charts on your dashboards, and ... You will know the disappointment, as I once knew it, by default DeepSeeWeb has at its disposal only a part of the charts from the large library ! But we are working with JavaScript, which means that nothing is impossible for us! And the truth is, the developers expected someone to get the urge to add something new. And they realized the possibility of adding, even made a beautiful form for this:
This is a simple JSON editor in which you need to declare an array of widgets and in it each new object will describe a separate widget:
url : path to the js file describing the widget
This file is an angular factory ( factory)
class : factory name in the specified file
name : widget name that we describe
type : widget type, chart indicates that the new widget will display a graph from the Highcharts library, there are also types “pivot”, “text”, “map” - table, text and map.
Now, back to the name property, where to actually get the name of the new widget, because in order to describe something, you need to create it. Creating new widgets turned out to be the most difficult task for me, but it was very simple, just read README carefully.
So, in order to create a new widget, you need to create a new class in Cache, I did this studio, which is included in the Cache kit.
ALL! Create a new file, write in it:
And done, a new widget is created. To display it on a dashboard, you need to put it there, go to the old DeepSee management portal and add a widget, and if earlier we selected from what was already,
Now we select the portlet we created from the category portlet:
Here is a list of all the widgets I created.
A custom widget that is not described in the settings looks exactly like this, it reports that it does not know such a widget and has no idea what to do with it.
Now we know how to add our own widgets and let's try. First, add a new widget from the Highcharts library.
For example, one that displays how many visits on our site were in the context of each hour.
According to Highcharts, these charts are called spiderWeb . Create the SpiderWeb.js file and write there:
Now let's create a class in Cache, and in order not to be confused, we will also call it SpiderWeb.
User here is the name of the folder in which the file will lie, the folder can be any. Next, add a new widget to the settings, as we recall, we already have one widget from the example above, so now the settings will take the following form:
Now, adding this widget in the old portal, we will see it on the new one exactly in the form we wanted.
The article turned out to be terribly long, I hope the information is useful, and next time it will be possible to start adding a widget that has nothing to do with the Highcharts library, it will be much more interesting, and you will see what possibilities DeepSee can open for you when used correctly.
And the openness of DeepSeeWeb allows the community to participate in the development of the project, which will allow it to absorb the best ideas. Although I am in 3 months of work, I made only one pull-request, here I am such a greed!
To foment interest, here's a screen shot of what I got as the first version of my work on the DeepSee BI system.
Of the new interesting features - a filter by date (right corner) and a text widget.
PS Sorry for the lack of data, we have not yet decided what to show and what to hide, so we hide everything.
→ Link to DeepSeeWeb
DeepSee is an OLAP cubes that "look" at the selected table of the selected area, such cubes can be filtered by all fields of the original table, which allows you to view data from anywhere. For example, one of the tasks was to display information about the births; DeepSee allowed creating tables based on cubes showing how many children were born in the last year, dividing them into groups by sex, weight, body length, mothers age and place of birth. Also, for each parameter, you can set the conditions for DrillDown - failing in the parameter, so instead of the year we can see how many were born in a particular month, day, hour.
Sounds good, right? The way it is!
However, as always in any barrel of honey there is a fly in the ointment, in the case of DeepSee, this is the design in which the data is presented:
In this form, it’s not worth anyone to show analytics, they will either take us for terrible conservatives or for people who have no taste.
Fortunately, I found salvation on github, a completely open product called DeepSeeWeb (DSW) - a site created using jQuery and AngularJS. And if pure DeepSee works inside the DBMS, i.e. to get to it, you need to go to the Caché management portal, then DeepSeeWeb works as a simple client to the server using REST requests, which are described in the MDX2JSON package installed in Caché, but nothing prevents us from writing our own requests, for this the main thing is to master “local” language - Caché ObjectScript (COS). Each request is:
"Declaration" in the default handler class (class file with code in COS), for example REST.cls. The following XDATA section is created in this class:
XData UrlMap
{
And in it we can determine the reaction of the server to client requests, each request is declared as follows:
where Url is the actual name of the request, http: // <our server> / <application name> / Url
in Url you can also pass arguments arg1, arg2 ... then the request will take the form
http: // <our server> / <application name> / Url / arg1 / arg2
then the request method (POST, GET, DELETE and etc.) is indicated, and Call ends - the function that will actually process our request.
Many should be interested in what kind of <application name> we embed in our request. In order to have access to REST functions, it is necessary to create a Web application in the DBMS.
Here we must give it the very name by which the server will understand what exactly we want from it and define a handler class for REST requests.
But, perhaps, enough theory, more details on this issue can be returned next time. And considering that there is very little information about DeepSee in the p-space, the next time it will certainly be.
Let's continue about DeepSeeWeb.
Let's look at it step by step.
Main menu:
The site uploaded all the dashboards previously created in DeepSee (old design) and brought them to us in the Metro style, preserving the original structure, which was in the folders, there. The wrench icon at the bottom of the screen does not slip away from attentive eyes, this is a transition to the setting mode for the appearance of the menu tiles.
You can change the color, icon, title and color of the text. However, the most interesting is the ability to output to the data tile one of the graphs that is on the selected dashboard.
After getting acquainted with the menu, let's move on and go inside the dashboard.
Each widget of the old design has its own analogue in the new one, the connection is due to the type of widget, when requesting a list of all widgets, the client receives their name, type and request to receive data, then compares the types existing on the client with the received ones and displays them. Widgets themselves are “drawn” using the Highcharts library , this is a very powerful tool for visualizing data, for development it provides dozens of various chart'ov - graphs, linear, column, circular, time lines, as well as many methods for customizing them, this Libraries are just a huge API that allows you to do anything you like with charts.
And now, as soon as you get acquainted with Highcharts, you will certainly want to see some charts on your dashboards, and ... You will know the disappointment, as I once knew it, by default DeepSeeWeb has at its disposal only a part of the charts from the large library ! But we are working with JavaScript, which means that nothing is impossible for us! And the truth is, the developers expected someone to get the urge to add something new. And they realized the possibility of adding, even made a beautiful form for this:
This is a simple JSON editor in which you need to declare an array of widgets and in it each new object will describe a separate widget:
url : path to the js file describing the widget
This file is an angular factory ( factory)
function PieChart(Utils) {
function CustomWidget($scope) {
……
}
}
angular.module('widgets')
.factory('PieChart', ['Utils', PieChartFact]);
class : factory name in the specified file
name : widget name that we describe
type : widget type, chart indicates that the new widget will display a graph from the Highcharts library, there are also types “pivot”, “text”, “map” - table, text and map.
Now, back to the name property, where to actually get the name of the new widget, because in order to describe something, you need to create it. Creating new widgets turned out to be the most difficult task for me, but it was very simple, just read README carefully.
So, in order to create a new widget, you need to create a new class in Cache, I did this studio, which is included in the Cache kit.
ALL! Create a new file, write in it:
Class <Название виджета> Extends %DeepSee.Component.Portlet.abstractPortlet
{
}
And done, a new widget is created. To display it on a dashboard, you need to put it there, go to the old DeepSee management portal and add a widget, and if earlier we selected from what was already,
Now we select the portlet we created from the category portlet:
Here is a list of all the widgets I created.
A custom widget that is not described in the settings looks exactly like this, it reports that it does not know such a widget and has no idea what to do with it.
Now we know how to add our own widgets and let's try. First, add a new widget from the Highcharts library.
For example, one that displays how many visits on our site were in the context of each hour.
According to Highcharts, these charts are called spiderWeb . Create the SpiderWeb.js file and write there:
function SpiderWebFact(BaseChart, Utils) {
function SpiderWebChart($scope) {
//Говорим виджету что бы он тянул scope из базового для всех
//highcharts-виджетов класса
BaseChart.apply(this, [$scope]);
var _this = this;
//указываем тип графика может быть bar, column, line, area, pie
this.setType('line',true);
//получаем данные с сервера
this.requestData();
//Описываем настройки виджета именно для SpiderWeb
var ex = {
options: {
plotOptions: {
series: {
//не пропускать на графике места в которых график равен 0, что бы не было //разрывов
connectNulls: true
}
},
//описываем что выводить в всплывающем окне при наведении на маркер графика
tooltip: {
shared: true,
useHTML: true,
formatter: function () {
var t = this;
/* jshint ignore:end */
var a= ""+t.x+":00";
var all = 0;
for(var i =0;i
' + '' + val + " Всего: "+all.toString().replace(/\B(?=(\d{3})+(?!\d))/g, " ")+"
";
return a;
}
}
},
yAxis: {
//основная директива SpiderWeb, именно она делает его таким, каков он есть
gridLineInterpolation: 'polygon',
lineWidth: 0,
min: 0
},
xAxis: {
//указываем что на конец каждого измерения необходимо добавить 00, потому что
//с сервера время приходит в формате 01,02,03 …
labels: {
formatter: function () {
return this.value + ':00';
}
}
}
};
//передаем только что объявленные настройки в список всех настроек виджета
Utils.merge($scope.chartConfig, ex);
}
return SpiderWeb;
}
//объявляем фабрику
angular.module('widgets')
.factory('SpiderWebChart', ['BaseChart', 'Utils', SpiderWebFact]);
Now let's create a class in Cache, and in order not to be confused, we will also call it SpiderWeb.
Class User.SpiderWeb Extends %DeepSee.Component.Portlet.abstractPortlet
{
}
User here is the name of the folder in which the file will lie, the folder can be any. Next, add a new widget to the settings, as we recall, we already have one widget from the example above, so now the settings will take the following form:
{
"widgets": [
{
"url": "src/factories/pieChart.js",
"class": "PieChart",
"name": "user.piewithvalues",
"type": "chart"
},
{
"url": "src/factories/ SpiderWeb.js",
"class": " SpiderWeb Chart",
"name": "user.spiderweb",
"type": "chart"
}
Now, adding this widget in the old portal, we will see it on the new one exactly in the form we wanted.
The article turned out to be terribly long, I hope the information is useful, and next time it will be possible to start adding a widget that has nothing to do with the Highcharts library, it will be much more interesting, and you will see what possibilities DeepSee can open for you when used correctly.
And the openness of DeepSeeWeb allows the community to participate in the development of the project, which will allow it to absorb the best ideas. Although I am in 3 months of work, I made only one pull-request, here I am such a greed!
To foment interest, here's a screen shot of what I got as the first version of my work on the DeepSee BI system.
Of the new interesting features - a filter by date (right corner) and a text widget.
PS Sorry for the lack of data, we have not yet decided what to show and what to hide, so we hide everything.
→ Link to DeepSeeWeb