Route graph for apache camel

In this article I will tell you about how you can build a route graph for applications with Apache Camel, monitor the status of these routes and collect metrics for them.
We use Apache Camel in spring applications and Apache ServiceMix. And if the routes in a separate service is a thing that is understandable and easily observable, then within the data bus, where there are many such routes, everything is not so simple.

<?xml version="1.0" encoding="UTF-8"?><blueprintxmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="
      http://www.osgi.org/xmlns/blueprint/v1.0.0
      http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd"><camelContextxmlns="http://camel.apache.org/schema/blueprint"><route><fromuri="file:camel/input"/><logmessage="Moving ${file:name} to the output directory"/><touri="file:camel/output"/></route></camelContext></blueprint>

Over the years of use, the tire has accumulated more than a hundred different routes of difficulty, and we have come to understand that it is becoming more and more difficult to remember all these connections. Drawing route maps with hands or describing them in the form of tables has ceased to seem a convenient and easily supported solution. But it began to seem that the automatic construction of the route graph would save everyone.

To construct a graph, we need vertices and edges. And of them we are blind to something beautiful!

Route elements

The entry point (it is one) for the route is described by the operator fromwith indication of the endpoint. Those. for

<fromuri="file:camel/input"/>

endpoint will be file:camel/input. He tells us that at the beginning of the route the files will be taken from the directory camel/input.

The exit points of the route (there are many of them, which is why I indicated the plural number) are determined differently - depending on the message exchange pattern, also with an indication of an endpoint. In the example above, this point is described by to. Those. for

<touri="file:camel/output"/>

endpoint will be file:camel/output. He tells us that at the end of the route files will be saved to the directory camel/output.

Endpoints are the vertices we need. The ribs will determine the routes themselves.

Getting route descriptions

Servicemix provides the ability to access various information using JMX and we will use jolokia to access this information via http.

As an example, take this description of the routes.

<?xml version="1.0" encoding="UTF-8"?><beansxmlns="http://www.springframework.org/schema/beans"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="
         http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
         http://camel.apache.org/schema/spring
           http://camel.apache.org/schema/spring/camel-spring.xsd"><camelContextxmlns="http://camel.apache.org/schema/spring"><routeid="the-clock-is-ticking"><fromuri="timer://foo?fixedRate=true&amp;period=1000"/><touri="jms:topic:timer?connectionFactory=demo"/></route><routeid="service-a"><fromuri="jms:topic:timer?connectionFactory=demo"/><touri="jms:queue:service-a?connectionFactory=demo"/></route><routeid="service-a-log"><fromuri="jms:queue:service-a?connectionFactory=demo"/><touri="log:service-a"/></route></camelContext></beans>

Route list

The method http://host:8181/jolokia/read/org.apache.camel:type=routes,*returns a list of routes with details.

Sample return data for a route service-a:

"org.apache.camel:context=a.xml,name=\"service-a\",type=routes": {
            "StatisticsEnabled": true,
            "EndpointUri": "jms:\/\/topic:timer?connectionFactory=demo",
            "CamelManagementName": "a.xml",
            "ExchangesCompleted": 173,
            "LastProcessingTime": 2,
            "ExchangesFailed": 0,
            "Description": null,
            "FirstExchangeCompletedExchangeId": "ID-...",
            "StartTimestamp": "2018-12-17T07:01:12Z",
            "FirstExchangeCompletedTimestamp": "2018-12-17T07:01:13Z",
            "LastExchangeFailureTimestamp": null,
            "MaxProcessingTime": 35,
            "LastExchangeCompletedTimestamp": "2018-12-17T07:04:05Z",
            "Load15": "",
            "DeltaProcessingTime": -8,
            "OldestInflightDuration": null,
            "ExternalRedeliveries": 0,
            "ExchangesTotal": 173,
            "ResetTimestamp": "2018-12-17T07:01:12Z",
            "ExchangesInflight": 0,
            "MeanProcessingTime": 4,
            "LastExchangeFailureExchangeId": null,
            "FirstExchangeFailureExchangeId": null,
            "Uptime": "2 minutes",
            "CamelId": "camel-3",
            "TotalProcessingTime": 827,
            "FirstExchangeFailureTimestamp": null,
            "RouteId": "service-a",
            "RoutePolicyList": "",
            "FailuresHandled": 0,
            "MessageHistory": true,
            "Load05": "",
            "OldestInflightExchangeId": null,
            "State": "Started",
            "InflightExchanges": 0,
            "Redeliveries": 0,
            "MinProcessingTime": 0,
            "LastExchangeCompletedExchangeId": "ID-...",
            "Tracing": false,
            "Load01": ""
        }

Parts and a lot of them of particular interest for the construction of the graph represent RouteId, Context, EndpointUri, State, Uptime.

It is important to mention that the method returns and metrics on the route: ExchangesTotal, ExchangesCompleted, ExchangesFailed, ExchangesInflight, etc.

The above method covers 90% of our data needs, but what it does not return is information about exit points from the route. To obtain this information, you need to use the method of obtaining details of the route and the method of obtaining the scheme. One of the methods is not enough, since in some cases the methods do not return all the data necessary to form a list of exit points.

Route details

Route details are obtained from the method
http://host:8181/jolokia/exec/org.apache.camel:context=a.xml,type=routes,name="service-a"/createRouteStaticEndpointJson(boolean)/true

Sample return data:

{
    "request": {
        "mbean": "org.apache.camel:context=a.xml,name=\"service-a\",type=routes",
        "arguments": ["true"],
        "type": "exec",
        "operation": "createRouteStaticEndpointJson(boolean)"
    },
    "value": "{\"routes\": {  \"service-a\": {    \"inputs\": [      { \"uri\": \"jms:\/\/topic:timer?connectionFactory=demo\" }    ],    \"outputs\": [      { \"uri\": \"jms:\/\/queue:service-a?connectionFactory=demo\" }    ]  }}\n}\n",
    "timestamp": 1545040570,
    "status": 200
}

Route map

The route map is obtained from the method
http://host:8181/jolokia/exec/org.apache.camel:context=a.xml,type=routes,name="service-a"/dumpRouteAsXml(boolean)/true.

The method returns the route map in xml form only if it has been framed. For example, if a route is described using org.apache.camel.builder.RouteBuilder(used when describing routes in a spring application), then the method will not return anything.

Sample return data:

{
    "request": {
        "mbean": "org.apache.camel:context=a.xml,name=\"service-a\",type=routes",
        "arguments": ["true"],
        "type": "exec",
        "operation": "dumpRouteAsXml(boolean)"
    },
    "value": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<route customId=\"true\" id=\"service-a\" xmlns=\"http:\/\/camel.apache.org\/schema\/spring\">\n    <from uri=\"jms:topic:timer?connectionFactory=demo\"\/>\n    <to uri=\"jms:queue:service-a?connectionFactory=demo\" id=\"to5\"\/>\n<\/route>\n",
    "timestamp": 1545040727,
    "status": 200
}

Draw graph

By combining all the information received, you can safely draw a graph, I took advantage vis.jsand got this result.

Points - points of entry and exit, edges - routes, and gray numbers on routes - metrics ExchangesTotal.

Graphing for several services

The described approach to graph construction is also suitable for the case when camel is used not only in the data bus, but also in applications. For example, having described the route in the application as follows:

@ComponentpublicclassEventRoutesextendsRouteBuilder{
    @Overridepublicvoidconfigure()throws Exception {
        from("jms:topic:timer")
                .inOnly("bean:service?method=handle");
    }
}

You can combine all the data on the routes from servicemixand the application and draw a general graph.

Note that a new route appeared in the diagram from jms:topic:timerto bean:service.

Conclusion

By implementing the described approach to building a route graph, we were able to get a general picture for both the bus and integrated services. Actually, the graph of our tire looks like this:

Proof of concept applications can be found here - github