Viewing entries in
automation

Comment

Using flint to orchestrate BMC Remedy

BMCs Remedy ITSM platform is one of the most commonly used and mature Service Management Systems - but automating it can be tough.

In our previous blog post we got you started with flint's orchestration platform. In this post we'll show you how to query Remedy Forms, how to handle the response, how to create new records or how to update existing ones.

Let's take a typical integration use case: An external system is called on a regular base. Depending on the response, new records are created in Remedy or existing records are updated.

In our example we have a custom build Remedy Application. This application requires up-to-date exchange rates between EUR and USD. We'll poll an external Webservice to get the current exchange rate and we'll store them in a Custom Remedy Interface Form. We'll check the form for existing records and update them if necessary.

To get this example running you need a BMC Remedy Server and a running copy of Flint. You'll also need:

AR Restful API

A Restful API wrapper around the BMC Remedy Java API. For a quick start just run:

docker run -d -p 8080:8080 vipcon/arapi

on your docker environment. This will pull the latest Version from Docker Hub and deploy it in your environment.

Remedy Toolkit

Our Remedy Toolkit wraps up the AR Restful API calls and provides some basic call handling. We're going to provide a native Remedy connector in the near future. Contact us to get a copy.

Exchange Rate Webservice

We'll also make use of the Exchange Rate Webservice provided by Fixer.

Flint Configuration

In our example we'll use two different HTTP connectors. One for the Remedy connection and one for the Exchange Rate WebService call. You can get away with only one, but using two will give you more control about scalability and security. E.g. You might want to deploy the Remedy Connector on a separate Grid Node within your DMZ.

In our example the connector configuration is very basic and does not require anything else than the name.

Fixer exchange rate connector

AR Restful API connector

We'll also use a Global Configuration for the Remedy connection. You can also provide these values with each request, but storing them only ones seems to be more appropriate.

Sample configuration for the Remedy connection

You can configure the following values:

arapi - The URL of your AR Restful API

connector - The name of the HTTP Connector that should be used.

username - The Login ID of the Remedy User that should be used.

password - The password of the Remedy User.

port - The port of the Remedy Server. If you use a Portmapper, enter 0.

server - The name or IP of the BMC Remedy AR Server

Remedy Interface Form

We'll use a very simple Remedy Interface Form to store the conversion rate of the day.

Remedy Interface Form

You can download the .def File of the Form here.

The Code / Workflow

For our interface we'll just need a simple Flintbit to query the Webservice and update the interface form.

First we'll get the current Exchange rates from Fixer.

//call the connector
def exchangeResponse=call.connector("Fixer")
         .set("method","get")
         .set("url",url)
         .set("timeout",15000)
         .sync()

This call will return something like this:

{  
   "base":"EUR",
   "date":"2016-02-15",
   "rates":{  
      "AUD":1.5624,
      "HRK":7.62,
      "USD":1.118,
      "ZAR":17.6114
   }
}

Based on the Run Date we can check the Remedy Interface form for existing records:

def query = "'Run Date' = \"${runDate}\""
def queryResponse = call.bit("remedy:base:query.groovy")
                  .set("form",interfaceForm) // Set arguments
                  .set("query", query)
                  .sync()

If we find existing entries, we'll update all of them:

queryResponse.data.each { myRecord ->
    log.debug "Update " + myRecord.getKey()
    def recordData = "{ \"${myRecord.getKey()}\": { \"Currency\": \"EUR -> USD\", \"Exchange Rate\":" + rate + ", \"Run Date\" : \"${runDate}\" } }"
    def updateResponse = call.bit("remedy:base:update.groovy")
                        .set("form",interfaceForm) // Set arguments
                        .set("data", recordData)
                        .async()
  }

For better performance on multiple entries we do the update asynchronously. Be careful if you do this for large number of records.

If we didn't find any existing records we'll create a new one:

def recordData = "{ \"MyRefId01\": { \"Currency\": \"EUR -> USD\", \"Exchange Rate\":" + rate + ", \"Run Date\" : \"${runDate}\" } }"
def createResponse = call.bit("remedy:base:create.groovy")
                      .set("form",interfaceForm) // Set arguments
                      .set("data", recordData)
                      .sync()

The full code looks like this:

import groovy.json.*

//build the url
def url="http://api.fixer.io/latest"
def targetRate = "USD"
def interfaceForm = "manyos:currency"

log.info "Query Fixer for latest Exchange Rates"

//call the connector
def exchangeResponse=call.connector("Fixer")
         .set("method","get")
         .set("url",url)
         .set("timeout",15000)
         .sync()

//get the response
def myBody = exchangeResponse.body

//Create a Object out of the JSON response
def jsonSlurper = new JsonSlurper()
def myRates = jsonSlurper.parseText(myBody)

log.info "Query Remedy for existing records"

//Get the actual Values
def rate = myRates.rates.USD
def runDate = myRates.date
def query = "'Run Date' = \"${runDate}\""

def queryResponse = call.bit("remedy:base:query.groovy")         // Provide path for flintbit
                  .set("form",interfaceForm) // Set arguments
                  .set("query", query)
                  .sync()

//Check if record for today already exists - if yes: update, if not: create
if (queryResponse.data.size() > 0) {
  log.info "Entry found for query: " + query + ". Update instead."
  //update all records
  queryResponse.data.each { myRecord ->
    log.debug "Update " + myRecord.getKey()
    def recordData = "{ \"${myRecord.getKey()}\": { \"Currency\": \"EUR -> USD\", \"Exchange Rate\":" + rate + ", \"Run Date\" : \"${runDate}\" } }"
    def updateResponse = call.bit("remedy:base:update.groovy")         // Provide path for flintbit
                        .set("form",interfaceForm) // Set arguments
                        .set("data", recordData)
                        .async()
  }
} else {
  log.info "Create new Record"
  //Create a new Record
  def recordData = "{ \"MyRefId01\": { \"Currency\": \"EUR -> USD\", \"Exchange Rate\":" + rate + ", \"Run Date\" : \"${runDate}\" } }"

  def createResponse = call.bit("remedy:base:create.groovy")         // Provide path for flintbit
                      .set("form",interfaceForm) // Set arguments
                      .set("data", recordData)
                      .sync()
}

You can download the code here.

Scheduling the Integration

To make sure that we'll always have the latest data, we'll let our integration run every hour. To do this we just need to place a schedules.conf in our FlintBox. To run hourly it should have the following content:

"my-schedule" {
  description = "runs our integration every hour"
  trigger = "ars:remedy.groovy"
  cron = "0 0 0/1 1/1 * ? *"
  enable = false
  input {
  }
}

Conclusion

Integrating an external system into BMC Remedy is simple and fast forward with flint. Using an orchestration tool like Flint make developing integrations a lot faster and easier. For a simple interface like this you now only need a few lines of code.

In the next post we'll look into ITSM Automation by Configuration with flint and our Service Management toolkit. We'll also start building our own cloud with Microservices, BMCs ITSM Suite, Flint and Docker.

Comment

Comment

Start with flints automation engine

Automation of IT processes can be a huge time saver. Unfortunately beginning with automation can be quite challeging and expensive.

I was really excited when I heard from my friends at Infiverve that they have released an orchestration tool called flint.

Flint is an automation platform that uses Groovy and Ruby to create automated services. One of their big advantages is, that they store all of their workflows are stored within a Git repository. So it is easy to test and deploy different versions of your workflows.

Let's get started and see how it works...

What's in the Box

Once you have installed flint you can login to your server at http://yourflintserver:3500. After login you will see the dashboard which gives you an overview about your automation platform.

Grid

The Grid shows you all the nodes that are currently configured. It acts as kind of cluster and jobs will be distributed between all active nodes of the cluster. As long as one node in your grid is working you have a running automation platform.

Flintbox

A flintbox is a link to a Git repository and stores all workflows (called Flintbits) for one purpose.

Connectors

They let you talk to external systems. Some sample connector types are: HTTP, Database, SSH, Amazon EC2 and many more. Since there is no BMC Remedy Connector available, we currently create our own. 

Listeners

You cannot only talk to the world but also listen. Right now MQTT and IMAP Listeners are available.

Scheduler

With schedulers you can run your workflows periodically. The configuration is based on the Cron Syntax.

Global Config

Is great to store and share connection strings or parameters across all workflows.

TOD

The Trigger On Demand console let you run and test your workflows. It's great for debugging and developing purposes.

User Management

Let you create and manage different user accounts. Right now the functionality is very basic and each user has full access. But thats supposed to change in future releases.

Start Coding

First create a new Git repository. For this example we create a new public repository on Github. You can also use your private Git Server if you want. 

Now you can add your repository as new Flintbox on your grid.

Don't forget to enable your new Flintbox.

Since flint does not come with it's own IDE you can use any Editor you want to create your workflow scripts. I prefer to use Atom, a free Text Editor that is available for Mac, Windows and Linux. The great thing about Atom is that it comes with a plugin for flint that makes development a lot easier.

You can now go to the folder <flinthome>/flintbox/<yourflintbox> and start creating your first flintbits. If you have flint installed on a remote machine and your are working on a Mac I can highly recommend Transmit to open your files.

Let's create a simple "hello.groovy" flintbit.

log.info "Hello World"

Now let's run it in flint. Open the TOD console, enter flintsample:basic:hello.groovy as flintbit name. Don't forget to enable logging on the LOGS tab and press run.

You should see the following output:

That was easy - right?

Now let's make it a bit more dynamic. Change your flintbit to:

name = input.get("name") ?: "World"
log.info "Hello " + name

If you provide some input to your flintbit now:

Your output will change as follows:

Start talking

We can now run basic flintbits and handle input data. It's now time to interact with the outside world. Let's create a flintbit that calls OpenWeatherMap to get the current temperature and humidity based for a City provided by an input parameter.

First we need to create a new HTTP connector that we can use.

Now create a new flintbit called weather.groovy with the following content.

import groovy.json.*

//get config values
def apiKey = config.global("weather.key")

//get input parameters
def city = input.get("city")

//build the url
def url="http://api.openweathermap.org/data/2.5/weather?q=" + city + "&appid=" + apiKey + "&units=metric"

//call the connector
def wResponse=call.connector("OpenWeather")
         .set("method","get")
         .set("url",url)
         .set("timeout",5000)
         .sync()

//get the response
def myBody = wResponse.body

//Create a Object out of the JSON response
def jsonSlurper = new JsonSlurper()
def myWeather = jsonSlurper.parseText(myBody)

//Get the actual Values
log.info "Humidity: " + myWeather.main.humidity + "%"
log.info "Temp: " + myWeather.main.temp + "°C"

//Provide some output
output.set("Wetter", "${myWeather.weather.main}")
output.set("Temp", myWeather.main.temp.toString())

We have a quite a few improvements here. We read the API Key for the OpenWeatherMap service from a global configuration object with this code:

//get config values
def apiKey = config.global("weather.key")

This makes sure that the API configuration is stored in a single place and can be used by more than on flintbit. The configuration will look like this:

We call the HTTP-Connector with the following code:

//call the connector
def wResponse=call.connector("OpenWeather")
         .set("method","get")
         .set("url",url)
         .set("timeout",5000)
         .sync()

//get the response
def myBody = wResponse.body

This performs a GET Action on the OpenWeatherMap Webservice and stores the Body (JSON reply from the API) in the myBody Variable. The variable contains the following String:

{  
   "coord":{  
      "lon":11.58,
      "lat":48.14
   },
   "weather":[  
      {  
         "id":804,
         "main":"Clouds",
         "description":"overcast clouds",
         "icon":"04n"
      }
   ],
   "base":"cmc stations",
   "main":{  
      "temp":0.57,
      "pressure":942.92,
      "humidity":88,
      "temp_min":0.57,
      "temp_max":0.57,
      "sea_level":1015.72,
      "grnd_level":942.92
   },
   "wind":{  
      "speed":8.59,
      "deg":259
   },
   "clouds":{  
      "all":88
   },
   "dt":1455131881,
   "sys":{  
      "message":0.0098,
      "country":"DE",
      "sunrise":1455085705,
      "sunset":1455121684
   },
   "id":6940463,
   "name":"Altstadt",
   "cod":200
}

To make life a bit easier we can parse the JSON response into an Object.

//Create a Object out of the JSON response
def jsonSlurper = new JsonSlurper()
def myWeather = jsonSlurper.parseText(myBody)

Now we can easily access all the return values. Let's write the current temperature and humidity to the log and return some information:

//Get the actual Values
log.info "Humidity: " + myWeather.main.humidity + "%"
log.info "Temp: " + myWeather.main.temp + "°C"

//Provide some output
output.set("Wetter", "${myWeather.weather.main}")
output.set("Temp", myWeather.main.temp.toString())

If you run the flintbit you'll see the following information in the log:

So the weather is quite uncomfortable right now :)

The output of the flintbit will be the following:

This output could now be used by other flintbits. For example we could easily store it in a database or the textfile.

conclusion

Flint is a really promising automation platform with a lot of potential. It's easy to get you started with automation and it has a really small resource footprint in your environment. A lot of connectors are already there and some more are coming. In the future there will be an SDK available to create your own connectors. We're currently creating one to BMC's Remedy Platform right now. Flint can be a game changer and can make process automation more widely available for a lot of users.

Comment