Developing In The New RDF

The RDF agent which discovers information on remote devices is centred around Processors. For the Agent to perform a discovery operation such as configuration discovery or topology discovery it uses processors defined in the RDF Agent for the given technology and version of the target device. Processors are developed per technology and per supported discovery operation. For example, see below.

Example: Corning one supports Configuration, Performance and Alarm Sync.

The above example shows what is currently supported in the old RDF platform in the SNMP-Manager application. For this functionality to be re-written in the RDF Agent, a developer or operations engineer would need to define processors in the RDF Agent for each of these operations for this technology. Processors are defined as interfaces in the RDF agent for each of the given discovery operations - Configuration, Performance, Alarm Sync And Topology Discovery.

In the RDF Agent project, processors are located in the 'com.errigal.rdf.processing.technology' package. All processors are located here separated by technology name. If you are adding a new technology stack to the RDF Agent, you would add a new package directory in processing/technology package. See below:

As processors are interfaces, all of these can be implemented within the RDF agent by a single Java class, across multiple Java classes. There is no right or wrong way to do this as it depends on the target device and technology. Please refer to the Code and Design Best Practices & Principals section of this wiki. All processors extend a common interface base known as 'MessageProcessorBase'. This base interface contains all the common behaviour definitions which is shared between all other processor Interfaces - AlarmProcessor.java, ConfigProcessor.java, PerformanceProcessor.java to define the correct behaviour for a processor you will need to understand what the expected functionality is when implementing interfaces methods.

An example definition and explanation of each of the Message Processor Base Interface methods are outlined below. Every processor interface in the RDF agent implements this base interface so it is important to understand what each of these definitions are and what is expected in the implemented behaviour. The below image shows the interface method signatures.

• The 'Build Request' method receives an incoming message object as an argument.
  • This argument contains all of the request information which was sent by the requesting application e.g target device IP address, technology, version etc
  • The incoming message is used to form a set of tasks objects which are used to complete given discovery operations.
• This method returns a Request object which contains the list of defined tasks
  • Your constructed task list would be a simple list such as List<Task> which contains all the relevant tasks needed to complete the request discovery operation defined in the incoming message.
• Finally a call to Request.builder().discoveryTasks(taskList).build() then can be made to create an instance of the request object.

See below for an example of a definition of the 'buildRequest' being overridden and defined as part of an RDF processor implementation.

  public Request buildRequest(IncomingMessage incomingMessage) {   //Interface Method

    /*Incoming Message is populated by the Requesting Application i.e SNMP Manager*/

    List<Task> taskList = new ArrayList<>();
    String ipAddress = incomingMessage.getHost();   /* IP address of target device for discovery data */
    String snmpVersion = incomingMessage.getSnmpVersion(); /*Which SNMP version to use for discovery */
    long timeout = incomingMessage.getTimeout();   /*Timeout for operations */

    /* Paths in RDF agent To required MIBs  */
    String mibPath = String.join(",", SNMP_V2, COMBA_WIRELESS_ENHANCEMNT_REPEATERS, COMBA_WIRELESS_REPEATERS,
                                      COMBA_WIRELESS_REPEATER_MBDA_COMMINFO, COMBA_WIRELESS_REPEATER_MBDA_MODULEINFO,
                                      COMBA_WIRELESS_REPEATER_MBDA_RFINFO);

    /* Query params have additional information about element  */
    Map<String, String> queryParamsObject = incomingMessage.getQueryParams();
    String discoveredName = "";

    if (queryParamsObject != null) {
      discoveredName = incomingMessage.getQueryParams().get("discoveredName");
    }

    /*Device Topology Is an object used to store discovery data  */
    deviceTopology.setDiscoveredName(discoveredName);
    deviceTopology.setIpAddresses(Arrays.asList(ipAddress));
    deviceTopology.setIsComponent(false);
    deviceTopology.setHardwareType("HUB");

    /* Determine the type of discovery requested... Is it Config or Performance, Incoming message defines this, then create tasks to do this discovery */
    if (incomingMessage.getDiscoveryType() == IncomingMessage.DiscoveryType.CONFIGURATION) {
      log.info("Creating Config Task.....");
      Map<String, CombaConfigParameter> combaConfigParams = CombaConfigParameterMapping.getMapping();
      combaConfigParams.forEach((String oid, CombaConfigParameter value) -> taskList.add(createParamTask(oid, "CONFIG", timeout, mibPath, snmpVersion, ipAddress)));
    } else if (incomingMessage.getDiscoveryType() == IncomingMessage.DiscoveryType.PERFORMANCE) {
      log.info("Creating Performance Task.....");
      Map<String, CombaPerformParameter> combaPerformParams = CombaPerformParameterMapping.getMapping();
      combaPerformParams.forEach((String oid, CombaPerformParameter value) -> taskList.add(createParamTask(oid, "PERFORM", timeout, mibPath, snmpVersion, ipAddress)));
    }

    /*Create a Request Object to encapsulate your task(s) for the discovery operation  */
    taskList.forEach( t-> log.info("Task Items: "+t.toString()));
    return Request.builder().discoveryTasks(taskList).build();
  }

Forming tasks to complete the definition of the Request Method can be done inline in the method itself or helper methods can be used. For example, if you want to perform a Topology discovery on a device, your formed tasks as HTTP could look like.

1. Login.
2. Retrieve Topology via HTTP.
3. Logout.

If your RDF agent processor only does topology, it would be fine to create such tasks inline in the 'buildRequest' method. But if your processor supports multiple operations such as configuration, performance and alarm sync it might be better to break your task creations into separate methods, this helps with readability and maintainability. An example of creating a login and logout tasks as utility methods can be seen below.

  private Task createLogoutTask(String url, HashMap<String, String> headers, long timeout) {
    MultiTechLoginRequestDTO dto = MultiTechLoginRequestDTO.builder().username(incomingMessage.getUsername()).password(incomingMessage.getPassword()).build();
    String logoutParams = gson.toJson(dto);
    return Task.builder().name("LOGOUT").type(TaskType.HTTP_POST)
      .url(url + "/api/logout").timeout(timeout)
      .headers(headers).httpRequestBody(logoutParams.getBytes()).build();
  }

  private Task createLogoutOtherUserTask(String url, HashMap<String, String> headers, long timeout) {
    MultiTechLogoutRequestDTO dto = MultiTechLogoutRequestDTO.builder()
            .logoutUser(incomingMessage.getUsername())
            .username(incomingMessage.getUsername())
            .password(incomingMessage.getPassword()).build();
    String logoutParams = gson.toJson(dto);
    return Task.builder().name("LOGOUT_OTHER_USER").type(TaskType.HTTP_POST)
            .url(url + "/api/logout").timeout(timeout)
            .headers(headers).httpRequestBody(logoutParams.getBytes()).build();
  }

• This method definition is used to process and handle the data or results of your defined tasks.
• The return type of this method is used to capture any sub-tasks which may be needed as your tasks defined in 'buildResponse' are processed.
  • Subtasks are not necessary so this can be null on method completion.
• Each of the tasks defined in 'buildResponse' and any subtasks each get passed through the processResponse method to be handled.
• The arguments for this method are the current task results and the next task results. In most instances, the next task reference is not needed or used in the current task execution.
• To store the results of these tasks(your discovery results) a class member object of type 'CommonDeviceTopology' is defined, using this object the necessary discovered data is stored.
• You don't know which task is currently being handled so it is typical to see logic is this method to check the tag of the current tag for its name, then it can be handled properly.

See below for an example of a definition of the 'processResponse' being overridden and defined as part of an RDF processor implementation.

  public Stack<Task> processResponse(Task task, Task nextTask) {  /* Interface Method */
    Stack<Task> subTasks = new Stack<>(); /* If any subtasks are required, add them to this data structure */
    if (task.hasErrors()) {  /* If there are any errors  with the current task do not proceed with it */
      log.info(task.getName());
      log.error(task.getName()+" Task Has Errors: "+task.toString());
      errors.addErrors(task.getErrors());
      String errorsStr = Arrays.toString(task.getErrors().toArray());
      log.info("Param Processor Error - " + task.getName() + " - " + errorsStr);
    } else if (task.getName().contains("CONFIG")) {   /* Process config releated tasks with this logic */
      SNMPResponse configResponse = (SNMPResponse)task.getResponse(); /* Result of the config task is here */
      log.info("Task Stuff "+task.toString());
      log.info("Task Name:" + task.getName() + " SNMP Performance Task Data: " + configResponse.toString());
      processConfigResponse(configResponse);  /* Process configuration response, stores result data in a Device Topology Object*/
      log.info("Config Device: "+deviceTopology.getConfigData()); 
    } else if (task.getName().contains("PERFORM")) {    /* Process performance releated tasks with this logic */
      SNMPResponse performResponse = (SNMPResponse)task.getResponse(); /* Result of the performance task is here */
      processPerformResponse(performResponse); /* Process performanbce response, stores result data in a Device Topology Object*/
      log.info("Performance Device: "+deviceTopology.getParameters());
    }
    return subTasks;
  }

• A simple getter definition for returning an Error object containing all encountered errors.
• Errors can occur in your defined tasks and when this occurs each error can be stored in a class member object called Errors.
• Errors are checked when handling Tasks in the 'processResponse', each Task has a boolean flag which can be accessed with 'task.getErrors()'
• Any recorded errors will be returned to the request application indicating there were some issues with the discovery process.

See below for an example of a definition of the 'getErrors' being overridden and defined as part of an RDF processor implementation.

public Errors getErrors() { /* Interface Method */
  return errors; /* return the populated Errors object, this is populated in processResponse if there are errors */
}

• Is Compatible is used so the RDF Agent knows which processor it can use to complete the discovery request.
• The requesting application can ask for discovery on a device in which the technology or version is not currently supported, this method allows processors to define if they support a particular technology and version.
• The arguments to the 'isCompatible' method are two Strings, one for the technology type and another is for the device's version.
• The incoming message passed from the requesting application contains information on the target device such as technology and version, this information is used to return a true or false determination if a processor supports this device or not.

public boolean isCompatible(String technologyName, String version) {
  return name.equals(technologyName) && supportedVersion.isCompatible(version); /*Technology Name and Version are got from incoming message */
}

As mentioned in the above sections, all of these methods are extended by specific processor interfaces for each of the discovery operations such as topology, alarm sync, config or performance. This section will go over the additional methods which need to be overridden when extending such interfaces.

• Getter which returns a CommonDeviceTopology object which can contain the discovered configuration parameters.

private CommonDeviceTopology response = new CommonDeviceTopology(); //CommonDeviceTopology stored the results of discovery operations
  
/*REST OF PROCESSOR - CommonDeviceTopology gets populated by Tasks */

/* Interface method for ConfigProcessor, contract to return Common Device Topology */
public CommonDeviceTopology getConfigs() {
 return response; //This is the object that the JSON response is formed and returned to the requesting application
}

• Getter which returns a CommonDeviceTopology object which can contain the discovered configuration parameters.
• Return type is object, but return a CommonDevice Topology containing performance parameters.

private CommonDeviceTopology response = new CommonDeviceTopology(); //CommonDeviceTopology stored the results of discovery operations
  
/*REST OF PROCESSOR - CommonDeviceTopology gets populated by Tasks */

public interface PerformanceProcessor extends MessageProcessorBase {
  public Object getPerformance(); //This is the object that the JSON response is formed and returned to the requesting application
}

• Again, the same interface as Config and Performance, but a different method name.

private CommonDeviceTopology response = new CommonDeviceTopology(); //CommonDeviceTopology stored the results of discovery operations
  
/*REST OF PROCESSOR - CommonDeviceTopology gets populated by Tasks */

public CommonDeviceTopology getTopology() {
  return deviceTopology;
}

• A recurring theme, the same interface as Config, Performance and Topology but different method name.
• Returns a CommonDeviceTopology object containing alarm data.

private CommonDeviceTopology response = new CommonDeviceTopology(); //CommonDeviceTopology stored the results of discovery operations
  
/*REST OF PROCESSOR - CommonDeviceTopology gets populated by Tasks */
public CommonDeviceTopology getAlarms() {
  return response;
}

Don't get confused with the word topology in the name, this object is for storing the results of ANY discovery operation e.g Config, Performance, Alarm Sync or Topology discovery. This section will go through how to populate the Common Device Topology object with discovered data. It is important to note, all the data that is added to this object is what is serialised into JSON and sent to Orchestrator and eventually to the requesting application. First of all, lets outline the properties of a Device Topology object. Please note this is Java, so access modifiers restrict direct access to properties, the appropriate getter's and setter's need to be used.

private String name;                 /* Device Name on Parent Device e.g DAS Controller */        
private String discoveredName;       /* Expected Discovered Name Of Parent Device e.g DAS Controller */
private List<AlarmData> alarmData;   /* Discovered Alarm list, note the Object is of type Alarm Data */
private List<ConfigNode> configData; /* Discovered Config param list, note the Object is of type Config Node */
private List<Parameter> parameters;  /* Discovered Performance param list, note the Object is of type Parameter */
private String hardwareType;         /* Hardware Type of System or Component Type */
private String elementType;          /* Network element type */
private List<String> ipAddresses;    /* Any associated IP's */
private Boolean isComponent;         /* Is this a component operation */