Overview

1. Asynchronous uni-directional data flow [from Nagios SBI to Netcool] : In this method of integration, Netcool shall receive events  as forwarded, but shall not acknowledge the event back in Nagios. This is useful when Nagios is not used by operators for RT monitoring.
2. Synchronous bi-directional data flow: An event in Nagios will flow to Netcool and will be confirmed back in Nagios as recieved by Netcool. On every update on the event [such as journal entry, acknowledgements] the event in Netcool, status shall be updated in Nagios.

Either options work based on the business/solution requirements. So without further ado:

Implementation:

1. Asynchronous uni-directional data flow [from Nagios SBI to Netcool]

To understand the implementation, I shall divide the steps as southbound implementation and northbound implementation. Southbound implementation refers to the changes/configuration on Nagios end, and Northbound implementation refers to updates in Netcool.

Southbound updates [On Nagios];

a) Create a script to send tcp socket messages or snmp traps or direct JDBC insert to NBI.

You can use snmptrap command for writing the script, if you are not a SNMP guy you can use a simple script to do socket message communication/JDBC inserts into Objectserver. Test this script.

sample snmp script:

Send trap

# Arguments:

# $1 = Management Station

# $2 = Community String

# $3 = host_name

# $4 = service_description (Description of the service)

# $5 = return_code (An integer that determines the state

# of the service check, 0=OK, 1=WARNING, 2=CRITICAL,

# 3=UNKNOWN).

# $6 = plugin_output (A text string that should be used

# as the plugin output for the service check)

#

# Sample

# /usr/bin/snmptrap -v 2c -c $2 $1 ” NAGIOS-NOTIFY-MIB::nSvcEvent nSvcHostname s “$3″ nSvcDesc s “$4″ nSvcStateID i $5 nSvcOutput s “$6″

b) Define a global event handler in Nagios: Global event handler will help execute the script on every state change on Nagios instance and will communicate, failure and seizure of the problem. How to configure GEH: http://nagios.sourceforge.net/docs/2_0/eventhandlers.html

Northbound updates [On Netcool]

If SNMP:

a) Download the Nagios MIB and compile with MIB2Rules

http://sourceforge.net/projects/nagiosplug/files/nagiosmib/

b) Update the rules file and include it  in mttrapd main ruleset

If socket:

a) Update the socket probe to parse message based on delimiters

b) Ensure all mandatory objectsesrver fields are accounted for

If JDBC:

a) Ensure all mandatory objectsesrver fields are accounted for

b) **CAUTION** Watch the objectserver profiler for IDUC consumption, as this is not so much of a conventional approach

DID YOU CATCH THE HEADFAKE?

Nagios an Netcool were just examples, you can integrate most FM-FM/FM-PM solutions using the aforementioned procedure, you just need to know the NBI data model, SBI data model, right triggers on the SBI system and right listner on NBI system. Made your life easy, din’t I? So start saving your company some money now!!

In the next post, I will talk about method 2 {bidirectional data flow}. Keep visiting!!

LTE [Long term evolution] is the one of the proposed 4th generation radio access network technology and if all goes as planned the world will be wireless and with much higher data rate after a successful implementation. Recent tests on the field have been successful and all the  investments planned by US telecom market indicates that this is definitely going to be the future of access Networks. The main node of this network would be the eNodeB which would encompass the functional behavior of multiple nodes of our current network paradigm. The end goal architecturally is to have a flat architecture for 4G networks. End goal from user perspective is increased data rate and quality, along with reduced cost and access anytime/anywhere.

SAE [System Architecture Evolution] on the other hand will be the core for the 4G networks, is focused on a all IP, flat architecture, improved data rate and reduced CAPEX/OPEX expenditures. Evolved packet core [EPC] to which the eNodeB will connect, serves as the central functional unit of the core architecture.

Now, what does all of the above mean to Service and Network management  as it is known today to what it would become in the coming years of 4G networks. Will get to this in my next post. Stay tuned!

References:

http://www.radio-electronics.com/info/cellulartelecomms/lte-long-term-evolution/sae-system-architecture-evolution-network.php

“Self-configuring and self-optimizing network use cases and solutions: Release 9”; 3GPP TR 36.902; Sept, 2009

Detailed below is the template:

1 Executive Summary

CONTENTS OF THIS SECTION: This section an overview of the content of the rest of the report, giving key facts that management would like to know about its contents.  The executive summary should give the most important aspects of the report while omitting details and some supporting information.  Generally speaking, the summary should be not longer than 1 page and preferably as short as possible while conveying the required information.

1B  [Optional, for mature organizations] Strategic Capability Network

Analysis of how the strategy aligns with the organizations capabilities and resources. You can safely skip this section if you already have a defined BSM strategy and a competitive analysis document detailing the value propositions that drive the business. For details refer the patent here and my analysis with an example here.

2 Introduction

CONTENTS OF THIS SECTION: This section gives the name of the system and describes its high-level functions.  This is expanded upon by the history and stakeholders sections.

2.1 History

CONTENTS OF THIS SECTION: This section provides the historical context for the system.  It answers how the system was developed and by whom.

2.2 Stakeholders

CONTENTS OF THIS SECTION: This section provides a list of the stakeholder roles important to the system.  For each, the section lists the concerns that the stakeholder has that can be addressed by the system.

3 Architecture & Problem Background

CONTENTS OF THIS SECTION: The sub-parts of Section 3.1 explain the constraints that provided the significant influence over the architecture.

3.1 System Overview

CONTENTS OF THIS SECTION: This section describes the general function and purpose for the system or subsystem whose architecture is described in this SAD.  Include a high-level context diagram of the system and summarize major inputs and outputs.

If you don’t know how to build an accurate context diagram, look here.

3.2 Goals and Context

CONTENTS OF THIS SECTION: This section gives the name of the system and describes its high-level functions that the BSM solution is offering and more importantly how the solution would fit into the current value chain of the organization.

3.3 Significant Driving Requirements

CONTENTS OF THIS SECTION: This section describes behavioral and quality attribute requirements (original or derived) that shaped the software architecture. Included are any scenarios that express driving behavioral and quality attribute goals.

This section should only list the key driving requirements and not detailed requirements for the solution.

4 Competative Landscape

CONTENTS OF THIS SECTION: This section lists and briefly describes the major competitors of the system.  Competitors are those systems that do the same thing as the system or those systems that could otherwise be used in place of the system.  It also gives a high level overview of the strengths, weaknesses, opportunities and threats of the system explained in more detail in the following sections.

4.1 Strengths

CONTENTS OF THIS SECTION: This section describes the functions that the system does well either in comparison with its competition or in absolute terms.

4.2 Weaknesses

CONTENTS OF THIS SECTION: This section describes the functions that the system does poorly in relation to its competitors or in absolute terms.  Also included could be features that competitors have but the system does not, or features that the system should have but does not given the stakeholders and high-level requirements described in the previous section.

4.3 Opportunities

CONTENTS OF THIS SECTION: This section describes what the opportunities are for the system.  Opportunities are factors external to the system (e.g., in the overall environment) such as general trends or actions of competitors that enable the system to increase its market share or usefulness to stakeholders.

4.4 Threats

CONTENTS OF THIS SECTION: This section describes the threats that the system is likely to experience.  Threats are factors external to the system such as general trends or actions of competitors that decrease the market share of the system or its usefulness to stakeholders; in the extreme case, threats might render the system obsolete.

5 Referenced Materials

CONTENTS OF THIS SECTION: This section provides citations for each reference document.  Provide enough information so that a reader of the SAD can be reasonably expected to locate the document.

6 Directory

6.1 Glossary

CONTENTS OF THIS SECTION: This section provides a list of definitions of special terms and acronyms used in the SAD . If terms are used in the SAD that are also used in a parent solution description document and the definition is different, this section explains why.

6.2 Acronym List

If you work in telecom or finance world, you would know as i do, the TLA’s [Three letter acronyms] are annoying from organization to organization. So, don’t assume – take 10 minutes and add value to your BSM document.

Acknowledgements:

SEI Architecture documentation

Professor Jeff Thompson

Professor J Vayghan

I would like to acknowledge Professor Jamshid Vayghan and Professor Jeff Thompson who have been instrumental while teaching Enterprise Architecture and Software Solutions Architecture which have given me an insight into how delivering Architecture related artifacts better.

This series shall detail the documentation guidelines for BSM solution strategy, architecture planning, implementation and lifecycle management. This series shall not include troubleshooting, production support and maintainance related documentation as these aspects will depend on tool suite, hardware, software and organization enviornment.

Groundworks 5.3 community edition is hosted on enterprise class application server/database, this came as a big relief after using Nagios which had some scalability issues. JBOSS and Mysql have given a good foundation to GroundWorks and provide some really nice Web 2.0 features like AJAX driven views/role driven dashboards etc… Furthermore GWS also provides some really cool reporting features. Note that the reporting functionality is for both realtime and historical information.

Hardware requirements for GWS are pretty straightforward i.e. 4 Gigs of RAM, 3 GHz CPU speed, 160 Gigs of harddrive, etc.. Net net – Nothing earthshaking.

What was really interesting is that GWS had a lot of plug ins incorporated, intuitive UI driven hostgroup/host setup, servicegroup/service setup, dependency control and service metric reporting features. This was a BIG plus and provided value of the shelf… For the existing Nagios users, GWS provides an seamless port over of existing functionality. Whats more is that installation took only 30 minutes!! Well do not confuse installation with ready solution — PLEASE. I am only referring to base solution.

If all of the aforementioned was not enough, the GWS solution provides a feature to discover  devices based on ping sweep and tcp layer discovery option as well; Auto discovery for free — Neat!!

So to summarize, GWS community edition turned out to be a very useful time investment due to following reasons:

1) Easy to install 20 minutes of initial setup

2) UI driven hostgroup/host/servicegroup/service/commands/dependencies

3) Historical and Realtime reporting features

4) Auto Discovery via ICMP protocol

5) AJAX driven role based views/dashboards

6) MySql archive

7) Checks for SNMP alarms [passive]

8 ) Active service checks [loads of them]

I would be interested to hear the feedback from others who have used enterprises edition on GWS BSM features provided in enterprise version of the solution.

So what are the various “Human” dimensions/perspectives of BSM

1. Skills: If you thought you can get BSM/Service Assurance implementations done by a bunch of system engineers with expertise on a bunch of tools; you are heading for a disaster. What you need is someone who can envision solution, understand the organization, communicate the ideas and implement them with agility. System engineers are an important part of the implementation, but they have biases towards what they know best which works out ok if what they know can scale to the needs of the organization.
2. Ability to work with end users is another very important aspect for the success of the BSM solution; a word of caution here – If you heard sales presentations saying BSM is only for VP’s, Directors to know the “global strategic view”; stay away!! BSM belongs as much to the VP’s as it belongs to the technician “Joe” sitting in the Operations Center trying to understand why his walk in the datacenter and him pulling the cable can cause a loss of millions of dollars.
3. P2P of value chain: No P2P is not peer to peer but people to people communication of “value” of BSM solution and understanding the importance of communication of service impacting information to all expected stakeholders.

So, next time you are working on BSM implementation, try this – WALK AROUND!!  Ensure BSM is a part of every individual goals and objectives, the value that the VP’s and Director’s are expecting out of the solution is communicated to as many folks as possible. Also, the value that the technician Joe is expecting is communicated back completing full cycle of “value” flow. That is what makes the business better!! Have you ever seen all of the aforementioned  happening in a BSM implementation? I can bet that the answer in more than 80% of the cases is “NO”; and we ask why 8 of 10 BSM efforts fail to meet the proposed expected value.

BSM will only be “BS” if WE(the community) miss out on the perspective which ensures the alignment of the most important asset/pillar for the success of a business i.e. “Humans”.

For those of you who are not familiar with this topic, let me give a quick background. Most equipment vendors, provide MIBs/Off the shelf management modules to manage equipment from fault management/service assurance perspective based on standard TCP or UDP protocols like SNMP, TL1, Socket communication etc.  Various Telecom/Financial giants NMS teams debate the feasibility of managing huge number of events often millions in number in terms of volume per day; correlation/deduplication does reduce events to more actionable alarms but it does not solve reduce the actual root causes. So this leads us to a bigger question, what are root causes?

Does a NOC or Front office technician really care for Authentication failure alarms or those annoying informational and warning alarms provided off the shelf by vendor to “effectively manage the network”?

Following are the organizational factors to consider for effective event management:

1) The size and skills of the Layer 1 support NOC/Front Office: Ok, so if the Front Office is 4-5 guys, can they really handle 3000 critical alarms a day? Do they really need those trending alarms indicating that a T1 might be impacted in 4 hours or would they rather focus on the customer impacting outages? [I know that some would argue the very org. structure; but I will not try to influence business decisions which consider multiple dimensions of the picture, technology being one of them.]

The size of the team responsible for incident management is key for the fault management/service assurance team to ensure quality of alarming meets the expectations of the Organization.

2) The size & complexity of Application platform/Network: Size and complexity of the Application platform/Network plays an important role in defining alarms.

Example: For layer 1/core network – Technicians may want to know all trends to mitigate incidents from happening where as for layer2/layer3 network – Technicians may want only events indicating incidents impacting services.

Note: Understanding the network/applications from usage perspective helps immensely.

3) Customers & Services: Provisioned services and customer associations are important to the overall business objective. Understand them!

After understanding the aforementioned, you will know the organizational perspective and volume management perspective of events.

Now for the most important dimension of the debate on quality of alarming which constitutes of  accuracy, completeness and actionable alarms. Considering this factor, one might argue that only if we manage all identified alarms vs. whatever provided off the shelf – we can reach the goal of quality. Yes, i agree.

One the other hand, few might argue that by discarding unknown alarming we let some information which might impact services go unnoticed. Yes, i agree to this too. But the challenge is to balance these discards to the right level showing events which indicate right impact on the service.

That is why the challenge is not in getting the Right tool, its all using the tool Right!!

Measuring total number of customer calls which are not for an identified issue by SA solution. This is the highest priority and would serve as a report card for SA solution.

Inputs on the overall business continuity and growth/volume planning [strategic]:
Defining the trend analysis accountability team member

Encouragement/goveranance [strategic]:
Encouraging the team members for leveraging SA solution to ensure availability of services at all times. Association of individual performance to overall application availability and csi

Updating our customers on identified issues proactively[tactical]:
What happens sometimes is although teams are experiencing an issue, teams do not notify customer right away and start fixing the problem. Two options to fix this, documenting a problem management process with the first step to be a customer notification and communication, or auto email from the tool on the issue. Starting with the former would be a better approach.