tirsdag 12. juni 2012

SCOM Reporting - High availability?

When moving RMS role to new MS, how to move Reporting references....

So reporting redundancy has been a pain point in SCOM 2007, and as to my knowledge it hasn’t been changed that much in 2012 either. It’s not like it’s a problem within SCOM rather the limitations lie in on how SRS works and gets configured. In this post I shall descibe a customer scenario where we had this Challenge With SCOM 2007, but during the near future I shall also be posting a workaround for SCOM 2012 as the same solution cannot be used due to removal of RMS role.

             So back in the SCOM 2007 days, you normally installed reporting and were asked to provide  information regarding the RMS, when you specified this it sort of became hardcoded in the reporting server. In case of RMS failure or move, you have to manually promote a MS to take over the RMS role and at the same time this information had to be updated in the reporting server. You started to get alerts stating Reporting failures. I have seen a couple of blogs describing setting up reporting in cluster, but out goal was to find a simple solution which was simple, efficient and easy to implement, despite technical level.

                So in order to do make this happen I used the DNS manipulation approach. I added a dummy a record with the RMS’s ip address. This actually did three things which make our lives easier afterwards.

                So in my DNS management console it looks a bit like this.

The idea is that if the RMS has to be changed, we just perform a simple change in the DNS console, and this takes care of the rest.  The reporting alias is registered at following locations

Located under HKLM\Software\Microsoft\Microsoft Operations manager\3.0\Reporting\DefaultSDKServiceMachine

And in the reporting configuration file

Now, with all this in place, you just have to change A host record to new ip and your good to go! Ths datawarehouse deployment error Messages should og away. If you want to se an immediate effect, you can run a ipconfig /flushdns on SRS server and just restart reporting.
This post describes how to install it on a New installed Reporting server, but the same approach can also be used on a Reporting server that needs to be made "easier to manage" in case of RMS movement.

lørdag 28. april 2012

SSL monitoring using SCOM and powershell (with .net)

Monitoring SSL certificates using SCOM and PS(.net)

My first blog entry so forgive the formalness, and spelling mistakes =P

So I was visiting a regular customer of mine who was repeating an old story like many of you have heared before, it goes like, our SSL on a web site had expired and we had to experience downtime along with many unhappy customers.
          While we were talking a question came up, is'nt is possible to monitor SSL certificates using SCOM? And my first response was yeah sure its possible. Hmmmm.... thats odd... why havent you told us about it and why were'nt did'nt we get any alerts before we started to have problems. And I was sitting there nicking like crazy and thinking why did'nt I do this?

So this started the whole procedure where I started to look at SCOM being a monitoring device for SSL sertificates. Now I see a lot of you guys going BOoooooring.... your going to serve us some shitty idea where you used synthetic transactions or URL monitoring right? NNNNNNoOooo I am not. Lets define the criteria for the requirement before we proceed.


                We want to be able to monitor SSL certificates remotely and get informed 15-20 days prior to certificate expiration so that the certificates can be renewed and redestributed before the current certificates expire. We also want to get rid of the XL page where we enter all our certificate information and maintain manually. 
                 Additional requirements? Hmmm we already have SCOM so it would be great to reuse it to generate alerts and notifications.

So now that the criteria has been defined lets get started with the proposed solution shall we?


              So the idea was to create a Powershell(PS) script which monitors certificates, creates alerts if the certificate is about to expire and if it is expired. The PS script does not creates the alerts, it just creates events in event log which are then caught by SCOM which is responsible for the alert generation.

SCOM elements:

                From SCOM perspective there are to requirements, first one is obvious. You need a working SCOM environment. And the other one is that the RMS or in SCOM 2012, the MS must be able to contact the destination server/client hosting the website over http/https. I shall assume at this point that we are not talking about ports and firewall configuration as you guys already are experts in these fields.
                By the way, I created a management pack in the authoring console, where a defined a couple of rules and monitors. I shall be uploading the managment pack as well, but as for now I am just describing the solution and providing the PS script. So, on the Mgmt server, a folder structure is created on the root of system drive. Here a simple CSV file is created which contains site address/URL, port number. Here both URL and ip address can be used, the script also checks the syntax or might rather call it formatting in the CSV file. If there are errors present an alert with line number and data present is shown in the Alert.

As you can see in the script below, I have just choosen some random events starting from 65001 as I never have seen them being used by any application or a bright programmer. You can choose your own, as far as the Filepath and eventSource are concerned they can be also changed as you wish.
The PS script is run as a scheduled task on the Mgmt server. I have added it in the mgmt pack, but it can also be created as a regular scheduled task. The mgmt pack also checks that the script has been run atleast once during the last 24 hours. Feel free to do any changes/improvements in the script and to comment.

PS script:

So the script looks something like this

#Define constants
$Filepath = "C:\serverlist\MonitoredServers.csv"
$EventSource = "Certificate_Monitoring"

# EventID definition:
# EventID 65001 = CSV file not found - Warning (2)
# EventID 65002 = CSV file is empty - Warning (2)
# EventID 65003 = CSV file contains errors (1)
# EventID 65010 = Could not connect to host or find host
# EventID 65020 = Certificate Expired
# EventID 65021 = Certificate about to expire
# EventID 65022 = Certificate OK
# EventID 65030 = Last runtime for this script

function checkCsvFile ##Checks for existence of CSV file

if (!(Test-Path -path $Filepath))
 $msg = "File not found: " + $Filepath
  $bcheckCsvFile = $false
 createEventlogObj 2 $msg 65001     #CSV File not found!
 $bcheckCsvFile = $true
  return $bcheckCsvFile
}# function readFromCsv ends here
function createEventlogObj     ##Creates and instance of eventlog and writes to logfiles    
param ($inputEventType,$OutputMessage,$EventID) #Can be 1 for error and 2 for warning and 3 for information!
#Should take a parameter which assigns Information(3),warning(2) or error(1)
$EventLog=new-object System.Diagnostics.EventLog("Application")
$EventLog.Source= "Script"

if ($inputEventType -eq 1)
    $Severity = $Error
    elseif($inputEventType -eq 2)
    $Severity = $Warning
    elseif($inputEventType -eq 3)
    $Severity = $Information
    #Condition is not possible to occur! Future use
    $Eventlog = $Null
    $Severity = $Null

}# createEventlogObj function ends here
function readCsvFile{           #####Reads in data from CSV file
$csv = Import-csv -path $Filepath -header ("servername","port") -Delimiter ","
$CurrentLine = 0
$csvCount = $csv.count

foreach ($line in $csv) {
$CurrentLine = $CurrentLine + 1
       if($($line.servername -ne "servername")-AND ($line.servername -ne "")-AND ($line.port -ne "")){#Do not process headers in csv file
            getCertificateDetails $line.servername $line.port #Call function and pass values
            } #If condition ends here
            # Write to event log with event
                if($CurrentLine -NE 1){
                    $msg = "Formatting error on line: "+ $CurrentLine
                    $ExtendedMsg = $msg + " Data in input file is: " + $line
  if($CurrentLine -LT $csvCount)
                     {createEventlogObj 1 $ExtendedMsg 65003} #Call function and log error to application log
                  }# If ends -Do not Log error for 1.st line

} #Foreach condition ends here
}#readCsvFile function ends here

function getCertificateDetails([String]$ConnectedServer="localhost",[String]$ConnectedPort=443)
#Now check servername and perform connection test
    if (($ConnectedServer -NE "") -AND ($ConnectedServer -NE $Null))
           $HostConnection = new-object system.net.sockets.tcpclient($ConnectedServer,$ConnectedPort)
            if (($HostConnection -EQ $null) -OR($HostConnection -EQ "")){
                 $msg = "Failed to connect to host: " + $ConnectedServer +" on port: "+$ConnectedPort
                    #Logg this to eventlog
                    #Variables to pass 2, Write-host text, EventID
                    createEventlogObj 2 $msg 65010 #Call function and log error to application log
                $stream = new-object system.net.security.sslstream($HostConnection.getstream())
                #send hostname on cert to try SSL negotiation 
                #Write-host "The value returned by stream is:"$stream
                        if (($stream -NE $null) -AND ($stream -NE "")){
                            $certificate = $stream.get_remotecertificate()          
                        #Now check the certificate for values.........
                        $validto = $certificate.getexpirationdatestring()
                        #write-host "Value in validto is:"$validto
                        $subject = $certificate.get_subject()    
                        $issuer = $certificate.get_issuer()    
                $DaysToExpire = New-Timespan $(Get-Date) $($validto)
                    if($DaysToExpire -le 0){
                    # Certificate is Expired - Error -Log details to event viwer and make scom generate alert!
                    $msg = "Certificate for " + $ConnectedServer + " has expired. It was valid to: "+ $validto + " Certificate was issued by:" + $issuer + " subject is: " + $subject
                        createEventlogObj 1 $msg 65020 #Call function and log error to application log
                    elseif(($DaysToExpire -le 30) -AND ($DaysToExpire -GE 1))
                    #Certificate about to expire - warning
                    $msg = "Certificate for " + $ConnectedServer + " is about to expire in: " + $DaysToExpire + " days and should be renewed. It is valid to: "+ $validto + " Certificate is issued by: " + $issuer + " effected certificate subject is: " + $subject
                    createEventlogObj 2 $msg 65021 #Call function and log error to application log
                    #Certificate valid - Information
                    $msg = "Certificate for: " + $ConnectedServer + " is valid for " + $DaysToExpire + " days "
                    createEventlogObj 3 $msg 65022 #Call function and log to application log
                    #Write information events with days left for expiray? Or do nothing!
                   } #If $stream value is zero test
                }#If $HostConnection ends here
    {# $ConnectedServer value is empty or Null- Log to eventlog
    #write-host "Do nothing! No hostname!"
}#Function getCertificateDetails end here

$procceed = checkCsvFile #Call function to check if CSV file exists... Must also check if empty, wrong format etc
if ($procceed -eq $true)
#Open CSV file and repeat the function for all servers..........
readCsvFile #Call function to open CSV file and then report status of all certs
#Write-host "CSV file not found! Check eventlog for details!"
$msg = "Done processing csv file at: " + $(Get-Date)
createEventlogObj 3 $msg 65030 #Call function and log error to application log

#****************************Script ends***********************************

I shall be adding a couple of pictures showing how this script is used in production as well as the MP whcih creates alerts in SCOM. I just need to figure out how to post them. Still my first blog post remember?
 So here comes the pictures are promised........

The above picture shows the location of files which can be changed in PS script.

Above picture shows contents of the CSV file

Above picture shows alerts that are generated in SCOM console

Above picture shows events that are generated in Application log