Automate F5 Backups

tape-bkp

I like to use Solarwinds CatTools to back up my network gear. It’s a great tool that allows you to easily schedule automated backups of network devices–or any command-line device for that matter.

This is where this post come in. There is no inherent template for backing up an F5 device using CatTools (at least right now).

I have a template that I’ve used over the years to create a UCS file based on the hostname and date, store a local copy in the event I need to restore it, as well as TFTP the file to the built-in TFTP server in CatTools for a remote backup.

CatTools Config:

First you’ll need a copy of CatTools somewhere that can reach your F5 via. SSH and have the ability to TFTP from the F5 to the CatTools server. I won’t go over the install or making sure your ACL’s allow this, but make sure you do this first.

  1. Open CatTools Manager
  2. Add your F5 devices under the Devices tab by clicking Add
  3. Select F5 as the vendor and F5.BigIP as the device type (although I don’t think this ultimately matters for much other than reporting).
  4. Pick a name and type in a Host Address (IP address of the F5 management or self IP with ssh permitted inbound)
  5. Pick SSH2 as your method. f5bkp1
  6. Click the Passwords tab, and add a user allocated just to CatTools. It will need to be an administrator with advanced shell. f5bkp2

Now you have your device(s) ready for backup. I like to make sure I have L3/4 connectivity by testing the Telnet/SSH button to make sure a prompt comes up. If not, work on your routing/ACL’s.

  1. Click the Add button under the Activities tab
  2. Under Activity, pick Device.CLI.Send commands option
  3. Add a description that makes sense to you. f5bkp3
  4. Click the Time tab, and set your schedule however you’d like.
  5. Under Devices, pick the F5(s) you added in the first steps.

The Script

Under the Options tab is where the commands will be entered on the device. The script has several lines which I’ll describe below.

export date=`date +"%y%m%d"​`

This sets an environment variable to the current date, year, month and day for file naming. Change this depending on your location or needs.

export filename=$HOSTNAME.$date.ucs

Now we set the filename to the hostname of the F5 and the date we just defined, followed by ucs.

tmsh save /sys ucs /var/local/ucs/$filename

Now we execute the tmsh command to create and save the UCS file to the directory we want.

NOTE: Depending on how frequently you set the backup, you could fill up your F5’s local storage. Although this would take a long time, you can create a script to auto-delete the files if you want, or delete them every 6 months or so when you are in the UI.

cd /var/local/ucs
tftp -m binary 192.168.1.10 -c put $filename

Now we move to that directory, and TFTP the file to the Cattools server. Change the 192 address to your Cattools server.

This is what it will look like in the UI:

f5bkp4

Click OK and run the activity. If you did everything right there should be UCS files in your TFTP directory in CatTools. If not, check ACL’s or routing to make sure you aren’t missing anything.

 

Automate F5 Backups

F5 HTTP Response-Code Alerts

503

UPDATE: Added more logic to remove reporting anomalies from the F5 device (inaccurate response code values). The IF statement before the email pipe will cause the report to not send if the value is empty or the top server responded with less than 15% of the total response codes of that type.

I love the F5 analytics feature, specifically for HTTP. The cool thing about the feature, is that you can get a very good idea of an issue based on response code behavior.

Let’s say you have a URL that is monitored by an external HTTP tool. This tool checks the URL to make sure it gets a 200 HTTP response-code back every 60 seconds. I just got an email that the check failed, but this URL has 10 nodes behind it. How do I know which node is failing? 

Well, you can manually log into the F5 and check through the UI, use the Rest API or some other utility that ties into the F5 API.

  • Manually checking is not ideal, we want automation.
  • The Rest API is great, but why invest a ton of time getting that working, if this is our only goal?

My preference would be to add this to the Analytics Profile on the F5, so any administrator could see it and know what’s happening. Unfortunately, F5 does not alert on HTTP response-code in the Analytics Profile (as of 12.1).

Hopefully F5 does this at some point, but until then, I’m going to show you how to do this with a bash script on the F5 device itself.

Prerequisites

  1. Determine which response code do you want to check for
  2. Determine how often you want to check for potential issues
  3. Make sure you have an email server that the F5 can forward the message through.
  4. Make sure you have access to the F5 advanced shell

The Script

I’ve done most of the heavy lifting for you. I intentionally defined any part that I thought may need to be customized based on usage as variables, change them to suit your needs.

Place it wherever you want, probably somewhere under /root is best so someone could find it easily. Before you schedule the script, make sure you chmod 755 it so it can be executed (chmod 755 /root/script.sh).

#!/bin/bash

#=============================
#** Define your environment **
#** specific variables here **
#=============================

#Which response code do you want to look for?
#Add a trailing space to prevent any possible time matches 
#(the year is followed by : in the output)
code="302 "

#How many minutes in the past do you want your script to look?
#Match this up with cron
howoften="30"

#Topx determines when an email is triggered. If you are implementing this 
#in a dev environment with not much going on, you may want to limit your 
#response code trigger to when it is in the top 4 of all response codes. 
#If you are running in a prod environment, maybe top 7 is fine. 
#If you are getting too many emails, make the number lower. If you want to 
#make sure things are working, make the number higher.
topx="8"

#Email configuration
smtpserver="192.168.10.1"
sender="f5script@email.com"
receiver="me@email.com"
subject="$code Error Notification"


#-----------------------------

# Determine number of response codes in the past x minutes. 
# Typically if response codes are in the topx response-code 
# types, something we care about is happening. 

numcode=(`tmsh show analytics report view-by response-code limit ${topx} range now-${howoften}m | grep $code | awk '{print $3}'`)

# Start our logic to send alerts
# Change the number to insert more granularity than your alerting
if [[ "$numcode" -gt 11000 ]]; then
 # What is the hostname of this F5 for the email
 hostname=`uname -n`
 
 # This will break down the top culprit of the
 # queried response code into three values in an array
 # IP|PORT|CODE
 badguy=(`tmsh show analytics http report view-by pool-member drilldown { { entity response-code values { ${code} } } } limit 1 range now-${howoften}m| tail -n+8 | sed 's/:/ /;s/ | / /'`)
 
 badguysname=`nslookup ${badguy[0]} | awk -F "name = " '{print $2}' | sed '/^\s*$/d'`
 if [[ -n "$badguy" && "$((${badguy[2]}*100/$numcode))" -gt 15 ]]; then
 ( echo "${code}Response Code Report";
 echo "------------------------------";
 echo "";
 echo "Overall, there have been *$numcode*, $code messages in the past $howoften minutes for all VIPs running on $hostname.";
 echo "";
 echo "The top offender of these messages is ${badguy[0]} on port ${badguy[1]}.";
 echo "";
 echo "This server has generated $((${badguy[2]}*100/$numcode))% of the total ${code}messages in this time window.";
 echo "";
 echo "nslookup tells us the bad server is: $badguysname ";
 echo "";
 echo "Total $code : $numcode"
 echo "Node $code : ${badguy[2]}"
 echo "Bad Node : $badguysname"; ) | mailx -S smtp="$smtpserver" -r "$sender" -s "$subject" -v "$receiver" 
 fi
else
 echo "${code}response codes are not in the top $topx overall for this device. Script ended, no email sent."
fi

The email portion is a bit clunky, but with the version of software I was running, I wasn’t able to send an HTML email. If you figure that out, or want to edit the email content, go for it.

Scheduling the Script

Now we need to schedule the script. Use crontab -e to schedule your script. Read this article from F5 for some tips if you need more info. Make sure you match up the timing of your cron task with the script defined time window.

Make sure you document that you have done this!! If you leave your job and a new administrator comes in, they would have to do some digging to figure out what you have done.

I would recommend testing this by setting your response code to 200, if you get emails based on the schedule you set, change to what you want to look for.

F5 HTTP Response-Code Alerts

Cloud-Based Web Application Check-Script

production_inspection

Monitoring applications can be tricky. As a network engineer it’s important that the applications that I serve up are available and working properly. To network engineers it’s important that resources are up and running from the Internet. This script checks a variety of aspects pertaining to a public URL.

  • Availability (TCP ports)
  • Content matching (HTTP GET)
  • Certificate Expiration
  • Security (SSL disabled, proper chain, etc.)
  • DNS (cert name mismatch, DNS lookup)

The script runs from a Linux server I have in the cloud. Use any platform you choose. All that is required is Perl, curl, openssl and netcat.The MIME::Lite and DATE::Calc Perl modules are also required.

The script will run through a list of URL’s defined by you and check for content we’d expect when the application is up. We run through a lot of other tests mentioned above. Take a look at the script and see what it checks for.

The script will push output to an HTML file that allows for “user-friendly” appearance in email alerts and the HTML file can be uploaded to a webserver.

Configure the script to run periodically with a cron job.

This is a free, easily modified script that can help monitor your applications without much overhead. Use and modify to your specific needs.

#!/usr/bin/perl
use MIME::Lite;
use Date::Calc qw(
Date_to_Days);

#############################################################
# -----VERSION NOTES-----              
# 1.0: Base configuration
# 1.1: Added color/formatting changes to output
# 1.2: Added ncat port validation after failure
# 1.3: Added email functionality
# 1.4: Modified code to send HTML instead of text
# 1.5: Save HTML output to file and SFTP to dashboard server
# 1.6: Added additional logic for failure types, e.g. 404, etc.
# 2.0: Added Email logic for detecting failures
# 2.1: Added cert checking logic


#############################################################
# To test an application, add the full URL of the app you
# want to test to the apps variable. Secondly add a string of 
# content that matches when the application is up and running. 
# If the content doesn't match, the application is marked as 
# down. 
#############################################################

# Applications short or common name goes here e.g. 'Customer WebUI'
@shortname = (	#App1
				'Google',
				#App2
				'Fake Domain'
			);
			  
# FULL application URL goes here to test for availability and content
# !!!!!MAKE SURE YOU ADD THE PORT NUMBER AFTER THE HOSTNAME FOR THE TEST TO WORK!!!!!
@apps = (	'https://www.google.com:443/',
			'https://fakedomain.domains.net:443/'
		);

# Strings to match against when you know the application is up go here. 
@validator = (	'Google Search',
				'Bad'
			);
			  
# Count number of variables in array to loop can run properly
$numapps = (scalar @apps);

# Define variables

# Needed for our while statement
$a = 0;
# If one failure check passed, don't check for other failures. Reduces redundant information.
$b = 0;
# If c = 1, app is down. Used for e-mail notifications
$c = 0;

# Define HTML array to be sent via. SFTP
my @html = ();

# Set datetime
$time = `TZ=":US/Eastern" date`;

# Add environmental header in HTML file
push @html, '<head><title>App Status</title></head>';
push @html, '

<h1>Application Status</h1>


';
push @html, '

<h3>The applications listed here are checked once every 30 minutes</h3>


';
push @html, '

Last Checked: ' ;
push @html, $time;
push @html, '

';

# Tell the user that something is happening
print "The test is running, please wait... \n";

# Begin the checks
while($a < $numapps){

 # Grab URL components to allow for more segmented testing, use variables as needed
 my($protocol, $host, $port, $uri) = $apps[$a] =~ m|(.*)://([a-zA-Z0-9\-.]+):([0-9]+)?(.*)?|;

 # Lets run some quick security tests first
 $sslv2check = `timeout 3 openssl s_client -connect '$host':443 -ssl2 2>/dev/null`;
 $sslv3check = `timeout 3 openssl s_client -connect '$host':443 -ssl3 2>/dev/null`;

 #********************
 #* CERT DATE LOGIC *
 #********************
 
 # Get todays year, month and date
 $year = `TZ=":UTC" date +%Y`;
 $month = `TZ=":UTC" date +%m`;
 $date = `TZ=":UTC" date +%d`;
 
 #Convert todays date to a numeric date count
 $now = Date_to_Days($year,$month,$date);
 
 # Grab certificate date
 $cert = `timeout 3 openssl s_client -connect $host:443 -tls1 2>/dev/null| openssl x509 -noout -enddate 2>/dev/null| cut -f2 -d'=' | xargs -0 date +%F -d 2>/dev/null`;

 # Split the output into DATE MONTH DAY
 my @certdates = split('-', $cert); 


 #End certificate expiration logic
 
 if ($sslv3check =~ /Server public key is/) {
 $sslv3check = 1;
 print "We matched the SSLv3 security check\n";
 }
 if ($sslv2check =~ /Server public key is/) {
 $sslv2check = 1;
 print "We matched the SSLv2 security check\n";
 }

 # The standard check with a 60 second timeout. This is what grabs the HTTP information response from server
 $check = `curl '$apps[$a]' -m 60 2> stderr.txt`;
 
 # If the check failed, we want to preserve that information. Stderr is saved to a variable and the checks below are ran. 
 undef $stderr;
 my $stderr = `cat stderr.txt`;
 
 # Matched DNS unresolvable 
 if ($stderr =~ /curl: \(6{1}\)/) {
 
 # I set this here so we don't run other redundant checks.
 $b = 1;
 
 print "We matched the DNS stderr log\n";
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a>: <b>We couldn\'t resolve the host that you specified</b> using public DNS servers. Something is amiss...'; 
 }
 #Matched a TCP reset
 elsif ($stderr =~ /curl: \(56\)/) {
 
 $b = 1;
 
 print "We matched the TCP Reset rule\n";
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a>: <b>We were sent a reset!!</b> Since we did not see the construction page, there are a few reasons for this:<ul>';
 push @html, '<li>The server is not accepting requests at all. Check the server locally and make sure it does the same with localhost.</li>';
 push @html, '<li>The firewall is blocking us, DNS could be wrong, or the firewall isn\'t listening on this port</li></ul>'; 
 } 
 #Matched a cert name mismatch
 elsif ($stderr =~ /curl: \(51\)/) {
 
 undef $check;
 $check = `curl -k '$apps[$a]' -m 60 2> stderr.txt`;
 
 print "We matched the Cert Name Mismatch rule\n";
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a>: <b>Requested DNS name does not match the servers certificate!!</b>';
 push @html, '<p>Check the cert that the server is providing. If the cert appears fine, make sure the utility is using the name you are expecting.';
 push @html, 'Depending on the browser, a client might not notice this, but it is best practice to fix this issue.</p>';
 }
 #Matched SSL chain failures
 elsif ($stderr =~ /curl: \(60\)/) {
 
 $b = 1;
 
 # There was an SSL issue. Lets run our curl check again without forcing chain validation and continue
 undef $check;
 $check = `curl -k '$apps[$a]' -m 60 2> stderr.txt`;
 
 print "We matched the Cert-chain stderr log, we will run curl again in insecure mode\n";
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a> <b style="color:green">is up</b> with caveats... <h3>NOTE:</h3>We found an issue with the certificate chain that your server provides. Validate the chain your server is sending.<br>';
 push @html, 'You can use <i>ssl-labs</i> to check the chain. Here is an automated link to check yourself.<ul><li>';
 push @html, '<a href="https://www.ssllabs.com/ssltest/analyze.html?d=';
 push @html, $host;
 push @html, '&hideResults=on&latest" target="_blank">SSL-Labs Test</a></li></ul>';
 push @html, 'The application may be working, but best practice we should make sure the chain your server is sending is what it should.<br>';
 
 if ($sslv3check == 1) {
 push @html, '<br><b>Note:</b> This server allows the SSLv3 protocol. <i>Shame on you!!</i><br>';
 }
 if ($sslv2check == 1) {
 push @html, '<br><b>Note:</b> This server allows the SSLv2 protocol. <i>Extra shame on you!!</i><br>'; 
 } 
 }
 
 # Comparison statement to see if the content of the curl contains validator
 
 # APP IS UP
 if (($check =~ /$validator[$a]/) && ($b == 0)) {
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a> is <a style="color:green">up!!</a>';
 
 if (($sslv3check == 1) || ($sslv2check == 1)) {
 push @html, '<br>This app accepts SSL, which is an unsecure protocol!!<br>';
 push @html, 'You can use <i>ssl-labs</i> to check the server. Here is an automated link to check yourself.<ul><li>';
 push @html, '<a href="https://www.ssllabs.com/ssltest/analyze.html?d=';
 push @html, $host;
 push @html, '&hideResults=on&latest" target="_blank">SSL-Labs Test</a></li></ul>';
 } 
 if ($sslv3check == 1) {
 push @html, '<br><b>Note:</b> This server allows the SSLv3 protocol. <i>Shame on you!!</i><br>';
 }
 if ($sslv2check == 1) {
 push @html, '<br><b>Note:</b> This server allows the SSLv2 protocol. <i>Extra shame on you!!</i><br>'; 
 }
 }
 # WE FOUND A 404
 elsif ($check =~ /404 - File or directory not found./) {
 
 $c = 1;
 
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a> is showing a <b style="color:red">404 error</b>, check the server!!';
 }
 # WE FOUND A Server Error
 elsif ($check =~ /An unhandled exception occurred/) {
 
 $c = 1;
 
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a> is showing a <b style="color:red">Server Error</b>, check the server!!';
 } 
 # DIDNT FIND VALIDATOR, RUN OTHER TESTS
 else {
 print "The first check failed, we will wait 30 seconds then check again.\n";
 # Wait 10 seconds, then run the test again, just to make sure
 `sleep 30s`;
 undef $check;
 $check = `curl '$apps[$a]' -m 60 2> stderr.txt`;
 
 # Skip this test if we matched something above
 if (($check !~ /$validator[$a]/) && ($b == 0)) { 
 $c = 1;
 
 push @html, '<hr><p>We didn\'t find this content: <i>';
 push @html, $validator[$a]; 
 push @html, '</i> in the <b>';
 push @html, $shortname[$a];
 push @html, '</b> application. <b style="color:red">The application appears to be down!!</p><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">Click here to check the URL</b></a><br>';
 
 # PORT CHECK SECTION
 # The content we expected was not found. As long as the host exists, lets run a port check on it. 
 if ($host ne "" ) {
 
 $ncoutput = `timeout 10s ncat -v '$host' '$port' &> ncat.tmp`;
 $cat = `cat ncat.tmp | grep Connected`;
 
 if ($cat =~ /Connected/) {
 push @html, '<h3>Summary:</h3><ul><li>From a network perspective, <b style="color:green">everything seems ok.</b></li>';
 push @html, '<li>We couldn\'t find the content we expected to see. We got there, but application may not be running/installed properly.</li>';
 push @html, '<li>If you see an \'Under Maintenance\' page, <b>check the inservice.txt file</b> on your server and make sure it loads locally.</li></ul>';
 push @html, 'Manually check the URL (above) and verify that what is presented is expected.</p>'; 
 
 if (($sslv3check == 1) || ($sslv2check == 1)) {
 push @html, '<br>This app accepts SSL, which is an unsecure protocol!!<br>';
 push @html, 'You can use <i>ssl-labs</i> to check the server. Here is an automated link to check yourself.<ul><li>';
 push @html, '<a href="https://www.ssllabs.com/ssltest/analyze.html?d=';
 push @html, $host;
 push @html, '&hideResults=on&latest" target="_blank">SSL-Labs Test</a></li></ul>';
 } 
 if ($sslv3check == 1) {
 push @html, '<b>Note:</b> This server allows the SSLv3 protocol. <i>Shame on you!!</i><br>';
 }
 if ($sslv2check == 1) {
 push @html, '<b>Note:</b> This server allows the SSLv2 protocol. <i>Extra shame on you!!</i><br>'; 
 }
 }
 else {
 
 push @html, '<h3>Summary:</h3><ul><li>We couldnt find <b>content in the code</b> from the URL that we were expecting to see. </li>';
 push @html, '<li>We then ran a pinch test from the Internet and it <b>also failed.</b></li></ul>';
 push @html, 'Our guess is that the server is either <b>hard-down, DNS is not resolving</b> or something on the network is <b>not responding.</b><br>';
 push @html, 'Next step is to view the URL from the web. If nothing loads, check the same URL local to the server.<br>';
 push @html, 'If that loads you are most-likely facing a network/DNS issue.</b>';

 if (($sslv3check == 1) || ($sslv2check == 1)) {
 push @html, '<br>This app accepts SSL, which is an unsecure protocol!!<br>';
 push @html, 'You can use <i>ssl-labs</i> to check the server. Here is an automated link to check yourself.<ul><li>';
 push @html, '<a href="https://www.ssllabs.com/ssltest/analyze.html?d=';
 push @html, $host;
 push @html, '&hideResults=on&latest" target="_blank">SSL-Labs Test</a></li></ul>';
 } 
 if ($sslv3check == 1) {
 push @html, '<b>Note:</b> This server allows the SSLv3 protocol. <i>Shame on you!!</i><br>';
 }
 if ($sslv2check == 1) {
 push @html, '<b>Note:</b> This server allows the SSLv2 protocol. <i>Extra shame on you!!</i><br>'; 
 }
 }
 }
 }
 # APP IS UP
 elsif (($check =~ /$validator[$a]/) && ($b == 0)) {
 push @html, '<hr><a href="';
 push @html, @apps[$a];
 push @html, '" target="_blank">';
 push @html, $shortname[$a];
 push @html, '</a> is <a style="color:green">up!!</a>';
 
 if (($sslv3check == 1) || ($sslv2check == 1)) {
 push @html, '<br>This app accepts SSL, which is an unsecure protocol!!<br>';
 push @html, 'You can use <i>ssl-labs</i> to check the server. Here is an automated link to check yourself.<ul><li>';
 push @html, '<a href="https://www.ssllabs.com/ssltest/analyze.html?d=';
 push @html, $host;
 push @html, '&hideResults=on&latest" target="_blank">SSL-Labs Test</a></li></ul>';
 } 
 if ($sslv3check == 1) {
 push @html, '<br><b>Note:</b> This server allows the SSLv3 protocol. <i>Shame on you!!</i><br>';
 }
 if ($sslv2check == 1) {
 push @html, '<br><b>Note:</b> This server allows the SSLv2 protocol. <i>Extra shame on you!!</i><br>'; 
 }
 }
 }

 #######################################
 # CERT CHECK HTML #
 #######################################
 
 #If we couldn't pull a cert, prevent script from crashing
 if ($cert eq "") {
 print "We couldn't connect to the domain to run a cert check.";
 }
 else {
 # Convert year/mo/day to numeric date count
 $certnow = Date_to_Days(@certdates[0],@certdates[1],@certdates[2]);
 
 # Subtract current date and cert expiry date
 $certdaysleft = ($certnow - $now);
 
 #If the cert is beyond expiration, let us know
 if ($certdaysleft < 0) {
 push @html, '<br><b style="color:red">NOTE!!</b> The certificate for this domain has expired <b>' . $certdaysleft . "</b> days ago!!";
 }
 elsif ($certdaysleft <= 14) {
 print "\nCert expires in " . $certdaysleft . " days!!\n";
 push @html, '<br><b style="color:red">NOTE!!</b> The certificate for this domain will expire in <b>' . $certdaysleft . "</b> days!!";
 }
 }
 
 ####################################
 # ---- EMAIL CONFIGURATION ---- #
 ####################################

 if ($c == 1) {
 
 print "Compiling email...please wait...\n";
 
 $from = 'appchecker@mydomain.com';
 $to = 'me@mydomain.com';
 my $subject = $shortname[$a] . ' appears to be down!!';
 print $subject . "Sending notification email!!\n";
 my $body = '<font face="calibri">
<h2>Health Check Failed:</h2>

<a href="' . @apps[$a] . '" target="_blank">Click here to check URL</a>' . 
 '
<h4>Time of failure: </h4>

' . $time . 
 '

A public cloud server checks for content behind this URL periodically. We ran two tests against it, and couldn\'t find what we were looking for (' . $validator[$a] . 
 '). Click the URL to validate. After manually checking, if the page loads as you\'d expect, it may have been under heavy load at that given time. ' . 
 'Check historical monitoring tools for any possible outages or heavy load.

</font>';

 $msg = MIME::Lite->new(
 From => $from,
 To => $to,
 Subject => $subject,
 Data => qq{$body}
 );

 $msg->attr("content-type" => "text/html"); 
 $msg->send;
 }
 
 # Make sure our error parsing file is empty after each run
 `cat /dev/null > stderr.txt`;
 $a++;
 undef $b;
 undef $c;
 undef $check;
 
}


####################################
#  ---- SFTP Configuration ----    #
####################################
#
# Optionally upload HTML file to an SFTP site for viewing
# SFTP the file to the site of your choice
# Authentication method uses keys not interactive passwords

# Make sure index.html exists and is empty before we start
`touch index.html`;
`cat /dev/null > index.html`;

# Define filename
my $indexfile = 'index.html';

# Write HTML array to file
open (FILE, ">> $indexfile") || die "\nProblem opening $indexfile\n";
print FILE @html;
close (FILE);

# Create a simple batch file to put file on your SFTP site first
# Copy file to SFTP server
`sftp -b mysftpbatch.bat mysftpsite.domain.com`;

# Remove temporary file
`rm -f ncat.tmp`;

print "\nThe test is done, check the URL results\n";
Cloud-Based Web Application Check-Script

Powershell: Find stale Users

old user

Having old, unused user accounts sitting in your domain can send red-flags flying by auditors. It’s easy to create an account and think it will be used for the foreseeable future, but often times this is not the case.

This script queries your domain for user accounts that have not logged in in the past 6 months. It creates an HTML report and emails you the results. The email can be reviewed and action can be taken based on your discretion. I have added an option for you to un-comment which will automatically disable accounts that fit this criteria. Powershell v3 with the new Active Directory module is required for this to work.

<# Workflow

Query AD for users who have not

logged in in over 6 months.

#>

 

#Import AD module

Import-Module ActiveDirectory

 

# 6 months ago

$6months = Get-date

$6months = $6months.adddays(-180)

 

$Format = @{Expression={$_.SamAccountName};Label=“User Login Name”},`

@{Expression={$_.name};Label=“Account Name”},`

@{Expression={$_.whencreated};Label=“Created”},`

@{Expression={$_.passwordneverexpires};Label=“Password Never Expires”},`

@{Expression={$_.lastlogondate};Label=“Last Logon”},`

@{Expression={$_.enabled};Label=“Enabled”}

 

#Filter for users who havent logged in for 6 months as the basis of the filter. Add some additional properties.

Get-ADUser -filter { (lastlogondate -le $6months) } -properties * | Sort-Object lastlogondate | ConvertTo-Html $Format -Title “AD User Report” > .\Documents\Users.htm

 

#Send the email

Send-MailMessage -to you@domain.com -Subject “AD Account Report” SmtpServer 192.168.1.1 -From server@domain.com -Attachments .\Documents\Users.htm

 

#Un-comment this line to automatically disable the accounts rather than send a report 

#Get-ADUser -filter { (lastlogondate -le $6months) -and (enabled –eq “true”) } | Set-Aduser -enabled $false

–>

Powershell: Find stale Users

Powershell: Query domain for expiring certificates

Certificate expirations can be a pain to manage and are often overlooked. Some people have spreadsheets, set calendar reminders or just wait until a customer complains. I used to have a Unix script that would search an entire subnet for servers with expiring certs, but it was not very robust, searching subnets can return some questionable results.

This script starts by querying active directory to get a list of computer names that match string(s) that you enter. Then powershell searches the servers for certs that are within 14 days of expiring. The script will then email you an HTML report if there are certs expiring, if not it will do nothing. This script is highly customizable, so tweak it as needed.

<#

Script to check AD computers

for expiring certificates.

Author: Ryan Clark

Date: 4/1/13

#>

 

 

#Import AD module

Import-Module ActiveDirectory

 

#Make sure computers array is empty incase script has been run
in this session before

$Computers.Clear()

 

#Fill array with computers you want to filter by this example
is computers that start with DEV and PRD

$Computers = Get-ADComputer -Filter ‘Name -like “DEV*” -or Name -like “PRD*”’ | Foreach {$_.Name}

 

#Count Computers

$CompNum= $Computers.count

 

#Set a date variable for today and two weeks ago. Change the
date in TwoWeeks to modify the expiration time

$Today = (Get-Date).ToString(yyyy/MM/dd)

$TwoWeeks= (Get-Date).AddDays(+30).ToString(yyyy/MM/dd)

 

#HTML Style config

$a = “<style>”

$a = $a + BODY{background-color:white;}”

$a = $a + TABLE{border-width:1px;border-style: solid;border-color: black;border-collapse: collapse;}”

$a = $a + TH{border-width:1px;padding: 5px;border-style: solid;border-color: black;background-color:#D0A9F5}”

$a = $a + TD{border-width:1px;padding: 5px;border-style: solid;border-color: black;background-color:#FAFAFA}”

$a = $a + “</style>”

 

#Run through each computer and check for certs within the
configured date period

$i=0

Clear-content C:\Admin\CertReport.htm

while($i lt $CompNum)

{

Write-Host “Working on:” $Computers[$i]

#If your certs are in a different store change the Cert:\ path

$Certs = invoke-command ComputerName $Computers[$i] ScriptBlock {Get-ChildItem Cert:\LocalMachine\My}

$CertCount=$certs.count

$j=0

 while($j lt $CertCount)

 { 

   if (($Certs[$j].NotAfter gt $Today) -and ($Certs[$j].NotAfter lt $TwoWeeks) )

   {

    $Certs[$j] | ConvertTo-Html -head $a -title “Expiry Information” -property PSComputerName,Subject,NotAfter
>> C:\Admin\CertReport.htm

   }

   ++$j

 }

++$i

}

 

#Either email a report or do nothing

if
((Get-Content “C:\Admin\CertReport.htm”eq $Null)

{

 write-host
“No expiring certificates. Ending script.”

}

else

{

 #Modify the -to field to send to another user or DL

 write-host
“Expired certificates found, emailing report”

 Send-MailMessage -to me@mydomain.com -Subject
“Certificate Report” SmtpServer x.x.x.x -From myserver@domain.com -Attachments C:\Admin\CertReport.htm

}

Powershell: Query domain for expiring certificates

Autodelete Files by Age

In my last post I covered how to back up files based on age. This is a nice script to supplement it, this will auto-delete files based on their age. It’s an easy way to backup files to a disk somewhere and not overflow it with backups.

This example is for a Windows host, to delete a file older than 3 days:

1. Create a batch script with the following

echo on
 rem Delete files older than 3 days
 FORFILES /P C:\Admin\Test\ /S /M 1*.bmp /D -3 /c " CMD /c del /q @FILE "

2. Modify the following flags to suit your need

/p = The path to search for the files you want to check the date of and remove
/s = Recurse subdirectories contained within the path specified using /p and check them as well
/m = The search mask to be used for the file type you want to check the date on (*.* being all files)
/d = The date to compare the files against. A standard date type can also be used (dd/mm/yyyy)
/c = The command to be used on a file that matches the /m and /d criteria
/q = Used within /c to instruct the del command to delete files quietly

3. Add the batch file to the scheduler based on your need

Autodelete Files by Age

Script a Backup with Date

Lets say you have a file or folder that you want to automatically backup. The backups need to have unique names so we will name them based on the date. Use this if you want to automate a backup job that you want to FTP somewhere else or just a simple backup without involving a third party app.

This example is using 7-zip on a Windows platform. I’m sure there are a million ways to do this in Unix.

1. Install 7-zip.

2. Create a new batch file and use the following as a guide.


cd "C:\Program Files\7-Zip"
set dt=%date:~10,4%%date:~4,2%%date:~7,2%
7z a -w C:\Backups\Backup%dt% C:\inetpub\wwwroot
In this example, the backups will go to the C:\Backup directory and be called BackupDate e.g. Backup20100310. In this example we are backing up the IIS directory.


Once you have this batch file written, you can schedule it in task scheduler to run whenever you want. Schedule it daily, monthly, etc.
Script a Backup with Date