How to monitor storage replication after Storage spaces direct node reboot (ex. after updates)

Hi!

I have two node Storage spaces direct scenario and after updating and rebooting one of the nodes in cluster I need to wait storage operations to complete (yes I am updating this scenario manualy :)).

If you want to check the progresss of this synchronization / repair of Storage spaces just drop this in Powershell on one of the nodes:

Get-StorageJob | Select Name,IsBackgroundTask,ElapsedTime,JobState,PercentComplete,@{label=”BytesProcessed (GB)”;expression={$_.BytesProcessed/1GB}},@{label=”Total Size (GB)”;expression={$_.BytesTotal/1GB}} | ft

You should get something like that…

Remember – if you have Storage spaces direct in two-node scenario you SHOULD WAIT for this job to complete – if you reboot second node to soon your CSV will go offline! So keep calm and Powershell! 🙂

System x3650 M5 – M5210 controller jbod mode – invalid arguments

Recently I wanted to use disks connected to M5210 raid controller as JBOD…
My controller has also additional cache and battery pack and in this configuration you are getting the error “The requested command has invalid arguments.” when trying to Apply change to JBOD…
001 002 003
well… If you physical remove the cache from your controller…
IMG_2476 IMG_2477
…and a reboot… you will be able to enable JBOD mode…
IMG_2479 IMG_2480 IMG_2481 IMG_2483 IMG_2484 IMG_2485 IMG_2486 IMG_2487 IMG_2488 IMG_2489 IMG_2490 IMG_2491 IMG_2492 IMG_2493 IMG_2494 IMG_2495 IMG_2496 IMG_2497
…finaly in Windows – disk manager you can see your disks…
IMG_2499

Important NOTE for those that would like to test Storage Spaces Direct technology that is comming with Windows Server 2016 (currently TP5)!

This will not work as BUS TYPE is still RAID and this is not supported – it should be SAS. The setup will fail at the step Enable-ClusterStorageSpacesDirect

008

The only solution is to buy a simple HBA as written here.

“…All SATA and SAS devices must be attached to a SAS Host Bus Adapter (HBA). This HBA must be a “simple” HBA, which means the devices shows as SAS devices in Windows Server…”

Mikrotik Netwatch ENHANCED!

Mikrotik has a great feature of NetWatch where you can ping something and if it is down you can run a script and if it comes back up you can run another script…

The problem is that sometimes you can lose only a ping or two and you would like your Mikrotik’s NetWatch to wait for a bit longer…

Becouse of that I have written a script that only after X seconds fires the configuration change and also checks if there is another script running (and it stops the execution).

So in Tools/Netwatch I have configured:
001
002
003
004
And now let’s go to the scripts:
005
006
007
008
And if we check the log how it works:

Example when NetWatch IP is down for more than 10 seconds and example wenh NetWatch IP is up for more than 10 seconds…

009

And here the example when during check something happens – so when IP goes down just for a couple of seconds and then returns  – and the second example when it is offline and it just comes up for a couple of seconds..

010

Scripts:

OnWlanUp:

:foreach j in=[/system script job find] do={:if ([/system script job get $j value-name=script] = “OnWlanDown”) do={/system script job remove $j;}}
:local countup value=0;
:local WLANSSID “wlan.novagorica.eu”;
:local SSIDUP [/interface wireless get “01-wlan.novagorica.eu” ssid];
:if ($SSIDUP != $WLANSSID) do={
:while (($countup < 10) && ([/ping address=10.10.0.1 interval=1 count=1]=1)) do={:set countup value=($countup+1); :delay 1000ms; :log warning message=”wlan.novagorica.eu – if UP check $countup”;};
:if ($countup=10) do={/interface wireless set “01-wlan.novagorica.eu” ssid=”wlan.novagorica.eu”; /interface wireless set “01-wlan.novagorica.eu” security-profile=01-wlan.novagorica.eu-noauth ; :log warning message=”wlan.novagorica.eu – gateway up!”;};
} else {:log warning message=”wlan.novagorica.eu – gateway up! – no need to change SSID”;}

onWlanDown:

:foreach i in=[/system script job find] do={:if ([/system script job get $i value-name=script] = “OnUp”) do={/system script job remove $i;}}
:local countdown value=0;
:local WLANSSID “wlan.novagorica.eu”;
:local SSIDUP [/interface wireless get “01-wlan.novagorica.eu” ssid];
:if ($SSIDUP = $WLANSSID) do={
:while (($countdown < 10) && ([/ping address=10.10.0.1 interval=1 count=1]=0)) do={:set countdown value=($countdown+1); :delay 1000ms; :log error message=”wlan.novagorica.eu – if DOWN check $countdown”;}; :if ($countdown=10) do={/interface wireless set “01-wlan.novagorica.eu” ssid=”wlan.novagorica.eu-offline”; /interface wireless set “01-wlan.novagorica.eu” security-profile=01-wlan.novagorica.eu-offline; :log error message=”wlan.novagorica.eu gateway down!”;};
} else {:log error message=”wlan.novagorica.eu – gateway still down! – no need to change SSID”;}

 

IPv6 in Windows environment for beginners – part 1 – ISP IPv6 delivery

Here we go… I have my environment up and running… So first of all we need to ask our ISP to give us IPv6 addreses.
In my case my provider – Telekom Slovenije is providing me IPv6 addresses in two different ways – depending on what kind of device I have at the locations.

So first case is where we have a L3 switch from our provider and in this case we have a /64 prefix on “wan” side of our router (connecting prefix) and /56 prefix routed over second IP address of connection prefix (which you can then split into /64 prefixes on your internal intefaces / vlans):

[admin@xxxx] /ipv6> address print
Flags: X – disabled, I – invalid, D – dynamic, G – global, L – link-local
#    ADDRESS                                   INTERFACE                                                           ADVERTISE
0  G 2a00:ee1:xxx0::2/64              ether3 – IPv6 internet uplink                              no
1  G 2a00:ee1:xxx2::100/64          lan                                                                            no

[admin@xxxx] /ipv6> route print
Flags: X – disabled, A – active, D – dynamic, C – connect, S – static, r – rip, o – ospf, b – bgp, U – unreachable
#      DST-ADDRESS              GATEWAY                  DISTANCE
0 A S  ::/0                     2a00:ee1:6700::1                1
1 ADC  2a00:ee1:xxx0::/64       ether3 – IPv6 interne…        0
2 ADC  2a00:ee1:xxx2::/64       lan                             0

Second case is where we have PPPoE session established and we receive /56 prefix by so called prefix delegation (in detail described @ GO6.si blog)

[admin@xxxx] > ipv6 dhcp-client print detail
Flags: D – dynamic, X – disabled, I – invalid
0    interface=pppoe-out1 status=bound duid=”0x00030001d4ca6d38f875″ dhcp-server-v6=fe80::230:88ff:fe04:99ec
request=prefix add-default-route=yes use-peer-dns=no pool-name=”isp-pd” pool-prefix-length=56
prefix-hint=::/0 prefix=2a00:ee2:68xx:xxxx::/56, 1h51m46s

[admin@xxxx] > ipv6 address print
Flags: X – disabled, I – invalid, D – dynamic, G – global, L – link-local
#    ADDRESS                                      INTERFACE                          ADVERTISE
0  G 2a00:ee2:68xx:xxxx::1/64       lan                                             no

When this is configured we should be able to ping some IPv6 addresses from our routers… As there is no advertising enabled everything in our network will remain calm and quiet waiting for further configuration.

IPv6 in Windows environment for beginners

Currently I am working on implementing dual-stack (so all servers and computers will run on IPv4 and IPv6 at the same time) in Windows envrironment with Active directory domain controlllers, other member servers (file server, DFS, SharePoint services…), Exchange server 2013, Lync/Skpye for business…

Purpose of this post is to walk you through the obstacles and difficulties while implementing both protocols to work together…

So basicaly we need to know the folowing:

We have one (or more) public IPv4 addresses which we NAT in our private networks where we have our servers…
Our providers gives us some IPv6 prefix for “wan” interface of our router and over that there is a routed prefix which we will use internaly (you need to know there are public – or globaly routed IPv6 addresses inside your network – SO TAKE CARE of your Firewall roules (We will cover that later)).

Simple steps to implement dual stack is to:

a. Get IPv6 from your provider
b. Have a router that understands IPv6 🙂
c. Configure router advertisment on internal network with M (managed (this will force users to use DHCPv6 instead of autoconfiguring IPv6 (SLAAC)) and O (other configuration (this will point clients to DHCPv6 server to get DNS servers (your domain controllers IPv6 addresses)) flag
d. Configure DHCP server on your Windows server with DHCPv6 parameters (prefix, exclusions, DNS servers (called: 00023 DNS Recursive Name Server IPv6 Address)
e. disable DHCP client on servers that use static IPv4/IPv6 addresses (if you do not do that your servers will autoconfigure additional IPv6 addresses as told by RA…) You can use Powershell: Set-NetIPInterface –InterfaceIndex <number> -Dhcp Disabled

Simple Hyper-V replica warning / critical state notification…

I have been searching for a simple solution to get the notification when something goes wrong with Hyper-V replica replication… The problem is that things can go realy bad if you have Hyper-V replica in critical state for long period of time (AVHD differential disk grows and can make a big mess on source server…)

So here is the PS script that checks replica health and sends mail if one or more of VMs have Warning or Critical replication state.

You can modify sending parameters by looking at examples @ http://petermorrissey.blogspot.com/2013/01/sending-smtp-emails-with-powershell.html

Just save the script as (for example) c:hvreplicamonhvreplicamon.ps1

Create a Task Scheduler taks (screenshots below)

if ((Get-VMReplication | select-string -inputobject {$_.Health} -pattern “Warning”) -like “Warning”)
{
$SMTPServer = “smtp.gmail.com”
$SMTPPort = “587”
$Username = “username@gmail.com
$Password = “password”
$to = “my@emailwhereiwanttoreceivealert.com
$subject = “Replica WARNING error on SERVERNAME”
$message = New-Object System.Net.Mail.MailMessage
$message.subject = $subject
$message.to.add($to)
$message.from = $username
$smtp = New-Object System.Net.Mail.SmtpClient($SMTPServer, $SMTPPort);
$smtp.EnableSSL = $true
$smtp.Credentials = New-Object System.Net.NetworkCredential($Username, $Password);
$smtp.send($message)
}
elseif ((Get-VMReplication | select-string -inputobject {$_.Health} -pattern “Critical”) -like “Critical”)
{
$SMTPServer = “smtp.gmail.com”
$SMTPPort = “587”
$Username = “username@gmail.com
$Password = “password”
$to = “my@emailwhereiwanttoreceivealert.com
$subject = “Replica CRITICAL error on SERVERNAME”
$message = New-Object System.Net.Mail.MailMessage
$message.subject = $subject
$message.to.add($to)
$message.from = $username
$smtp = New-Object System.Net.Mail.SmtpClient($SMTPServer, $SMTPPort);
$smtp.EnableSSL = $true
$smtp.Credentials = New-Object System.Net.NetworkCredential($Username, $Password);
$smtp.send($message)
}

Task Scheduler task parameters

task1

task2

task3

Windows server 2012 R2 – A port on the virtual switch has the same MAC as one of the underlying team members on Team Nic Microsoft Network Adapter Multiplexor Driver

If you have two or more NICs joined in a Team by using Windows server teaming solution and then you use this Team as a base for Virtual switch in Hyper-V and you enable “Allow management operating system to share this network adapter” like:

003 004 005

you will find warnings in system Event log:

A port on the virtual switch has the same MAC as one of the underlying team members on Team Nic Microsoft Network Adapter Multiplexor Driver

001

Check your NICs by using Powershell cmdlet: get-netadapter | ft Name,MacAddress

002

You need to change MAC addresses of your Virtual Switch management interface (vEthernet (xxx)) by using Powershell cmdlet: set-netadapter -name “vEthernet (XXX)” -macaddress xx:xx:xx:xx:xx:xx

007

and MAC addresses of your network cards (in my case 4 NICs) you can leave Team interface mac address alone…

009

There will be no errors in Event log any more 🙂