Friday, February 28, 2014

VMWare, CDP and non Cisco switches

So, I've been playing with CDP... everyone does it sooner or later. I'd come across documentation that claims the Brocade ICX-6610 switches that I've got talk CDP- which got me to thinking "Wouldn't that be nice to see from my VMHosts?"

Which got me digging through my switches for configuration docs- sparse at best, however... there's not much to document, especially with all you need to get it running. While trying to get this working I made some discoveries-
VMWare won't display CDP from non-Cisco devices. I have no idea why.
VMWare will output CDP just fine, meaning I can see my hosts from my switch!

To get VMWare to output CDP is pretty straight foward, with my preferred way being


Tuesday, January 28, 2014

Holy crap true believers!

And yes, stan the man would probably frown on that- but still.

So it's been the usual 4-5 months (or more) since my last update... talk about the more things change...

I've taken a job in Houston Texas to work for an MSP that I've known for awhile. In the two months that I've been here, I've learned quite a bit about tech that I've only wished I could work with. For instance- the HUS series of SANs from Hitachi. Beautiful boxes, easy to physically install... and incredibly easy to configure. I never thought I'd find something as simple as iscsi-target, but I was definitely wrong.  Add on top of that, a crash course in hacking vCloud. That's right, today I get to migrate VM's under vCloud using the REST API's. Tons more details at virtuallyghetto, but the highlight is- make sure your datastores are visible to vCloud.

Inside vCloud, you have to do all your management using vCloud Director- a web based headache, but it has it's purposes. To add your new datastores, first you have to add them the usual way using vmware's viclient or via the web console- either will work fine. Next, you have to log in via the web to your vCloud Director- not fun if you've not done this before.

You'll have to login as a system administrator for your vCloud, and navigate to "manage & monitor", find "Provider vDCs" and then select the right provider. In here, you'll add your datastore.

Not quite brain surgery, but nerve wracking when you're convinced it's going to explode on you at a moment's notice.

Saturday, August 24, 2013

VMWare DataCenters and My stupidity == learning new trick

So I decided in a demo environment that I would try to move an ADDC from one datacenter to another- without realizing that you can't. Oh sure, there's a slightly convoluted way of doing so via off-lining the VM, and doing a cold migration between datacenters, but as I found out very quickly- that can be filled with some very nasty gotchas.

Some background on this VM- it's just the test auth for a proof of concept (POC) system, so it does double duty as the RAS host as well. VPN in, do my work, happy clam. Note- this box is the VPN server.

Did I mention it's the VPN server yet? Very important point, and one that makes me damn glad I had an out of bandwidth management setup.

Anyway, I start the preparations to move this VM by... shutting down the VM. Makes sense, it's a cold migration right? Yeah, guess who winds up disconnected? So without a hint of panic, I try to login to the host itself, thinking that AD going offline just popped VCenter. Can't reach the host- very quickly realize I can't reach any of the hosts. Now the panic briefly pops in, however it doesn't last long as I realize I can just log into the console.

And discover another mild gotcha.

The esxi shell is disabled by default.

Mind you, I've not played with 5.1 under the hood that much, so a quick tour of google and I find out how to access the shell once again- F2 at the console, login and navigate to "Troubleshooting Mode Options", enable the shell, and exit out. Finally, I've got shell!

So I login, execute quick vim-cmd vmsvc/getallvms in order to locate the vmid of the vm in question, followed by vim-cmd vmsvc/power.on . No real panic as yet, but that's because I figured I'm already fired anyway, how much worse can it get? Less than 5 minutes later, I'm able to login to the VPN and restart my connection to VCenter.

When I notice the error logs make no mention of VCenter ever having an issue. The only issue on the whole system was the fact that the esxi shell had been enabled on one of the hosts. Then it dawns on me- my original thought of short downtime not having any effect on the network was correct, however in my raging stupidity I'd forgotten where the vpn lived. That issue gets fixed today.

However, in looking around, I had an "AH HA!" moment that I needed to test (in my homelab... even if it's a POC, it's not for my tests- just the customer's). What if I join the new host to the current datacenter, do a live migration to the new host, remove the host and rejoin it to the correct datacenter? One quick test with a host running one half of my local AD (Yes, in the home test lab I run two dc's minimum... I hate rebuilding them.That'll teach me to cut corners).

Removing the host from the datacenter does not delete the VMs on that host. Joining a host with VMs doesn't delete the VMs (but I already knew that)- So what's to keep this from working? As near as I can tell nothing- just make sure the VMs you are trying to move aren't in a cluster, as the cluster will more than likely try to bring them back after you've removed the host.

In conclusion- if you can remove and join hosts from the vmware datacenter, you can do a live migration of VMs between datacenters.It just takes a bit of forethought and planning.

And being smart enough to remember where your VPN lives.

Wednesday, July 31, 2013

It's been awhile again, I know. Bad poster!

And man have I been busy! The list of items is monstrous indeed- everything from vmware 5.1, server 2012 to NFS. I've been learning user administration (again! seems it changes every few years, go figure), more powershell, more exchange, and getting to play with great toys, like synology's RS2212+ (it's a thing of beauty, and I will be writing about it, and possibly buying one for the house), as well as redeveloping mass vmware deployments of server 2008 (I know, old os, but I had to deploy over 160 VMs in a single weekend)

I'll be detailing everything over the coming weeks, provided I don't get swamped again.

Monday, October 8, 2012

ESEUTIL, Unitrends UEB and my own stupidity

So I just recently got a copy of the Unitrends Enterprise Backup appliance for vmware installed and licensed over the weekend. Now me being me, I just had to try and dive in head first. Couldn't get a lot of things working, but I did get exchange backups working- which is a massive plus in my book.

So, verified the backup, and it cleared my exchange logs- yay! Talk about making me happy. Until a nagging voice at the back of my mind reminded me of recent experiences with backup exec (and the fact that there's no corresponding restore exec). A restore was needed! So I went through the process- but I wanted to restore only a single mailbox. Talk about a nail biter- as I'm going through the options, I created the restore share by selecting my mail db backup, selecting the option of "Next (Select Files/Items)" and...
Waiting. 16GB db plus about 30GB in log files- takes awhile. Cool thing is, this restores to a local samba share, which you then access from your exchange box.

Which is where the stupid on my part comes in- you'll see it soon.
So, I check the db and start trying to bring it to a clean shutdown state- (for those of you who don't know how to do this, check out this blog: ExchangeServerPro, excellent write up!) when I started running into trouble- my db checked the way I expected it to, my log files were good, however during the recovery phase...

Operation terminated with error -1032 (JET_errFileAccessDenied, Cannot access file, the file is locked or in use)

And it doesn't tell me which file. Of course, since I'm such a genius, I decide that it must mean the db is locked... nope, db's not locked.

Oh! right, the samba share is read-only! Why didn't I realize this sooner?
So I move the db to a directory I've got read/write access to, and it's all good, right?

Oh hell no.

Operation terminated with error -1032 (JET_errFileAccessDenied, Cannot access fi le, the file is locked or in use) exchange 2010

Now, I get a bit irritated and throw handle and process explorer (both from sysinternals, good stuff!) only to find out that db is never locked. In fact, the restore process gets to ~90% before failing without ever opening a log file or the db!

WTF?

Well, turns out that the log directory needs to be read/write accessible too... something I would've figured out almost 10 minutes sooner had I just checked the event logs for the ESE errors...

So, while the samba mount is cool, it's not as useful as I had hoped. But it does make it easier to get at the files and use the standard microsoft tools for manipulating the files (powershell, robocopy, xcopy and all the rest)

Problem solved, files copying, face red with shame.
Hopefully this sheds a bit of light on the problem in case anyone else is as dense as I can be.

Friday, October 5, 2012

NFS in android- why isn't this built in?

This seems like such a no brainer- right now using fileshares with android devices tends to be fairly painful, even with utilities such as astro, es file explorer, etc.

Why? Because none of those let you actually mount filesystems.

You want to view/use a file on a network share, you have to copy it to your device, work on it and then copy it back. That's such a waste, especially considering how long linux has had access to smbmount, nfsmount and the like. And they are so easy to setup. I mean, incredibly easy. But yet for an android device, this is a nightmare of rooting, installing apps (not that I've got anything against CIFS manager mind you, except that you've got to root your device to use it, and it's got to be running CM7), and running a gauntlet of compatibility issues.

NFS/SMB read capabilities are right there at the OS level, literally compiling in NFS support for the kernel is a check box or switch during compile time, and it takes pretty much no effort to supply. Not giving us this ability means that device storage built into our devices is getting wasted needlessly. Filesharing is a common every day solution for a lot of things, and even in the home user market no less. Network attached storage is even an option now on some routers- so the technology is being made easily available.

By including this is in the stock android deployments, we could get much easier use from our devices, and it would be yet another step to making BYOD less of a headache for system's administrators- ie, the data your users are using is still locked on the fileserver, and not on their devices.

Sunday, September 9, 2012

Equallogic SAN + VMWare ESXi 5.0 == annoying at times

I know a lot of you will laugh at me for this, but I'm hoping at least a few people find this useful. If you've got monitoring setup correctly on your EQL boxes (and you are running at least 5.0.5 firmware), you might be getting emails like this:
-----


ERROR event from storage array DABIGDISK
subsystem: MgmtExec
    event: 7.4.3
     time: Sat Sep  8 23:37:04 2012

iSCSI login to target 'xxx.xxx.xxx.xxx:3260, iqn.2001-05.com.equallogic:0-8a0906-19f16ee02-6040000003b487d0-vss-control' from initiator 'xxx.xxx.xxx.xxx:58885, iqn.1998-01.com.vmware:FEEKYFEEKY-07e33139' failed for the following reason:
                Initiator tried to bypass the security phase but we cannot.

-----
yup, sorry vmware, no biscuit.
It's an error, and it's kind of a stupid one, but it's not vmware's fault, it's actually a slight flaw (or duh, actually) on the equallogic side. There's two settings that need to be correctly configured, one of which is under Group Discovery -> iSCSI (refer to pic)



As you can see, it's a pretty simple check box. The other location is under VDS/VSS, in which case you simply don't give your vmware boxes permissions to use VDS/VSS unless you've already got the means in place to support it. 

Simple, stupid fix- and I was stupid enough to not even notice it until I actually ran into the headache tonight.