A few months ago, someone asked me what I did – that is to say, what sort of work encompasses a “typical day” for me as a long-term, on-site consultant. In response, I noted everything I did into a notepad++ file to actually see for myself (because it’s very easy to get lost in the work to pay attention to each little thing). So here’s the result: a typical workday for Mr. Non-Descript (sanitized to protect the innocent!). If there’s enough interest in this type of post, I may do it more often (comments are always welcome!).
6:05a - That silly alarm – but wait, it’s a WFH day (hit the snooze again)!!
6:40a - Ok, get up, get coffee… kiss the wife, pet the dogs, and go to the basement.
6:45a - Fire up the laptop, kick open the VPN, and login.
7:05a - Launch e-mail and start scanning new messages by color codes
- red for “from a manager”
- blue if directly to me in the To: or CC: fields
- green for anything matching active project numbers
It’s a sea of blue e-mails… nice.
7:30a - Realize I forgot to log into the corporate instant messaging; log in.
7:32a - Get an IM from a BP regarding IP problem (not kidding, 2 minutes).
Some work was done on their virtual server and now it doesn’t talk on the network. Ok, what work was it? Patches! (at least it wasn’t “nothing changed” again). Start getting a history – the servers had different IPs for three weeks and the analyst just noticed. Ok, so they were “inaccessible” for three weeks? Must be important servers I suppose.
About 10min later I had all the info and log histories that showed how they were migrated between clusters, moved across VLANs, and re-IP’d. In short, they were configured how they were supposed to be, DNS was good (A and PTR records), VMs were actually working, and there was no real problem. Note to caller: please use FQDN instead of IP to connect to your servers.
8:00a - more coffee… now…
8:05a - Started looking at a work request to update security for a vCenter role – and get interrupted.
8:15a - Somewhat frantic IM from a coworker about the storage team zeroing out some LUNs.
- check if any VMs are affected? None (whew).
- how many LUNs show on a host -> Configuration tab -> Storage Adapters -> vmhba (FC1 and 2)? None
- suppress alerts for now and put the vSphere hosts in Maintenance Mode.
- contact SAN to bounce the fabric ports to the hosts’ storage back online… and get interrupted via IM again.
8:30a - IM from a person in a meeting needing a quick answer about SSO design for disaster recovery.
- explain current design and operations
- explain the difference between failover and recovery
- explore benefits of having independent infrastructure components for operating a site for disaster (don’t rely on outside locations for recovery – have DR-located DNS boxes, vCenter, load servers, etc.)
8:55a - back to the SAN issue
- checked the current state after port bounce on fabric – it seemed to work!
- Hmmm… LUN counts are not balanced across the two HBAs. 34 on one, 37 on the other… should be 37/37.
- Something is messed up on the storage side. Let SAN people know (I suspect a few devices on the array are mapped to the wrong fabric).
9:30a - IM about problems deploying VMs with RDMs – it “just won’t work.”
- Look at the VM config and then the “Add Storage” on the host for available LUNs – there it is.. a 2TB device that matches the NAA they’re attempting to configure on the guest.
- We are not yet using VMFS5 due to the known “heap bug” so we can’t create RDMs > 1.99TB.
- Have analyst re-request the device to be 2047GB instead of 2048GB from the SAN team.
10:00a - Lost track of the day already – notes are fuzzy.
10:15a - Get more coffee (no, really… i can stop anytime i want!)
10:20a - read more e-mails and respond to some.
11:00a - lunch at my basement office (a.k.a. keep working on e-mails and review daily PMRB report).
12:15p – IM on capacity planning and utilization questions vs industry avg. This actually took a while to talk about since “utilization” can mean different things to different people. Much of the time was figuring out what the requestors really wanted to know.
1:00p - Planning this year’s workload and prioritizations.
- Prep for a meeting to discuss upcoming work efforts and their priorities to the environment
- Look at proper placement of the work and what we could leverage our development guys to script vs. manually execute.
- Debate top “x” number of initiatives to adopt and hold the rest for later.
2:00pm - start reading an e-mail chain about as long as a book on how a vendor thinks virtualization is the reason their app is behaving poorly. Start gathering information and get hooked into an on-going facilitated session on root-cause analysis of the issue.
3:00pm - Hear back from SAN team about the “8:55a issue” – it was as I expected: some devices were sourced from the array to only one fabric. It has been fixed and I checked out the hosts.
3:30pm - done working for the day.
3:30pm to 6:00pm - relax/downtime.
6:00pm to about 8:30pm - study for next certifications (both VCAP5-DCD and EMC Storage Foundations).
Play some games, watch some YouTube, go to bed.