Cyber Security Awareness Month - Day 19 - Remote User VPN Access ? Are things getting too easy, or too hard?
It seems lately to me that in IT we no longer seem to have downtime, even in traditional "9 to 5" companies. Laptops, smartphones, iPads and every other gadget out there all are internet connected, and more and more people seem to be online every waking moment. And if they’re online, chances are they’re VPN’d in to keep tabs on things at work while they’re surfing social sites, playing flash games or whatever. This is especially true now that VPN access is so easy, in fact it's now included in a number of smart phones and tablets.
Which brings us to the poor folks in IT. Since everyone is online 24-7, and we’re seeing business sales offices or business partners from 12 timezones over with VPN connections in, this brings up a whole raft of problems:
When exactly can we do system maintenance? I’m tired of waking up at oh-dark-early, only to find 6 users logged that you need to track down before you can start an upgrade. You can’t seem to pick any time as a maintenance window without causing someone a problem. Who gets access to what. All too often people have skipped over the data classification and server zoning steps. Without those done, just exactly what is that business partner allowed to have when they’re VPN’d in?
The prevalence of cheap laptops, tablets, phones and electronic doo-dads, all with internet access and VPN access (especially now that we have SSL VPNs) seriously starts to blur the line as to what the corporate desktop is. Worse yet, it blurs the line over who has bought and paid for that corporate desktop. No matter what our policies say, we have way too many personally owned devices out there that have VPN access to corporate resources, but don’t have corporate security tools, logging or, well, anything else. But you can bet they’ve got malware on them from the kids in the family ! (or the grown-up kids). And just exactly how do you enforce a VPN policy and deny access to someone who wants to work after hours for free? It’s a real challenge to make that point to a senior manager.
We’d really like to hear about any challenges you have faced on the topic of VPN access, and how you have solved them. Even if in your view you lost the battle on one issue or another, please share – someone else may have a different approach that might help you out. As always – our comment form stands ready to field any and all comments, questions and answers !
=============== Rob VandenBrink Metafore
Comments
Likewise, having a pile of users on at 2am is not an issue, provided that they were given notice that "2am is MY time." Personally, we generally don't do 2am maint - we use daytime maint. outages to exercise various disaster recovery facets. Aside from file-shares (where dumping a server would cost someone a bunch of edits), half the time we don't announce a darned thing - just take the server away, and see how people (and systems, and human processes) react. If all goes well, no customer will notice. And with mission crit systems... most employees won't notice, either.
S
Oct 19th 2010
1 decade ago
I can understand your thinking, but at many organizations that kind of IT BS just doesn't fly, period. sounds like you work in a very IT friendly environment.
you wrote:
>>"Your maint. windows do not actually need to be affected by VPN - any more than your carriers allow you to affect theirs."<<
here, we are not even allowed to update DNS entries without a change request form being filled out and a time window for said change to be submitted for approval. It must be nice to be able to just tell your users "tough luck". we are not allowed to do that here.
in our situation I have found the best solution is to work within the confines of our system and in doing so I have gotten upper management to agree that a time window with 48 hours notice is generally sufficient to prepare end users for any downtime they might experience due to the IT change request.
we submit the request which states how long the planned operation will take, which business units will be affected, and to what degree. details a backout plan in case the operation fails and a few other minor informational items. this gives our management and users exactly the information they need to understand what will be down and when it will be down. it is subject to management approval. if some departments are too busy we are asked to schedule a different time.
sometimes frustrating, but overall, working WITH our users and getting them to work WITH us has really been a huge benefit to everyone involved in any IT changes or upgrades.
husaragi
Oct 19th 2010
1 decade ago
You're being a tad myopic. This is a life safety environment. We spend a pile of effort to make sure things work when other things fail. This has nothing to do with "IT Friendly". It is all about "Process Integrity".
Last I checked, reality doesn't care about "convenience". Servers fail, PRIs get cut, and old ladies have strokes while driving cars, and drive those cars through the wall of your building, into your switch room. They do not schedule these events, either.
And somewhere in that reality, if you're going to label something "mission critical", and call it "High Avail", then you really need to live up to that expectation. And that means users need to live up to that expectation, too. I think the error in your perception is that you expect some major service or process interruption during these maint. windows. You're mistaken; as I said, for the easy stuff, the customer doesn't notice; for the important stuff, the user doesn't notice, either.
Repeat after me: "Process Integrity". And then repeat the 2nd mantra: "A DR process must be exercised."
You cannot get more "working WITH the users" than that. The IT systems exist for the sake of the Process, not the users. And guess what - those users exist for the sake of the Process, too. If the user cannot maintain Process Integrity during an outage of some sort? Then they are as defective as anything else.
And as I said, you cannot get more "working WITH users" than that.
S
Oct 20th 2010
1 decade ago
the issue here is not what happens when there is an UNEXPECTED emergent situation with your IT infrastructure. the issue being discussed is PLANNED maintenance and the availability of systems 24/7 because thats how people want to work...not because you promised high availability.
you've completely confused the issues and crossed into an entirely different realm when you start talking about disasters like a car driving into the switch room. where did you get that?.
there is no error in perception here. sometimes there ARE major interruptions when doing IT upgrades...you have to know that. as I said...IT friendly...your IT people appear to have much more say in what gets worked and when (provided its not an emergency) than we do here.
i dont mean to make this an argument but you seem to think you can just cookie cut some IT solution and plug it into any company.
all I am getting at here is that we can not just take down a server for maintenance whenever we want simply because we informed the users we were going to do so. it must be nice to be able to do that since its apparently what you do where you work. bring that crap over here and you wont last 2 weeks ok? your system WONT work here in this organization. kk thanks for playing.
husaragi
Oct 20th 2010
1 decade ago
Nick M
Oct 20th 2010
1 decade ago
husaragi
Oct 20th 2010
1 decade ago
husaragi
Oct 20th 2010
1 decade ago
I got that story from a call I was on. After we finished cutting her out of the car, I noticed that there wasn't a single TBU mixed in with the pile of fine Dell logos that we were standing on. I then pawed through many layers of crap to find anything to salvage before we foamed over the gasoline that was soaking into the everything. Go figure, the guy had a dozen Raid5 arrays, so he "didn't need a backup." He had no plan for those arrays being scattered across the floor by a car moving at 30mph, or the building catching fire, or the pipes feeding the toilet on the next floor up being sheared (and I mean *all* the pipes). Too bad - it was the end of his business, and eventually had to sell his house as well. But hey! No time for downtime, users needed it 24/7! Website Too Critical! Users! kk thx for playing! Look at me! Too Critical! Users more important than the GOAL we're trying to achieve!!
I should point out that our 600 call per year district handles about 3 car-buildings per year. Bigger districts will do more, they are considered a routine call. The fact that you have no concept of this? Is significant, and you should consider what that fact means.
As far as confusing issues - no. Outage is outage. Doesn't matter if by car or by keyboard - the only difference is the scope of impact, level of pre-plan, and response.
Losing a single server should not create a crisis. If it does, then that setup isn't adequate for the risks involved. Period. q.v. "single point of failure". If you choose to hedge on a failure not taking place... you're gaming the odds, which means cost (or more often, lack of engineering skill) is more important than uptime. Period. When you eventually lose the bet? it cannot be a crisis - you accepted the risk, earlier. Period.
Because really - if uptime actually matters, your first priority is to make it possible. If you can't, then you hire someone who can. And if you can't afford them, THEN you consider taking risks. Right? But at that point, "uptime" is no longer the top dog - "Risk" is.
So as I said - in an environment where process integrity actually matters, maint. windows are trivial. It's almost as if we'd planned for them, or something, instead of reacting to them AFTER we turned everything on.
Another error - there should be no such thing as an UNEXPECTED emergent situation. We use words like "eventualty" in this field. Learn that word. It forms the backbone of proper computer and network and software and process engineering. And once you learn that word, you discover that "Reboot Wednesday" is Yet Another Eventuality, just like Vacations, Earthquakes, Death, and Taxes. You can even accomplish such goals as rebooting a server without people noticing, it's amazing!
With the "eventuality" culture comes the attitude that Excuses are not Results. If your stuff is so important that it must not go away... yet it CAN go away? You're doing it wrong. And seriously - nobody cares why. If it's that important? Hmm... Excuses... or Results. Pick one.
Since your users will kill you if you dump a single box... well, we can guess that you're not after Results as far as Uptime is concerned. Kind of makes you wonder how Google manages all of their machines, cheaply, without asking your permission before they maintain them. Magic!
Utopia starts with you, figuring out what your priorities are, and not confusing them (uptime vs hedge as top priority, for example.) And as I said - playing the hedge is a totally legitimate tactic - but it is not about uptime, it is about odds. And once your goals are clear, you can actually get Results.
S
Oct 20th 2010
1 decade ago
seriously. i KNOW all the stuff youre saying.
now...bring all that over to my CEO and see how fare you get with any of it.
the issue you are not considering here...***the decisions are not MINE to make***
if they were then your arguments might be valid. so...i say once again. enjoy your utopia because many of us dont get to live in that wonderful little world of yours.
eventually...unexpected....semantics. give me a break. because you use the work eveutally rather than the word unexpected its somehow a different situation? wow.
husaragi
Oct 21st 2010
1 decade ago
The fact that you have no concept of this? Is significant, and you should consider what that fact means.
really? you know me? you know something...ANYTHING about what i have an understanding of? you know where i work and how large my organization is? what we do? what we deal with on an annual basis as far as calls go? hrm...you must be good.
and what does your example of the car have to do with anything? again...youre all up in arms about redundancy and backups and what??? thats not even what we are talking about.
"the only difference is the scope of impact, level of pre-plan, and response."
gosh....the level of pre-plan, scope of impact, and response....IS what makes those situations different...wow.
you're just making way too many assumptions based on knowledge about our situation at this company and my level of expertise that you know nothing about.
husaragi
Oct 21st 2010
1 decade ago