Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I've seen a good number that do once the org makes it a priority and allowed money to be spent on learning and experimenting.

From there, some people still want to stick to their old knowledge, but chances are you still will need that. Only start ups are "easily" going 100% Serverless. And even then many still want a few 24/7 servers. So there's still a job for older ops.

Ops people tend to be slower because brand spanking new tends to be over hyped and the brand new thing was designed with a goal for ease of use and performance. Security will be added on perhaps next year. Systems manager is cool, but if you already need to use something else to orchestrate, it's often easier to use that and keep a smaller toolset.



It’s not about going serverless, it’s about how to automate the management of servers. When my manager first came, the one ops guy we had was manually going through a list of EC2 instances and patching them. That’s an old mindset.

We have a third party web application that was on one server and when it went down, they would get an alert and reboot it. That was an easy win - put it behind a load balancer and an autoscale group with a minimum and maximum of 2 and http health checks so it would just kill the instance and bring a new one up.

This same guy saw a lot of EC2 instances that he didn’t recognize come up in the middle of the night and wanted them to follow our naming conventions APP0x and wanted to know the IP addresses. It took me forever to get him to understand that they were ephemeral, were part of an autoscaling group based on a queue, and that he couldn’t put whatever old school alerting system on them and they were all tagged with the corresponding autoscaling group and environment. The best I could tell him was the subnet that they would be launched in and the corresponding CIDR block.


Just curious - if the ephemeral instances are caused not by regular usage, but a mining botnet, will you have the tools to detect what is wrong (before the bills skyrocket) and mitigate?


That’s interesting. I guess we could set up something that used a combination of CloudTrail/CloudWatch Event/lambda that monitored any EC2 instances that are launched without the required tags and alert someone.

We do have billing alerts set up already that would warn us something went wrong we would have to investigate to find out what.

Better alerting and monitoring is one of our goals this year.

Heck I’m still finding places with hard coded keys in programs instead of letting the SDKs get the keys from the default configuration file (development) or from the instance role when they are run on EC2/lambda. Unfortunately all of the examples do it and the developers didn’t know any better.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: