@Makyen et al: I've (re) added the rules regarding Metasmoke, CPU usage, and RAM usage to the handling system. If CPU pegs again for a long period of time, the hypervisor system is being ordered to reboot the VM. This will force it to, essentially, asplode whatever's going on, and force restart the entire system. AND trigger alarm notices ;) If CPU is pegged at 90%+ for more than 10 minutes, the hypervisor force-reboots the guest OS.
@Makyen standard VMware alarm is 90%+ for 5 minutes before it triggers a critical alarm state.
and by 'peg' i mean it stays at 90% for more than 5 minutes
it warns at 75%+ for >= 5min but doesn't trigger anything
TBH I don't think anything on Metasmoke should really be pegging the CPU for more than 5 minutes, even during the initial setup process and MS deployments it never holds for that long
adjusted the reboot condition for 90% for more than 10 minutes
OK. Thanks for telling me. I don't know what MS' CPU usage looks like under normal, or moderately stressed, conditions, so really can't comment, other than that I trust your judgement. Admittedly, 10 minutes sounds like it's substantially less likely to trigger at a time which isn't a problem than 5 minutes.
@Xnero As an example, we've had regexes in the list which normally take milliseconds to run, or less, but which have ballooned to taking tens of minutes to complete when they happen to be run against specific posts. In the past, this has resulted in taking down SD instances, even, progressively, all instances. At this point, the CI testing includes more input cases, which would at least fail for those specific regexes, and ones which might be somewhat similar.
i upgraded and replaced the VCenter system that manages the ESX cluster
so the alarms that existed previously went poof
so I just readded these alarms and trigger handlers.
(and if it fails to gracefully shut down the guest, it goes to "Hard Reset" which is the equivalent of unplugging and replugging a frozen computer, so it hard powers off and reboots)
That, and unfortunate recent per core pricing change keeps me away; but the snapshots saved me many many hours of work, and for that I remain appreciative.
@SmokeDetector delete/undelete behavior, then vandalism to complain/whine/moan about things. User has been squished for this behavior now, but has also gotten a permanent content lock on that post to prevent vandalism.
I was all set to buy this month, then they announced several huge changes (DDR5, CXL, maybe PCIe 6, 3D cache stacking, new chassis design, E3 EDSFF); and FOMO (fact of missing out) put my plans on hold - for repentance I've pledged to add $100/month to my budget.
@Rob There will always be new things on the horizon. :) You have to pick the point where you either need something or are willing to have what's currently available, with the understanding that it will be outdated/superseded by additional technology advances in hte not too distant future.
@Makyen True, but I was going to purchase this month (that was the plan several months ago, no more waiting) since it looked like most of the vulnerabilities in the CPU had been addressed and PCIe 4 was good enough. But now it's a case of what's to come won't plug in to the old motherboard (except for PCIe, it's still backwards compatible). I like to keep my equipment several years, before I retire it to a lesser position, so buying a dinosaur this year doesn't seem like a good investment.
I can't afford to buy evey three years. I used to be able to because the new computer would be 2-3x faster for little more than half the price; but those days are long gone, now it's more like 2x for approximately the same price.
